Commit Graph

28 Commits

Author SHA1 Message Date
0c9d44e9c9 Legal texts for dorfteich.online; legal template covers the newer processing
All checks were successful
CD / Build and push images (push) Successful in 1m18s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m20s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 4m7s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m37s
CI / Import/export fidelity gate (push) Successful in 47s
- deploy/legal/dorfteich-online-{impressum,datenschutz}.md: ready-to-
  paste Markdown for the flagship instance (Admin → Legal pages after
  the wizard) — operator standard texts (§5 DDG, §18(2) MStV, VSBG
  no-participation, UGC/liability/copyright notices) plus a privacy
  policy grounded in what THIS instance actually does: Hetzner hosting
  with DPA, session cookie only (no banner), rate-limit IPs, proxy logs
  ≤30d, transactional + digest mail with unsubscribe, plaintext content
  with versions/comments and public pages, hashed API tokens with
  audited writes, nightly backups incl. the encrypted-tunnel mirror to
  the operator's private server, self-service export,
  deletion/pseudonymization, LfDI BaWü.
- docs/self-hosting/legal-template.md: review checklist and both
  language templates extended for the processing added since #82 —
  comments/version history, notifications/digest mails, API tokens,
  off-host backup copies.

No legal advice; texts follow the operator's standard building blocks
and should get a final human read before publishing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 19:00:05 +02:00
6d710caa50 Go-live checklist: prod mirror verified live after the v0.2.0 deploy
All checks were successful
CD / Build and push images (push) Successful in 1m8s
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m11s
CD / Promote to Int (push) Successful in 10s
CI / Lint, typecheck, test (push) Successful in 4m3s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m34s
CI / Import/export fidelity gate (push) Successful in 47s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 15:54:24 +02:00
203f7c98a5 Tick the go-live mirror item: BASEL mirror live on Test, Prod prepared (#84)
All checks were successful
CD / Build and push images (push) Successful in 1m10s
CD / Deploy to Test (push) Successful in 10s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 10s
CI / Lint, typecheck, test (push) Successful in 4m2s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m36s
CI / Import/export fidelity gate (push) Successful in 46s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 12:33:00 +02:00
52192eb05f Backup mirror to BASEL: rsync of the sets after every successful run (#84)
All checks were successful
CD / Build and push images (push) Successful in 3m51s
CI / Lint, typecheck, test (push) Successful in 4m5s
CD / Deploy to Test (push) Successful in 11s
CI / Build container images (push) Has been skipped
CD / Smoke tests against Test (push) Successful in 1m11s
CD / Promote to Int (push) Successful in 12s
CI / Auth e2e pack (push) Successful in 5m52s
CI / Import/export fidelity gate (push) Successful in 47s
The operator-level extra beside the admin-configured Nextcloud target
(#103), unblocked now that the ONE→BASEL tunnel is stable again.

- sidecar: optional mirror step (mirror.ts) driven purely by env —
  BACKUP_MIRROR_TARGET (rsync-over-ssh), BACKUP_MIRROR_SSH_KEY (private
  key on the secrets volume, never in image or repo),
  BACKUP_MIRROR_SSH_PORT. Runs after the prune of every successful run,
  so --delete aligns the remote retention with the local one (the
  newest-complete-set guarantee carries over). Only set files travel
  (db-*.dump, files-*.tar.gz); status files and bundles stay local.
  Host key pinned via accept-new into .mirror_known_hosts on the backups
  volume; fixed remote modes (dirs 750, files 640, symbolic --chmod —
  octal needs rsync ≥ 3, macOS dev machines ship 2.6.9). rsync +
  openssh-client added to the sidecar image.
- status: additive `mirror` block in status.json (outcome, transferred
  count, lastSuccessAt carried across failures) — shown on the admin
  backup card; failures alert via a new backupMirrorFailed mail (de+en)
  while the local run still counts as succeeded.
- deploy/backup-basel.md: complete BASEL-side walkthrough — dedicated
  user dorfteich-backup with a /home/ home and a bash login shell,
  explicitly avoiding the Debian backup-user (UID 34) pitfalls
  (nologin shell rejects rsync sessions, /var/backups home), key
  placement through the api container onto the secrets volume, .env
  values, on-demand verification.
- tests: rsync-arg/stats-parsing units plus an integration suite against
  the real rsync binary (local target; skips where rsync is absent) —
  transfer, idempotent re-run (0 files), retention alignment, failure
  path carrying lastSuccessAt.

Verified live against the real BASEL host from a native sidecar run:
initial transfer, host-key pinning, retention alignment after a local
prune, idempotency, and the failure path (surfaced in status.json while
the local run stayed green). BASEL side provisioned per the doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 12:20:32 +02:00
5cef359b8f Nextcloud backup target: admin-configured, manual + scheduled uploads, in-app restore (#103)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m45s
CD / Build and push images (push) Successful in 3m49s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m35s
CI / Import/export fidelity gate (push) Successful in 47s
Off-host backups for every self-hoster, configured entirely in the admin
UI — supersedes the host-specific mirror plan behind #84.

shared:
- webdav.ts (new package entry like token-crypto): minimal WebDAV client
  with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT
  (streamed), GET, DELETE; Nextcloud DAV path derived from the plain
  server URL, explicit DAV bases pass through
- backup-status.ts: additive remote-upload status in status.json, the
  restore-status.json contract (running/succeeded/failed + staleness
  bound), the backup_command/backup_maintenance NOTIFY channels, and the
  one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz)
- backup-set.ts moved here from apps/backup (api lists local sets)

backup sidecar:
- reads the backup.* instance settings directly from the database (admin
  changes apply next run; local retention row overrides the env) and the
  app password from the secret store
- after each successful set: bundle dump + files archive + manifest into
  ONE self-contained tar.gz, upload via WebDAV per schedule
  (off/daily/weekly; manual runs always upload), prune remote bundles —
  never the newest — and record the outcome in status.json; upload
  failures alert via a new backupUploadFailed mail (de+en)
- command listener on backup_command (run / restore) with a serial queue
  against the nightly timer
- restore orchestrator: restore-status.json → maintenance NOTIFY →
  grace → (remote: download + manifest-verify bundle) → terminate other
  DB connections → shared perform-restore path (same code as restore.sh)
  → final status + maintenance exit

api:
- MaintenanceGuard (global, registered before the setup gate): 503
  maintenance_mode while restore-status says running; health endpoints
  and the new public GET /backup/restore-status stay exempt; a stale
  running state (crashed sidecar) unblocks after 30 min
- MaintenanceStateService watches the file and restarts the api after a
  successful restore (fresh caches, migrate-on-start for older dumps);
  main.ts refuses to touch the database while a restore runs — a
  container restarting mid-restore must not race pg_restore with
  migrate deploy
- worker sweeps (conversion, mail outbox, scheduler) catch transient
  database failures instead of dying on an unhandled rejection — the
  restore's connection termination crashed the api in verification
- backup admin endpoints under /admin/system/backup: settings (live
  connection test before save, password write-only into the secret
  store), nextcloud/test, sets (local via the ro backups mount + remote
  via WebDAV), run + restore (type-to-confirm backstop, source
  validation) — commands travel as NOTIFY payloads; audit actions
  backup.settings_changed/run_triggered/restore_requested
- readyz: new warning-level backup_remote check while a target is
  configured (26 h daily / 170 h weekly bound)

collab:
- maintenance listener: on enter, persist + close every live session and
  refuse new connections until exit (failsafe timeout 30 min) — no
  in-memory document may write pre-restore content back afterwards

web:
- Admin → System backup section: status card with remote facts and a
  "Back up now" button, the Nextcloud settings form with test button,
  and the restore picker (local + remote sets, type-to-confirm)
- global maintenance screen: any 503 maintenance_mode flips the SPA to a
  status page polling the exempt endpoint, reloading when the instance
  returns

Verified end-to-end against a live stack (fresh DB, native api + sidecar,
fake WebDAV server): configure → test → manual backup → bundle upload →
readyz/sets/status surfaces → remote restore with maintenance gate,
marker rollback and api restart; suites: shared 21, backup 9, collab 11,
api 58 files green, lint + i18n:check + typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 10:39:18 +02:00
a9e901c449 Run operations QA on every release candidate before the prod gate (#90)
All checks were successful
Release / Build release images and notes (push) Successful in 1m8s
CD / Build and push images (push) Successful in 1m9s
CD / Deploy to Test (push) Successful in 9s
Release / Release-candidate operations QA (push) Successful in 41s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 10s
Prod deploy / Deploy the released images to Prod (push) Successful in 15s
CI / Lint, typecheck, test (push) Successful in 3m35s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m32s
CI / Import/export fidelity gate (push) Successful in 48s
New deploy/release-qa.sh, wired as the release workflow's second job: it
boots the PREVIOUS release with pre-seeded fixture content in a scratch
environment, swaps the api to the candidate against the same database
(migrations auto-apply, readiness green, content intact — the
seed_fixture/assert_fixture pair is the update-fixture contract future
migrations extend), asserts the degraded-readyz semantics on the
candidate (200 + warn-level checks without sidecars), and runs a full
backup/restore roundtrip with the candidate's sidecar into a second,
empty database. The wizard e2e already guards fresh installs in CI
(issue #81). Verified green on the host for v0.1.0→v0.1.1; a simulated
destructive migration made the suite fail loudly (negative test,
not committed). A human pushes the prod tag only when both release jobs
are green — the documented pre-approval gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 00:29:53 +02:00
28f05e270d Add the release pipeline with a tag-based manual gate and Prod stack (#89)
All checks were successful
CD / Build and push images (push) Successful in 1m5s
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m6s
CD / Promote to Int (push) Successful in 10s
CI / Lint, typecheck, test (push) Successful in 3m33s
CI / Build container images (push) Has been skipped
Release / Build release images and notes (push) Successful in 1m1s
Prod deploy / Deploy the released images to Prod (push) Successful in 14s
CI / Auth e2e pack (push) Successful in 5m31s
CI / Import/export fidelity gate (push) Successful in 47s
Pushing vX.Y.Z builds the four semver images and publishes a Gitea
release whose notes list the changes since the previous release with a
migration call-out derived from the migrations diff (the repo is
trunk-based — commit subjects stand in for PR titles). Deploying to Prod
is a separate human act: pushing prod-vX.Y.Z-<suffix> — Gitea 1.22 has
no environment approvals, so the tag push is the gate — verifies the
release images exist, pins TAG in the Prod .env, restarts the stack, and
waits for readiness; rollbacks are new suffix tags on the previous
release. The Prod stage is provisioned on ONE (ports 8120-8122, secrets
generated on the host, full backup profile); deploy/go-live.md carries
the executed mechanics and the operator checklist that blocks the DNS
switch (DNS, Caddy block, wizard/SMTP, legal texts, monitors, Prod
drill, BASEL mirror).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 00:17:13 +02:00
4b55fb92ac Write the self-hosting guide and add the optional caddy TLS profile (#88)
All checks were successful
CD / Build and push images (push) Successful in 1m7s
CD / Deploy to Test (push) Successful in 10s
CD / Smoke tests against Test (push) Successful in 1m8s
CD / Promote to Int (push) Successful in 10s
CI / Lint, typecheck, test (push) Successful in 3m13s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m18s
CI / Import/export fidelity gate (push) Successful in 46s
docs/self-hosting/README.md is the complete operator contract: install
from the two reference files, first-run wizard walkthrough, update
procedure with the one-release downgrade window, backup/restore with the
sidecar, readyz-based troubleshooting (incl. the classic proxy/WebSocket
and APP_BASE_URL/CSRF mistakes), and a build-from-source note; linked
from the repository README; English-only by documented decision. The
reference compose gains a `caddy` profile (new Caddyfile) that publishes
80/443 and terminates TLS via Let's Encrypt for $DOMAIN — localhost uses
Caddy's internal CA for smoke tests. deploy/self-hosting-verify.sh
scripts the clean-machine test: a fresh directory with only the
published files boots to the wizard answering over TLS, then removes
itself; verified green on the stage host.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 21:21:10 +02:00
d95c18e9e8 Automate the monthly restore drill with a scratch-stack workflow (#87)
All checks were successful
CD / Build and push images (push) Successful in 1m5s
CD / Deploy to Test (push) Successful in 10s
CD / Smoke tests against Test (push) Successful in 1m7s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 3m18s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m14s
CI / Import/export fidelity gate (push) Successful in 45s
New scheduled workflow (monthly + on demand) runs deploy/backup/drill.sh:
it reads the drilled stage's backups volume strictly read-only, restores
the latest successful set into a throwaway Postgres and volumes under a
unique drill prefix via the backup image's restore path, boots the api
against the result, and verifies readyz (database + migrations), row
counts, rendered content in the page cache, a public API request, and a
media byte-check against the attachments table — then tears everything
down, also on failure. Each run reports its outcome as a comment on the
pinned "Restore drills" issue (#98). docs/operations/restore-runbook.md
carries the manual procedure, which doubles as the Prod relocation path;
pre-go-live the drill restores the Test set (switch the source volume at
go-live, #89 — off-host fetch from the BASEL mirror stays with #84).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 20:58:06 +02:00
0ef96147e0 Extend readyz with backup freshness and a degraded status level (#85)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m11s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m44s
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 10s
CI / Auth e2e pack (push) Successful in 5m13s
CI / Import/export fidelity gate (push) Successful in 46s
readyz now enumerates database/migrations (hard failures, HTTP 503),
converter/renderer, and a new backup check that reads the sidecar's
status.json from the read-only backups mount and warns when the last
successful backup is older than 26 h. Warning-level checks surface as
overall status "degraded" while staying HTTP 200 — monitors alert on
the body keyword, Docker healthchecks keep using the liveness endpoints
so a degraded instance is never restart-looped. The status.json shape
moved to @dorfteich/shared as the contract between the sidecar and its
readers (#85/#86); deploy/monitoring.md defines the Uptime-Kuma monitor
set per stage. The api image also pre-creates /data/backups node-owned
so the shared backups volume stays writable for the sidecar regardless
of which container initializes it, and the sidecar's scheduler survives
runs that cannot even record their status.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 19:02:59 +02:00
8dbff86537 Add backup sidecar: nightly dump, volume archive, prune, status, restore (#83)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m9s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m47s
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 9s
CI / Auth e2e pack (push) Successful in 5m25s
CI / Import/export fidelity gate (push) Successful in 45s
New apps/backup service (ADR 0015): nightly pg_dump -Fc plus one tar of the
uploads/plugins volumes as a consistent restore set on a new backups volume,
retention prune that never removes the newest complete set, atomic
status.json for the readiness/admin consumers (#85/#86), and a failure mail
sent directly via nodemailer (the api may be the broken part) with de/en
texts in the shared mails catalog. BACKUP_RUN_ONCE=1 gives the on-demand
path; deploy/backup/restore.sh automates the documented restore runbook.
The pure secret-store helpers moved to @dorfteich/shared so the sidecar
resolves the wizard-written SMTP relay exactly like the api.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 17:29:15 +02:00
f0a82bad20 Add the first-run setup wizard API with env-backed secret store (#80)
Some checks failed
CD / Build and push images (push) Successful in 3m16s
CI / Lint, typecheck, test (push) Successful in 3m5s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Failing after 3m35s
CD / Promote to Int (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m6s
CI / Import/export fidelity gate (push) Successful in 43s
CI / Build container images (push) Has been skipped
When the api runs against a database without the setup.completedAt
marker, a global SetupGuard answers every non-exempt route with 503
setup_required; only /setup/*, health probes, and the session routes
stay reachable. The wizard steps (POST /setup/admin|instance|smtp|
registration|complete) write straight to their production homes; the
Site Admin step signs its creator in, later steps require that session.
Completing sets the marker and locks every step permanently (410, also
across restarts, and not reopenable via PATCH /admin/settings).

SMTP entered in the wizard is verified with a live delivery test first
(failure blocks the step with the transport error as detail) and then
persisted to the new env-backed secret store: a mode-600 dotenv file on
the new `secrets` volume (SECRETS_FILE). Explicit container env always
wins over the store; empty compose-passed strings count as unset. The
mail transport now resolves lazily through SmtpConfigService so wizard
changes apply without a restart.

SETUP_ADMIN_* env pre-seeds the whole wizard at boot for automated
deploys; a backfill migration marks instances that already have a Site
Admin as completed, and seed/vitest global-setup do the same for
fixture databases. The setup e2e suite provisions its own fresh
database (CREATE DATABASE + migrate deploy) per run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 15:10:28 +02:00
6e56403102 Document runner capacity increase in the provisioning log
All checks were successful
CD / Promote to Int (push) Successful in 9s
CI / Lint, typecheck, test (push) Successful in 2m48s
CI / Auth e2e pack (push) Successful in 3m46s
CI / Import/export fidelity gate (push) Successful in 43s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 1m0s
CD / Deploy to Test (push) Successful in 11s
CD / Smoke tests against Test (push) Successful in 1m3s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 08:33:29 +02:00
514cdf5ead Move stages, runner and deploy target to dedicated host ONE
All checks were successful
CD / Build and push images (push) Successful in 3m45s
CI / Lint, typecheck, test (push) Successful in 3m39s
CI / Auth e2e pack (push) Successful in 3m43s
CI / Import/export fidelity gate (push) Successful in 43s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m12s
CD / Promote to Int (push) Successful in 10s
Gitea itself relocated to the dedicated Hetzner server ONE
(one.101010.cloud); this commit moves the rest of the delivery chain
along: cd.yml now deploys to one.101010.cloud (DEPLOY_HOST_KEY secret
rotated to ONE's host key), the act_runner one-dorfteich replaces
vps-dorfteich, and the stage/deployment docs describe the new topology
(Test/Int migrated with volumes; Prod joins on the same host at M8).
The Int promotion gate variable RUN_INT_DEPLOY was removed — the new
host has the capacity, so promote-int deploys again by default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 08:05:36 +02:00
621aa47244 Add plugin storage, install API, and directory watcher (#71)
Some checks failed
CI / Auth e2e pack (push) Waiting to run
CI / Import/export fidelity gate (push) Waiting to run
CI / Build container images (push) Waiting to run
CD / Build and push images (push) Failing after 1m33s
CD / Deploy to Test (push) Has been skipped
CD / Smoke tests against Test (push) Has been skipped
CD / Promote to Int (push) Has been skipped
CI / Lint, typecheck, test (push) Has been cancelled
Backend for installing plugin ZIPs (ADR 0008, plugin-architecture.md
§Lifecycle, security.md §Plugins). Consumes the #70 SDK for validation.

- Schema: `plugins` (id, name, version, apiVersion, kind, mode, manifest
  jsonb, removedAt soft-delete) + `pond_plugins` (per-pond activation) +
  `PluginInstanceMode` enum; migration 20260710130000_plugins.
- `PluginPackageService`: pure, stateless ZIP → validated package via
  fflate — structure check, manifest validation (SDK), apiVersion gate,
  kind/bundle/styles rules, CSS sanitation (no @import / external url() /
  expression()), zip-slip and unpacked-size guards. Each failure carries a
  stable PluginErrorCode; manifest issues travel as ApiError details.
- `PluginStorageService`: on-disk layout `<PLUGINS_DIR>/<id>/<version>/`;
  atomic writeVersion (staging dir + rename, no 404 window mid-update),
  removeVersion/removePlugin, traversal-safe asset resolution, dropzone +
  quarantine dirs.
- `PluginsService`: install/update (update only to a strictly higher
  version, preserving the admin's instance mode; files land before the
  metadata pointer flips) / uninstall (refused while required; soft-delete
  + files removed + pond activations dropped) / list / get.
- `POST/GET/DELETE /admin/plugins` (SiteAdminGuard, multer memory upload),
  error→HTTP-status mapping. Public version-pinned static serving at
  `GET /plugins/:id/:version/*rest` with immutable cache + nosniff, only for
  the installed current version.
- `PluginWatcherService`: watches `<PLUGINS_DIR>/_dropzone/`, runs the same
  validation, installs valid drops and quarantines invalid ones with the
  error logged; inert under NODE_ENV=test (tests drive processDropped).
- SDK: `compareVersions`/`isHigherVersion`. shared: `PluginView`,
  `PluginInstanceMode`, `PLUGIN_ERROR_CODES`, `PLUGINS_DIR` env, plugin
  error i18n (de+en). Compose: `plugins` volume + `PLUGINS_DIR`.
- Tests: package unit test (valid + each invalid class) and an e2e DB test
  (GUI install + immutable serving, non-admin 403, invalid-manifest details,
  dropzone install + quarantine, atomic higher-only update, required-guarded
  uninstall that removes files and tombstones metadata).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-10 16:55:26 +02:00
8a68ef68e7 Add PDF export via Gotenberg (#67)
All checks were successful
CD / Build and push images (push) Successful in 4m3s
CI / Lint, typecheck, test (push) Successful in 3m5s
CI / Auth e2e pack (push) Successful in 4m7s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 12s
Server-side PDF export for reading/sharing (ADR 0009), rendered by a new
internal Gotenberg (headless Chromium) sidecar.

- Sidecar: `gotenberg/gotenberg:8` in the compose stack (internal, pinned,
  healthcheck); api `GOTENBERG_URL` env; a `renderer` readyz check at
  warning-level (mirrors the converter) so PDF export degrades gracefully when
  Gotenberg is down without failing readyz.
- Export HTML: `buildPdfHtml` renders a self-contained document (no app chrome)
  — the page's content with images inlined as data URIs, the pond's fonts
  inlined as base64 `@font-face` + applied via CSS variables (ADR 0016), print
  CSS (A4, page-break rules, a title header), and page numbers from Gotenberg's
  footer. Plugin-block fallbacks are a marked TODO(#79) for M7.
- Fonts in the api image: the api Dockerfile now bakes the font catalog in
  (`build-fonts.mjs` with FONTS_OUT) so the exporter can read a pond's chosen
  WOFF2 and inline them; a missing file falls back to the system stack.
- Job flow: `POST /pages/:id/export {format: pdf}` builds the HTML (read
  permission checked by the guard) and enqueues an `export_pdf` job on the #62
  queue with the HTML as input; the worker branches `to === 'pdf'` to the
  `GotenbergRenderer` (html → pdf) instead of pandoc, retrying an unreachable
  sidecar and failing a refused render (`renderer_unavailable`/`render_failed`,
  de+en). The client polls and downloads `GET /jobs/:id/result`.
- Frontend: the page-menu PDF button is now a real export (PDF added to
  EXPORT_FORMATS; the disabled placeholder removed).
- Tests: export.service.db PDF cases (HTML has title/font-variable/inlined
  image; renderer-down fails with `render_failed`); e2e PDF export self-skips
  without a Gotenberg sidecar (like the .docx case). Verified locally against
  real Gotenberg — a valid PDF with the pond font embedded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-10 12:11:12 +02:00
f500198c5d Add per-pond fonts: catalog, build, application, and admin UI (#66)
All checks were successful
CD / Build and push images (push) Successful in 3m24s
CI / Lint, typecheck, test (push) Successful in 3m6s
CI / Auth e2e pack (push) Successful in 4m8s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
Self-hosted Google Fonts with per-pond selection (ADR 0016), the GDPR
"zero external requests" posture (security.md, CSP `font-src 'self'`).

- Catalog: a curated 15-family OFL/Apache list in shared (family, weights,
  category, license, google-webfonts-helper id). `deploy/fonts/build-fonts.mjs`
  validates every entry has license info (fails the build otherwise),
  downloads the WOFF2 weights into apps/web/public/fonts/ (gitignored), and
  generates the @font-face stylesheet — run at image build time from the web
  Dockerfile (with retries), never from a visitor's browser.
- Application: PondFontScope sets --font-heading/body/mono (+ weights) from
  pond.settings.fonts on the editor + read view; the existing global CSS
  already reads those custom properties, so headings/body/code re-resolve to
  the pond's fonts. A pond with no settings arrives with the defaulted values
  (Roboto 400 / Roboto 200 / Fira Code), so the vision defaults always render.
- Admin UI: pond-settings 'Appearance' section — three slots (family + weight)
  with a live preview, Pond-Admin-gated (fonts added to updatePondInputSchema
  and merged in PondsService.update); a font catalog attribution page (/fonts)
  listing families and licenses. New `font` i18n namespace (de+en).
- CSP: strict Content-Security-Policy in nginx.conf (default-src 'self';
  font-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self'; …) —
  the app's scripts are all external files, inline styles cover CSS variables.
- Tests: shared catalog-integrity unit test (the invariant the build enforces);
  e2e fonts pack — no request leaves the origin when rendering a pond (the GDPR
  network assertion), a font choice applies to a page and persists, and a pond
  without settings renders the defaults.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-10 11:16:29 +02:00
4755c18ef5 Add conversion job queue and pandoc sidecar integration (#62)
All checks were successful
CD / Build and push images (push) Successful in 4m9s
CI / Lint, typecheck, test (push) Successful in 2m50s
CI / Auth e2e pack (push) Successful in 3m44s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m15s
CD / Promote to Int (push) Successful in 11s
Import/export conversions run asynchronously against an internal pandoc-server
sidecar with limits and graceful failure (ADR 0009). This is the plumbing;
the import (#63) and export (#65) features enqueue jobs onto it.

Sidecar & config:
- pandoc/core:3.6 in HTTP server mode added to the Compose stack, internal
  network only, with a wget healthcheck on /version; the api depends on it
  healthy and reaches it via the new PANDOC_URL env (default http://pandoc:3030).
- readyz gains a warning-level `converter` check: an unreachable sidecar
  degrades import/export but never flips the instance to unready (new `warn`
  status on ReadinessCheck).

Conversion flow (apps/api/src/import-export/):
- ConversionJob table (per-request work queue, distinct from the name-keyed
  maintenance Job table): owner, formats, input/result bytes, status, attempts,
  lockedAt. Migration + owner cascade.
- PandocConverter (abstract) + PandocServerConverter: POST / with
  {text,from,to,standalone}; binary input formats (docx/odt/…) are base64-encoded
  in `text`; 60 s AbortController timeout; input/output size caps. Failures map
  to distinct localized codes — converter_unavailable / converter_timeout
  (retryable) and conversion_failed (final).
- ConversionWorker: claims one job at a time with `FOR UPDATE SKIP LOCKED`
  (safe against overlapping sweeps and a second process), recovers a stale
  RUNNING lock, retries transient failures up to 3 attempts then fails. A 2 s
  sweep plus wake-on-enqueue means a queued job survives an API restart.
- ConversionJobService.enqueue (size-limited) + owner-scoped GET /jobs/:id
  (poll) and GET /jobs/:id/result (stream the output); a foreign/unknown id is
  404. ConversionJobView in @dorfteich/shared.

Tests:
- conversion-job.e2e.db.test.ts (fake converter injected via a new createTestApp
  override hook): enqueue→convert→poll→result; foreign/unknown job 404; a
  persisted PENDING job picked up by a fresh app's worker (restart survival);
  sidecar-down fails after 3 retries while the API stays healthy.
- pandoc.converter.test.ts: success, non-200→conversion_failed, refused→
  converter_unavailable, and a delay-injecting server→converter_timeout.
- Verified locally against a real pandoc/core:3.6 container: markdown→html,
  markdown→docx (valid PK/OOXML bytes), and a docx→markdown round-trip.

Local: typecheck, lint, i18n:check, build all green; api 193 tests
(9 new), shared 121, web 50.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-10 04:06:27 +02:00
d4ebcfcfbe Add collaboration token issuance and connection authentication (#34)
All checks were successful
CD / Build and push images (push) Successful in 2m45s
CI / Lint, typecheck, test (push) Successful in 1m56s
CI / Auth e2e pack (push) Successful in 2m1s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m9s
CD / Promote to Int (push) Successful in 12s
The api mints a short-lived (60 s) HS256 JWT per page open after an interim
permission check; the collab server authenticates every connection with it
(ADR 0003/0007 — the only JWTs in the system).

- packages/shared: browser-safe token schema/types in `collab-token`, and the
  Node `crypto` sign/verify in `token-crypto` behind its own subpath export
  (`@dorfteich/shared/token-crypto`) so the web bundle never pulls in
  `node:crypto`. Only HS256 is produced/accepted; the signature is checked in
  constant time before any untrusted field is read.
- api: `GET /pages/:id/collab-token` (auth-required) returns
  {token, mode, expiresInSeconds}; `mode` is rw/ro via the interim access
  service; issuance is logged at debug level without the token value.
- collab: `onAuthenticate` verifies the token, checks the pageId matches the
  document name, stores {userId, mode} context, and enforces `ro` via
  Hocuspocus' read-only connection flag. Hocuspocus' own signal handling is
  disabled so index.ts remains the single shutdown owner.
- Shared COLLAB_TOKEN_SECRET env for api + collab (compose, dev overlay,
  .env.example, stage docs); a dev default keeps native dev/test/CI running.

Tests: shared token round-trip/rejection; api endpoint e2e (auth required,
claims, 404 for non-members/unknown ids); collab integration via
HocuspocusProvider (valid token connects; expired/tampered/mismatched-page/
wrong-secret rejected; read-only writes dropped, verified with two clients).

Closes #34

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 15:52:19 +02:00
8316c617d2 Add collaboration server skeleton (Hocuspocus) with health, container, and CI/CD (#33)
All checks were successful
CD / Build and push images (push) Successful in 2m36s
CI / Lint, typecheck, test (push) Successful in 1m50s
CI / Auth e2e pack (push) Successful in 1m58s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
Bootstrap apps/collab as a Hocuspocus WebSocket server (ADR 0003):
- pino JSON logging (service=collab) and shared Zod env validation
  (collabEnvSchema); structured connection open/close logs.
- /healthz endpoint (process liveness + PostgreSQL ping) served via the
  onRequest hook, matching the container-internal path and the proxied
  /collab/healthz path; any WebSocket handshake is accepted for now
  (authentication arrives with #34, persistence with #35).
- Dockerfile (ESM workspace build) and a compose service on the frontend
  and internal networks with a healthcheck; dev overlay service and a new
  COLLAB_PORT variable.
- CD builds, pushes, and promotes the collab image; CI builds it on PRs;
  the smoke suite asserts /collab/healthz through the reverse proxy.
- deployment.md/stages.md: proxy routing, per-stage COLLAB_PORT, checklist.

Closes #33

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 14:53:44 +02:00
0fae699018 Add file storage service and image upload API (#27)
All checks were successful
CD / Build and push images (push) Successful in 2m2s
CI / Lint, typecheck, test (push) Successful in 1m43s
CI / Auth e2e pack (push) Successful in 1m48s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 10s
Implements the FileStorage abstraction (uploads/<pondId>/<fileId> on the
mounted volume), the attachments model, and POST /ponds/:id/files, GET
/media/:fileId, DELETE /files/:id. Uploads are validated by sniffing
magic bytes rather than trusting the client's Content-Type/filename
(catches a renamed .html-as-.png), checked against the max_file_bytes
and storage_bytes quotas, and served with nosniff + immutable caching.

Closes #27
2026-07-08 10:35:03 +02:00
4dd452c0af Docs: warn that stage re-seeds must pass the password overrides
All checks were successful
CD / Build and push images (push) Successful in 40s
CI / Lint, typecheck, test (push) Successful in 1m22s
CI / Auth e2e pack (push) Successful in 1m42s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m5s
CD / Promote to Int (push) Successful in 10s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UpQz6ypHJsLfMf4S6fyQEB
2026-07-05 21:03:43 +02:00
bef6d8e4dc Make stage mail + base URL configurable; seed password overrides
All checks were successful
CD / Build and push images (push) Successful in 1m49s
CI / Lint, typecheck, test (push) Successful in 1m15s
CI / Auth e2e pack (push) Successful in 1m41s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 7s
CD / Smoke tests against Test (push) Successful in 1m5s
CD / Promote to Int (push) Successful in 9s
- compose: pass APP_BASE_URL and SMTP_* through to the api container so
  stages can use a real relay (defaults still match the dev Mailpit
  overlay); document the new keys in .env.example and stages.md
- seed: FIXTURE_ADMIN_PASSWORD / FIXTURE_USER_PASSWORD env overrides so
  shared stages get non-public fixture passwords; credential is re-hashed
  on every run so re-seeding applies a changed password

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UpQz6ypHJsLfMf4S6fyQEB
2026-07-05 10:42:32 +02:00
f00fb19f32 Add mail outbox with SMTP delivery worker and templates
MailService renders transactional mails (verify-email, reset-password)
from the new de/en `mails` i18n namespace — text plus minimal HTML
with escaped interpolation — and enqueues them into mail_outbox.
MailWorker delivers pending rows every 15s through an injectable
transport (nodemailer; faked in tests) with quadratic backoff and a
permanent FAILED state after five attempts, logged as a warning.
SMTP_* and APP_BASE_URL join the environment schema with defaults
matching the new Mailpit container in the dev overlay.

Closes #12

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:46:12 +02:00
d675d69c21 Record stage provisioning completion
Some checks failed
CD / Build and push images (push) Failing after 5s
CD / Deploy to Test (push) Has been skipped
CD / Smoke tests against Test (push) Has been skipped
CD / Promote to Int (push) Has been skipped
CI / Lint, typecheck, test (push) Failing after 8s
CI / Build container images (push) Has been skipped
Test/Int directories, Caddy vhosts, deploy user, act_runner, and
registry access are live on the VPS; Gitea Actions is enabled
instance-wide. This commit doubles as the first full pipeline trigger.

Part of #9

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 00:12:59 +02:00
f870c3855d Document stage provisioning on the VPS
Some checks failed
CI / Lint, typecheck, test (push) Failing after 1m9s
CI / Build container images (push) Has been cancelled
deploy/stages.md walks through the Test/Int setup on 188.245.116.44:
stage directories and .env values, reverse-proxy vhosts (incl. the
/collab WebSocket route needed from M3), act_runner registration,
deploy user with per-stage SSH keys, and registry access. Root steps
are marked and executed together with the repo owner.

Part of #9

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:32:21 +02:00
6e5a632c31 Add production Dockerfiles and the Compose stack with dev overlay
Multi-stage images: web (workspace build baked into unprivileged
nginx with SPA fallback, asset caching, /healthz) and api (pnpm deploy
bundle with the prisma CLI for migrate-on-start, non-root, node-based
healthcheck). deploy/compose/docker-compose.yml defines the stage
stack (web, api, db) with frontend/internal networks, localhost-only
published ports for the host reverse proxy, log rotation, and named
volumes; .env.example documents every variable. compose.dev.yml layers
hot-reloading dev containers (or database-only usage) over the same
definition. Verified locally: full stack healthy, SPA fallback, readyz
green after automatic migration, db not reachable from outside.

Closes #6

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:30:21 +02:00
ca0f7cf4b1 Add Prisma with PostgreSQL, automatic migrations, and /readyz
apps/api gains Prisma (instance_settings as the first model) with the
initial migration applied automatically at startup via prisma migrate
deploy, a lazy-connecting PrismaService, and GET /api/v1/readyz
reporting named checks (database reachable, migrations applied) with
200/503. DATABASE_URL joins the validated environment schema;
MIGRATE_ON_START=false skips deploys for tests and tooling. An
idempotent seed script and a Compose dev overlay with PostgreSQL
(host port 5434 — 5433 is taken locally) complete the loop.

Closes #3

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:16:44 +02:00