Commit Graph

75 Commits

Author SHA1 Message Date
0c9d44e9c9 Legal texts for dorfteich.online; legal template covers the newer processing
All checks were successful
CD / Build and push images (push) Successful in 1m18s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m20s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 4m7s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m37s
CI / Import/export fidelity gate (push) Successful in 47s
- deploy/legal/dorfteich-online-{impressum,datenschutz}.md: ready-to-
  paste Markdown for the flagship instance (Admin → Legal pages after
  the wizard) — operator standard texts (§5 DDG, §18(2) MStV, VSBG
  no-participation, UGC/liability/copyright notices) plus a privacy
  policy grounded in what THIS instance actually does: Hetzner hosting
  with DPA, session cookie only (no banner), rate-limit IPs, proxy logs
  ≤30d, transactional + digest mail with unsubscribe, plaintext content
  with versions/comments and public pages, hashed API tokens with
  audited writes, nightly backups incl. the encrypted-tunnel mirror to
  the operator's private server, self-service export,
  deletion/pseudonymization, LfDI BaWü.
- docs/self-hosting/legal-template.md: review checklist and both
  language templates extended for the processing added since #82 —
  comments/version history, notifications/digest mails, API tokens,
  off-host backup copies.

No legal advice; texts follow the operator's standard building blocks
and should get a final human read before publishing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 19:00:05 +02:00
eeb0ef6794 Documentation set: features, manuals (user/pond-admin/site-admin), API, MCP, developer guide
All checks were successful
CD / Build and push images (push) Successful in 1m9s
CD / Deploy to Test (push) Successful in 10s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 4m7s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m34s
CI / Import/export fidelity gate (push) Successful in 47s
Seven audience-targeted documents (English first, German translation to
follow), linked from the README and a new docs/manual/ index:

- docs/features.md — public-facing feature overview: what Dorfteich
  can do and why that matters
- docs/manual/user-guide.md — everyday use: editor, wikilinks, labels,
  search, comments, watches/digests, import/export, settings
- docs/manual/pond-admin-guide.md — pond configuration: members/roles,
  access rules incl. label scoping and public pages, labels, comment
  policy, plugins, API/MCP opt-ins, files, export
- docs/manual/site-admin-guide.md — instance administration: wizard,
  settings, quotas, uploads, API/MCP switches, legal pages, plugins,
  users, and the system panel (jobs/backups/audit/storage)
- docs/manual/api-guide.md — example-driven public-API walkthrough
  (tokens, reading, writing through the collab-safe path, labels,
  comments, error semantics)
- docs/manual/mcp-guide.md — connecting AI assistants: switches, token
  scopes, Claude Code one-liner, mcp-remote bridge, tool table, audit
  and safety properties
- docs/developer/extending.md — plugin development (sandbox contract,
  SDK, block plugins, bundled apps/fullscreen, shipping) and core
  contributions (stack, dev environment, gates, house rules)

README: documentation index, repository-layout rows for docs/manual and
docs/developer, and the stale "architecture phase" status brought up to
reality. All relative links verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 17:18:30 +02:00
97f94f247b draw.io reference plugin: fullscreen editing, inline SVG rendering
All checks were successful
CD / Build and push images (push) Successful in 3m54s
CD / Deploy to Test (push) Successful in 10s
CI / Lint, typecheck, test (push) Successful in 4m9s
CI / Build container images (push) Has been skipped
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 13s
CI / Auth e2e pack (push) Successful in 5m36s
CI / Import/export fidelity gate (push) Successful in 47s
A new block plugin bundling the OFFICIAL draw.io editor — nothing ever
loads from diagrams.net; the sandbox CSP pins every request to the
plugin's own version-pinned asset path (zero-external-network verified
live via a request-capture run).

Plugin (packages/plugins/drawio):
- block data { xml, svg }: xml is the draw.io source (document of
  record), svg the rendered snapshot as raw markup — render mode,
  office/PDF exports (the existing fallback renderer already inlines
  data.svg) and the public view all show the diagram without running
  diagram code
- edit mode: snapshot + "edit in fullscreen" (an empty block opens the
  editor immediately); the bundled editor runs in a child iframe of the
  plugin's own assets and speaks draw.io's JSON embed protocol —
  Save & Exit exports xmlsvg, persists { xml, svg } via blockData, and
  drops back to the inline size
- build.mjs fetches the pinned release (v30.3.6) into a gitignored
  vendor/ cache (fonts-build pattern; skipped in CI — plugin.js still
  bundles, the installable ZIP needs a dev machine) and packs a trimmed
  webapp subset: no dev sources, no embed.diagrams.net integrations
  bundle, no standalone viewers, no MathJax/templates/PWA — 27 MiB ZIP,
  85 MiB unpacked, de+en editor languages

Host/SDK extensions (generic, not drawio-specific):
- new ui.enterFullscreen()/exitFullscreen(): the surface's frame becomes
  a viewport-covering overlay — same sandboxed iframe, only geometry
  changes; destroy removes the frame, so a vanished plugin can never
  leave the app covered
- sandbox CSP: connect-src/frame-src now allow the plugin's OWN asset
  path (was 'none') — bundled apps lazy-load their resources and run in
  a child frame, but the api and external hosts stay unreachable; HTML
  assets are served with the same CSP so a packaged page cannot widen
  the rules, and child frames inherit the sandbox attribute
- plugin size limits raised (ZIP 5→64 MiB, unpacked 20→256 MiB) for
  bundled-app plugins; content types for xml/txt/ico assets

Verified end to end against a local stack (9/9): install via dropzone
(85 MiB validation), block insert, fullscreen entry, bundled editor
boots inside the double sandbox (German UI), shape drawn, Save & Exit
persists, snapshot renders inline, survives reload, zero off-origin
requests throughout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 14:15:30 +02:00
52192eb05f Backup mirror to BASEL: rsync of the sets after every successful run (#84)
All checks were successful
CD / Build and push images (push) Successful in 3m51s
CI / Lint, typecheck, test (push) Successful in 4m5s
CD / Deploy to Test (push) Successful in 11s
CI / Build container images (push) Has been skipped
CD / Smoke tests against Test (push) Successful in 1m11s
CD / Promote to Int (push) Successful in 12s
CI / Auth e2e pack (push) Successful in 5m52s
CI / Import/export fidelity gate (push) Successful in 47s
The operator-level extra beside the admin-configured Nextcloud target
(#103), unblocked now that the ONE→BASEL tunnel is stable again.

- sidecar: optional mirror step (mirror.ts) driven purely by env —
  BACKUP_MIRROR_TARGET (rsync-over-ssh), BACKUP_MIRROR_SSH_KEY (private
  key on the secrets volume, never in image or repo),
  BACKUP_MIRROR_SSH_PORT. Runs after the prune of every successful run,
  so --delete aligns the remote retention with the local one (the
  newest-complete-set guarantee carries over). Only set files travel
  (db-*.dump, files-*.tar.gz); status files and bundles stay local.
  Host key pinned via accept-new into .mirror_known_hosts on the backups
  volume; fixed remote modes (dirs 750, files 640, symbolic --chmod —
  octal needs rsync ≥ 3, macOS dev machines ship 2.6.9). rsync +
  openssh-client added to the sidecar image.
- status: additive `mirror` block in status.json (outcome, transferred
  count, lastSuccessAt carried across failures) — shown on the admin
  backup card; failures alert via a new backupMirrorFailed mail (de+en)
  while the local run still counts as succeeded.
- deploy/backup-basel.md: complete BASEL-side walkthrough — dedicated
  user dorfteich-backup with a /home/ home and a bash login shell,
  explicitly avoiding the Debian backup-user (UID 34) pitfalls
  (nologin shell rejects rsync sessions, /var/backups home), key
  placement through the api container onto the secrets volume, .env
  values, on-demand verification.
- tests: rsync-arg/stats-parsing units plus an integration suite against
  the real rsync binary (local target; skips where rsync is absent) —
  transfer, idempotent re-run (0 files), retention alignment, failure
  path carrying lastSuccessAt.

Verified live against the real BASEL host from a native sidecar run:
initial transfer, host-key pinning, retention alignment after a local
prune, idempotency, and the failure path (surfaced in status.json while
the local run stayed green). BASEL side provisioned per the doc.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 12:20:32 +02:00
04e21a0aac Built-in MCP endpoint (Streamable HTTP) on top of the public API (#105)
All checks were successful
CD / Build and push images (push) Successful in 3m50s
CI / Lint, typecheck, test (push) Successful in 4m2s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 11s
CD / Smoke tests against Test (push) Successful in 1m14s
CD / Promote to Int (push) Successful in 13s
CI / Auth e2e pack (push) Successful in 5m37s
CI / Import/export fidelity gate (push) Successful in 47s
AI clients talk to the instance directly at /api/mcp — under the /api/
path (deviation from the issue's literal /mcp) so every existing reverse
proxy already routes it; no deployment changes anywhere.

- Transport: official @modelcontextprotocol/sdk server, STATELESS — each
  POST builds a fresh server+transport pair, no session store, replicas
  stay trivial; GET/DELETE answer 405. Auth per PAT bearer (#104 tokens),
  per-token rate limit (429 + Retry-After).
- Own switches, independent of REST: instance mcp.enabled (admin
  settings, default off; off = 404, feature invisible) + pond setting
  mcpEnabled (pond-settings toggle, default off) — pinned independent in
  both directions by tests.
- Tools (thin wrappers over the #104 services, same permission gates,
  audit-logged writes): list_ponds, list_pages, read_page, search,
  create_page, update_page (replace semantics through the collab-owned
  restore path — open editors converge), add_comment, list_labels,
  set_page_labels (exact replace), export_pond (link to the REST ZIP).
  Tool errors carry the api error codes; results carry stable slugs/ids.
  MCP resources stay the documented stage-2 stretch goal.
- Deliberately on the SDK's low-level Server API with a hand-written tool
  table (mcp-tools.ts): the typed registerTool generics drove tsc out of
  memory in a program this size; manual Zod validation keeps the wire
  behavior explicit.
- PublicApiService exposure filtering parameterized ('api' | 'mcp',
  shared pondFeatureEnabled helper) — one implementation, two switches.
- Docs: "Connect Claude Code / MCP clients" section in public-api.md
  (claude mcp add one-liner + mcp-remote bridge for stdio clients).

Verification: 8-test e2e pack driving the real MCP SDK client over
Streamable HTTP against a listening api (initialize + tools/list, switch
independence in both directions, anonymous/garbage 401, opt-in 404
semantics, page roundtrip incl. restore-NOTIFY, labels/comments, read
scope blocked from writes with scope_required); live check through the
web proxy against the seeded stack (tools list, create, read, update,
search — LIVE CHECK PASSED); full api suite 61/61 files green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 11:36:02 +02:00
0c85293830 Public REST API v1: personal access tokens, instance switch + per-pond opt-in (#104)
Some checks failed
CI / Lint, typecheck, test (push) Failing after 1m39s
CI / Auth e2e pack (push) Has been skipped
CI / Import/export fidelity gate (push) Has been skipped
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m51s
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 11s
Token-authenticated machine access at /api/public/v1 — the foundation for
the built-in MCP endpoint (#105).

Personal access tokens:
- api_tokens table (SHA-256 hash, scope read|write, optional pond
  restriction, expiry, revocation, throttled last-used) + migration;
  secrets are dt_pat_<random>, shown exactly once
- lifecycle endpoints under /users/me/api-tokens (session-only — a leaked
  token can never mint more tokens) with audit entries
  api.token_created/api.token_revoked
- settings UI section (create with scope/expiry/pond restriction,
  one-time reveal with copy, list with status + revoke), de+en

Activation (404 semantics per #60 on both levels):
- instance setting api.enabled (default off, admin settings switch)
- pond setting apiEnabled (default off, pond settings toggle; the
  PondsService settings-merge learned the key — the #92 lesson)

Surface (/api/public/v1, excluded from the SPA's global prefix):
- me, ponds, pages (list/read as Markdown+HTML, create from Markdown via
  the shared pipeline, PATCH title/content, DELETE to trash), search
  (permission-filtered + narrowed to exposed ponds, highlights as **…**),
  markdown ZIP export, labels (tree, create/rename/recolour/move/delete,
  assign/unassign), comments (threads, create, resolve/reopen)
- content replacement travels the collab-owned document path: the new
  state lands as a MANUAL version "API update", then the established
  restore NOTIFY applies it — open editors converge, history stays
  append-only, no second lineage (VersionsService.replaceContent)
- hand-maintained OpenAPI 3.1 document at /openapi.json, pinned to the
  controller by a route-coverage test in both directions

Enforcement:
- PublicApiGuard: instance switch → bearer PAT auth (request.user is the
  token's user) → per-token rate limit (429 + Retry-After) → scope
  (403 scope_required) → pond opt-in + token restriction
- the shared PermissionGuard then applies the unchanged permission model;
  PageParamSource gained pondSlugParam for the slug+slug routes
- no cookies anywhere → no CSRF surface (pinned by a hostile-Origin test)
- every write audit-logged as api.write with the token attributed

Tests/verification:
- 12-test e2e pack: lifecycle, switches, permission matrix
  (reader/editor/outsider × scopes), restriction, page roundtrip incl.
  restore-NOTIFY assertion, labels, comments incl. policy, search
  narrowing, ZIP export, rate limit; full api suite 60/60 green
  (quota fixture via per-user override — never the instance default)
- new collab-pack test proves an open editor converges onto an API
  content replacement (green against a local seeded stack)
- UI smoke against the built SPA: token create/reveal/revoke, pond
  opt-in persists, admin switch persists (10/10)
- docs/self-hosting/public-api.md + README link

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 11:17:03 +02:00
5cef359b8f Nextcloud backup target: admin-configured, manual + scheduled uploads, in-app restore (#103)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m45s
CD / Build and push images (push) Successful in 3m49s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m35s
CI / Import/export fidelity gate (push) Successful in 47s
Off-host backups for every self-hoster, configured entirely in the admin
UI — supersedes the host-specific mirror plan behind #84.

shared:
- webdav.ts (new package entry like token-crypto): minimal WebDAV client
  with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT
  (streamed), GET, DELETE; Nextcloud DAV path derived from the plain
  server URL, explicit DAV bases pass through
- backup-status.ts: additive remote-upload status in status.json, the
  restore-status.json contract (running/succeeded/failed + staleness
  bound), the backup_command/backup_maintenance NOTIFY channels, and the
  one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz)
- backup-set.ts moved here from apps/backup (api lists local sets)

backup sidecar:
- reads the backup.* instance settings directly from the database (admin
  changes apply next run; local retention row overrides the env) and the
  app password from the secret store
- after each successful set: bundle dump + files archive + manifest into
  ONE self-contained tar.gz, upload via WebDAV per schedule
  (off/daily/weekly; manual runs always upload), prune remote bundles —
  never the newest — and record the outcome in status.json; upload
  failures alert via a new backupUploadFailed mail (de+en)
- command listener on backup_command (run / restore) with a serial queue
  against the nightly timer
- restore orchestrator: restore-status.json → maintenance NOTIFY →
  grace → (remote: download + manifest-verify bundle) → terminate other
  DB connections → shared perform-restore path (same code as restore.sh)
  → final status + maintenance exit

api:
- MaintenanceGuard (global, registered before the setup gate): 503
  maintenance_mode while restore-status says running; health endpoints
  and the new public GET /backup/restore-status stay exempt; a stale
  running state (crashed sidecar) unblocks after 30 min
- MaintenanceStateService watches the file and restarts the api after a
  successful restore (fresh caches, migrate-on-start for older dumps);
  main.ts refuses to touch the database while a restore runs — a
  container restarting mid-restore must not race pg_restore with
  migrate deploy
- worker sweeps (conversion, mail outbox, scheduler) catch transient
  database failures instead of dying on an unhandled rejection — the
  restore's connection termination crashed the api in verification
- backup admin endpoints under /admin/system/backup: settings (live
  connection test before save, password write-only into the secret
  store), nextcloud/test, sets (local via the ro backups mount + remote
  via WebDAV), run + restore (type-to-confirm backstop, source
  validation) — commands travel as NOTIFY payloads; audit actions
  backup.settings_changed/run_triggered/restore_requested
- readyz: new warning-level backup_remote check while a target is
  configured (26 h daily / 170 h weekly bound)

collab:
- maintenance listener: on enter, persist + close every live session and
  refuse new connections until exit (failsafe timeout 30 min) — no
  in-memory document may write pre-restore content back afterwards

web:
- Admin → System backup section: status card with remote facts and a
  "Back up now" button, the Nextcloud settings form with test button,
  and the restore picker (local + remote sets, type-to-confirm)
- global maintenance screen: any 503 maintenance_mode flips the SPA to a
  status page polling the exempt endpoint, reloading when the instance
  returns

Verified end-to-end against a live stack (fresh DB, native api + sidecar,
fake WebDAV server): configure → test → manual backup → bundle upload →
readyz/sets/status surfaces → remote restore with maintenance gate,
marker rollback and api restart; suites: shared 21, backup 9, collab 11,
api 58 files green, lint + i18n:check + typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 10:39:18 +02:00
a9e901c449 Run operations QA on every release candidate before the prod gate (#90)
All checks were successful
Release / Build release images and notes (push) Successful in 1m8s
CD / Build and push images (push) Successful in 1m9s
CD / Deploy to Test (push) Successful in 9s
Release / Release-candidate operations QA (push) Successful in 41s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 10s
Prod deploy / Deploy the released images to Prod (push) Successful in 15s
CI / Lint, typecheck, test (push) Successful in 3m35s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m32s
CI / Import/export fidelity gate (push) Successful in 48s
New deploy/release-qa.sh, wired as the release workflow's second job: it
boots the PREVIOUS release with pre-seeded fixture content in a scratch
environment, swaps the api to the candidate against the same database
(migrations auto-apply, readiness green, content intact — the
seed_fixture/assert_fixture pair is the update-fixture contract future
migrations extend), asserts the degraded-readyz semantics on the
candidate (200 + warn-level checks without sidecars), and runs a full
backup/restore roundtrip with the candidate's sidecar into a second,
empty database. The wizard e2e already guards fresh installs in CI
(issue #81). Verified green on the host for v0.1.0→v0.1.1; a simulated
destructive migration made the suite fail loudly (negative test,
not committed). A human pushes the prod tag only when both release jobs
are green — the documented pre-approval gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 00:29:53 +02:00
4b55fb92ac Write the self-hosting guide and add the optional caddy TLS profile (#88)
All checks were successful
CD / Build and push images (push) Successful in 1m7s
CD / Deploy to Test (push) Successful in 10s
CD / Smoke tests against Test (push) Successful in 1m8s
CD / Promote to Int (push) Successful in 10s
CI / Lint, typecheck, test (push) Successful in 3m13s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m18s
CI / Import/export fidelity gate (push) Successful in 46s
docs/self-hosting/README.md is the complete operator contract: install
from the two reference files, first-run wizard walkthrough, update
procedure with the one-release downgrade window, backup/restore with the
sidecar, readyz-based troubleshooting (incl. the classic proxy/WebSocket
and APP_BASE_URL/CSRF mistakes), and a build-from-source note; linked
from the repository README; English-only by documented decision. The
reference compose gains a `caddy` profile (new Caddyfile) that publishes
80/443 and terminates TLS via Let's Encrypt for $DOMAIN — localhost uses
Caddy's internal CA for smoke tests. deploy/self-hosting-verify.sh
scripts the clean-machine test: a fresh directory with only the
published files boots to the wizard answering over TLS, then removes
itself; verified green on the stage host.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 21:21:10 +02:00
6e99cb35fc Trigger the restore drill on demand via drill-* tags (#87)
All checks were successful
Restore drill / Restore the latest backup into a scratch stack (push) Successful in 28s
CD / Build and push images (push) Successful in 1m4s
CD / Deploy to Test (push) Successful in 11s
CD / Smoke tests against Test (push) Successful in 1m9s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 3m19s
CI / Build container images (push) Has been skipped
CI / Import/export fidelity gate (push) Successful in 45s
CI / Auth e2e pack (push) Successful in 5m9s
Gitea 1.22 cannot dispatch workflows through the API or UI (that lands in
1.23), so pushing a drill-* tag is the on-demand path next to the monthly
schedule.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 21:01:00 +02:00
d95c18e9e8 Automate the monthly restore drill with a scratch-stack workflow (#87)
All checks were successful
CD / Build and push images (push) Successful in 1m5s
CD / Deploy to Test (push) Successful in 10s
CD / Smoke tests against Test (push) Successful in 1m7s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 3m18s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m14s
CI / Import/export fidelity gate (push) Successful in 45s
New scheduled workflow (monthly + on demand) runs deploy/backup/drill.sh:
it reads the drilled stage's backups volume strictly read-only, restores
the latest successful set into a throwaway Postgres and volumes under a
unique drill prefix via the backup image's restore path, boots the api
against the result, and verifies readyz (database + migrations), row
counts, rendered content in the page cache, a public API request, and a
media byte-check against the attachments table — then tears everything
down, also on failure. Each run reports its outcome as a comment on the
pinned "Restore drills" issue (#98). docs/operations/restore-runbook.md
carries the manual procedure, which doubles as the Prod relocation path;
pre-go-live the drill restores the Test set (switch the source volume at
go-live, #89 — off-host fetch from the BASEL mirror stays with #84).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 20:58:06 +02:00
c8aac13dfb Add Site-Admin system panel with persistent audit trail (#86)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m14s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m45s
CD / Deploy to Test (push) Successful in 10s
CD / Smoke tests against Test (push) Successful in 1m11s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m20s
CI / Import/export fidelity gate (push) Successful in 45s
New /admin/system panel (operations.md §Maintenance jobs): the maintenance
job list shows every registered job with truthful last-run data (new
Job.lastDurationMs recorded by the scheduler) and a manual trigger that
respects the run-mutex and is itself audit-logged; a backup card mirrors
the sidecar's status.json including the freshness verdict; an audit-log
viewer filters by actor, action, and time range with pagination; and a
storage overview lists the largest ponds. Auth events and admin actions
(grants, members, user/quota admin, plugins, settings, setup) now land in
a new audit_log table through a central AuditService — which keeps
emitting the established stdout log line — while content activity stays
log-only by design. All endpoints are Site-Admin-only; covered by API DB
tests and a Playwright pack in CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 20:03:05 +02:00
0ef96147e0 Extend readyz with backup freshness and a degraded status level (#85)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m11s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m44s
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 10s
CI / Auth e2e pack (push) Successful in 5m13s
CI / Import/export fidelity gate (push) Successful in 46s
readyz now enumerates database/migrations (hard failures, HTTP 503),
converter/renderer, and a new backup check that reads the sidecar's
status.json from the read-only backups mount and warns when the last
successful backup is older than 26 h. Warning-level checks surface as
overall status "degraded" while staying HTTP 200 — monitors alert on
the body keyword, Docker healthchecks keep using the liveness endpoints
so a degraded instance is never restart-looped. The status.json shape
moved to @dorfteich/shared as the contract between the sidecar and its
readers (#85/#86); deploy/monitoring.md defines the Uptime-Kuma monitor
set per stage. The api image also pre-creates /data/backups node-owned
so the shared backups volume stays writable for the sidecar regardless
of which container initializes it, and the sidecar's scheduler survives
runs that cannot even record their status.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 19:02:59 +02:00
8dbff86537 Add backup sidecar: nightly dump, volume archive, prune, status, restore (#83)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m9s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m47s
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 9s
CI / Auth e2e pack (push) Successful in 5m25s
CI / Import/export fidelity gate (push) Successful in 45s
New apps/backup service (ADR 0015): nightly pg_dump -Fc plus one tar of the
uploads/plugins volumes as a consistent restore set on a new backups volume,
retention prune that never removes the newest complete set, atomic
status.json for the readiness/admin consumers (#85/#86), and a failure mail
sent directly via nodemailer (the api may be the broken part) with de/en
texts in the shared mails catalog. BACKUP_RUN_ONCE=1 gives the on-demand
path; deploy/backup/restore.sh automates the documented restore runbook.
The pure secret-store helpers moved to @dorfteich/shared so the sidecar
resolves the wizard-written SMTP relay exactly like the api.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 17:29:15 +02:00
fd2bdb3fb8 Add instance legal pages with public rendering and footer links (#82)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m7s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m14s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m16s
CD / Promote to Int (push) Successful in 9s
CI / Auth e2e pack (push) Successful in 5m9s
CI / Import/export fidelity gate (push) Successful in 45s
Imprint and privacy policy are two new Markdown instance settings
(legal.imprint, legal.privacyPolicy), edited by Site Admins in a new
"Legal pages" admin section with a toggleable rendered preview. The
preview uses the same shared pipeline the server renders with
(markdown → schema doc → escaped HTML), so stored markup can never
smuggle script to visitors.

The pages render publicly at /legal/imprint and /legal/privacy — as an
SPA route plus, like #56, a self-contained server-rendered HTML
document under /api/v1/legal/:kind. The endpoints are setup-exempt:
legal information stays reachable even while the first-run wizard is
pending. Unconfigured pages show a localized notice instead of 404ing,
and Site Admins additionally get a warning banner linking to the
settings. A new footer with both links appears on every SPA view
(editor, auth screens, public pages) and in the server-rendered
documents, whose shared shell moved to public/html-shell.ts and now
renders its chrome in the instance default locale (ADR 0012).

docs/self-hosting/legal-template.md ships imprint and privacy-policy
templates in English and German whose sections mirror Dorfteich's
actual processing activities (accounts, sessions, rate-limit IPs,
proxy logs, transactional mail, content, export, deletion, no
third-party requests), with a review checklist tied to security.md
§Privacy.

New `legal` i18n namespace (de+en); api and web e2e coverage including
a new CI legal pack (footer navigation, notice vs. admin banner, and
the admin form publishing a text end to end).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 16:40:04 +02:00
f0a82bad20 Add the first-run setup wizard API with env-backed secret store (#80)
Some checks failed
CD / Build and push images (push) Successful in 3m16s
CI / Lint, typecheck, test (push) Successful in 3m5s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Failing after 3m35s
CD / Promote to Int (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m6s
CI / Import/export fidelity gate (push) Successful in 43s
CI / Build container images (push) Has been skipped
When the api runs against a database without the setup.completedAt
marker, a global SetupGuard answers every non-exempt route with 503
setup_required; only /setup/*, health probes, and the session routes
stay reachable. The wizard steps (POST /setup/admin|instance|smtp|
registration|complete) write straight to their production homes; the
Site Admin step signs its creator in, later steps require that session.
Completing sets the marker and locks every step permanently (410, also
across restarts, and not reopenable via PATCH /admin/settings).

SMTP entered in the wizard is verified with a live delivery test first
(failure blocks the step with the transport error as detail) and then
persisted to the new env-backed secret store: a mode-600 dotenv file on
the new `secrets` volume (SECRETS_FILE). Explicit container env always
wins over the store; empty compose-passed strings count as unset. The
mail transport now resolves lazily through SmtpConfigService so wizard
changes apply without a restart.

SETUP_ADMIN_* env pre-seeds the whole wizard at boot for automated
deploys; a backfill migration marks instances that already have a Site
Admin as completed, and seed/vitest global-setup do the same for
fixture databases. The setup e2e suite provisions its own fresh
database (CREATE DATABASE + migrate deploy) per run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 15:10:28 +02:00
0003063c39 Add pageTool plugins with toc and page-index references (#77)
All checks were successful
CI / Auth e2e pack (push) Successful in 4m37s
CI / Import/export fidelity gate (push) Successful in 43s
CI / Lint, typecheck, test (push) Successful in 2m54s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m14s
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m8s
CD / Promote to Int (push) Successful in 9s
The read-only widget surface over page/pond data (ADR 0008 extension
point `pageTool`):

- Host: PageToolsPanel lists the pond's active pageTool surfaces behind
  disclosures — each sandbox iframe mounts lazily on first open and tears
  down on close. The same surfaces are insertable as plugin_block embeds
  (#76's insert picker now offers pageTool points too; the sandbox drives
  both through the same render lifecycle).
- New `ui.scrollToHeading(headingId)` capability: outline ids are derived
  from the doc and never stamped into the DOM, so the host resolves the id
  to its heading position via the shared extractOutline and scrolls the
  matching rendered heading.
- `readPond.listPages` now carries label *names* per summary
  (PagesService.pluginPageSummaries) — the page-index filter chips work on
  data the viewer could resolve anyway; per-page permission filtering
  stays in the service as before.
- Reference plugins packages/plugins/toc and packages/plugins/page-index:
  real SDK consumers (createPlugin + windowTransport), bundled with
  esbuild into the package ZIP; i18n de/en is inlined at build time — the
  sandbox CSP forbids runtime fetches, the i18n/ files stay the single
  source. The toc re-fetches its outline on a slow poll, so live heading
  edits appear once the collab server has re-derived the content cache.
- e2e page-tools.spec.ts covers the acceptance criteria: live outline
  updates after the persistence debounce, heading click scrolls, embedded
  page-index navigates via ui.openPage, and a label-restricted reader
  never sees the denied page in the index.
- CI: the auth-e2e job now runs the section-styles (missed in #75),
  plugin-blocks, and page-tools packs, with login-rate-limit resets.
- plugins.e2e.db.test clears the plugin registry up front: a local dev DB
  is shared with the e2e stack, whose installed real `toc` would otherwise
  collide with the fixture of the same id.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 13:27:30 +02:00
e32f961047 Complete section-style plugins: CSS gate, injection, picker, export (#75)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 2m54s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m9s
CD / Deploy to Test (push) Successful in 11s
CD / Smoke tests against Test (push) Successful in 1m9s
CD / Promote to Int (push) Successful in 10s
CI / Auth e2e pack (push) Successful in 3m57s
CI / Import/export fidelity gate (push) Successful in 43s
Second half of #75 on top of the section node (2e96173/784f21d):

- Install gate for section_style CSS (plugin-css.ts): every rule must be
  scoped under one of the plugin's own .dt-style-<pluginId>-<styleId>
  classes (enforced, not rewritten — grouping at-rules checked inside,
  @font-face/@keyframes exempt, statement at-rules rejected); positioning
  out of the content flow (anything but static/relative) is rejected as an
  overlay vector; "</style" is rejected as a breakout vector for inlined
  embedding. Hostile fixtures from the acceptance list are pinned in
  plugin-css.test.ts.
- Web: usePondPlugins loads the pond's active plugins once per visit;
  SectionStyleSheets links each active style plugin's immutable
  styles.css; SectionStyleMenu (toolbar) wraps/restyles/unwraps with a
  picker fed from the plugins' i18n titles. Sections show a faint dashed
  hint while editing so unstyled (plugin-disabled) sections stay findable.
- PDF export: PluginsService.sectionStyleCssForPond inlines the pond's
  active section-style CSS into the Gotenberg HTML, so styled sections
  survive the network-isolated render; covered in export.service.db.test.
- Reference plugin packages/plugins/section-styles-basic (callout, info,
  warning, colored-box; theme-neutral semi-transparent backgrounds), a
  workspace package whose tests validate it against the SDK schema and
  whose real files run through the api install gate.
- e2e section-styles.spec.ts: install → wrap → computed background in edit
  and read mode → unwrap → neutral fallback after disabling the plugin.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 11:43:36 +02:00
514cdf5ead Move stages, runner and deploy target to dedicated host ONE
All checks were successful
CD / Build and push images (push) Successful in 3m45s
CI / Lint, typecheck, test (push) Successful in 3m39s
CI / Auth e2e pack (push) Successful in 3m43s
CI / Import/export fidelity gate (push) Successful in 43s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m12s
CD / Promote to Int (push) Successful in 10s
Gitea itself relocated to the dedicated Hetzner server ONE
(one.101010.cloud); this commit moves the rest of the delivery chain
along: cd.yml now deploys to one.101010.cloud (DEPLOY_HOST_KEY secret
rotated to ONE's host key), the act_runner one-dorfteich replaces
vps-dorfteich, and the stage/deployment docs describe the new topology
(Test/Int migrated with volumes; Prod joins on the same host at M8).
The Int promotion gate variable RUN_INT_DEPLOY was removed — the new
host has the capacity, so promote-int deploys again by default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-11 08:05:36 +02:00
621aa47244 Add plugin storage, install API, and directory watcher (#71)
Some checks failed
CI / Auth e2e pack (push) Waiting to run
CI / Import/export fidelity gate (push) Waiting to run
CI / Build container images (push) Waiting to run
CD / Build and push images (push) Failing after 1m33s
CD / Deploy to Test (push) Has been skipped
CD / Smoke tests against Test (push) Has been skipped
CD / Promote to Int (push) Has been skipped
CI / Lint, typecheck, test (push) Has been cancelled
Backend for installing plugin ZIPs (ADR 0008, plugin-architecture.md
§Lifecycle, security.md §Plugins). Consumes the #70 SDK for validation.

- Schema: `plugins` (id, name, version, apiVersion, kind, mode, manifest
  jsonb, removedAt soft-delete) + `pond_plugins` (per-pond activation) +
  `PluginInstanceMode` enum; migration 20260710130000_plugins.
- `PluginPackageService`: pure, stateless ZIP → validated package via
  fflate — structure check, manifest validation (SDK), apiVersion gate,
  kind/bundle/styles rules, CSS sanitation (no @import / external url() /
  expression()), zip-slip and unpacked-size guards. Each failure carries a
  stable PluginErrorCode; manifest issues travel as ApiError details.
- `PluginStorageService`: on-disk layout `<PLUGINS_DIR>/<id>/<version>/`;
  atomic writeVersion (staging dir + rename, no 404 window mid-update),
  removeVersion/removePlugin, traversal-safe asset resolution, dropzone +
  quarantine dirs.
- `PluginsService`: install/update (update only to a strictly higher
  version, preserving the admin's instance mode; files land before the
  metadata pointer flips) / uninstall (refused while required; soft-delete
  + files removed + pond activations dropped) / list / get.
- `POST/GET/DELETE /admin/plugins` (SiteAdminGuard, multer memory upload),
  error→HTTP-status mapping. Public version-pinned static serving at
  `GET /plugins/:id/:version/*rest` with immutable cache + nosniff, only for
  the installed current version.
- `PluginWatcherService`: watches `<PLUGINS_DIR>/_dropzone/`, runs the same
  validation, installs valid drops and quarantines invalid ones with the
  error logged; inert under NODE_ENV=test (tests drive processDropped).
- SDK: `compareVersions`/`isHigherVersion`. shared: `PluginView`,
  `PluginInstanceMode`, `PLUGIN_ERROR_CODES`, `PLUGINS_DIR` env, plugin
  error i18n (de+en). Compose: `plugins` volume + `PLUGINS_DIR`.
- Tests: package unit test (valid + each invalid class) and an e2e DB test
  (GUI install + immutable serving, non-admin 403, invalid-manifest details,
  dropzone install + quarantine, atomic higher-only update, required-guarded
  uninstall that removes files and tombstones metadata).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-10 16:55:26 +02:00
0c6494f209 Enforce permissions in API guards and retire interim access (#52)
All checks were successful
CD / Build and push images (push) Successful in 2m54s
CI / Lint, typecheck, test (push) Successful in 2m25s
CI / Auth e2e pack (push) Successful in 2m58s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m12s
CD / Promote to Int (push) Successful in 12s
Every route now declares its access rule explicitly and is enforced
through the shared resolution algorithm (permissions.md):

- PermissionGuard + decorators (@RequiresPondRole, @RequiresPagePermission,
  @RequiresAttachmentPermission, @AuthenticatedOnly) applied to every
  route; a route-enumeration test proves full coverage alongside
  @Public()/Site-Admin-guarded routes.
- 404/403 policy (documented in README conventions): denied reads answer
  404 (existence hiding), denied writes on readable things answer 403;
  trash views need write capability (ADR 0013).
- PermissionService resolves page/pond questions via the shared resolver,
  with an in-process pond-context cache (grants + label parents) that is
  invalidated on every grant/label-tree change and TTL-bounded as a
  multi-process safety net. Grant changes also fire pond_access_changed
  for collab revalidation (#39/#53).
- shared: pond-scope resolution (hasPondRole, canSeePond) next to the
  page resolver; grant wire schemas + GrantView.
- Owner Pond-Admin grants: migration backfill for all existing ponds,
  created transactionally with every new pond (shared + personal + seed).
- Grant CRUD under /ponds/:id/grants (pond_admin-gated) with structural
  and referential validation, last-admin protection, audit logs.
- InterimAccessService deleted; page lists, search, backlinks, phantom
  links, and trash listings are filtered per page through the resolver;
  collab tokens are now truly ro for readers.
- Fixture-matrix e2e (reader/editor/pond admin/foreign, label-deny,
  authenticated-subject, revoke-then-immediate-deny cache test).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY
2026-07-09 16:31:41 +02:00
8316c617d2 Add collaboration server skeleton (Hocuspocus) with health, container, and CI/CD (#33)
All checks were successful
CD / Build and push images (push) Successful in 2m36s
CI / Lint, typecheck, test (push) Successful in 1m50s
CI / Auth e2e pack (push) Successful in 1m58s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
Bootstrap apps/collab as a Hocuspocus WebSocket server (ADR 0003):
- pino JSON logging (service=collab) and shared Zod env validation
  (collabEnvSchema); structured connection open/close logs.
- /healthz endpoint (process liveness + PostgreSQL ping) served via the
  onRequest hook, matching the container-internal path and the proxied
  /collab/healthz path; any WebSocket handshake is accepted for now
  (authentication arrives with #34, persistence with #35).
- Dockerfile (ESM workspace build) and a compose service on the frontend
  and internal networks with a healthcheck; dev overlay service and a new
  COLLAB_PORT variable.
- CD builds, pushes, and promotes the collab image; CI builds it on PRs;
  the smoke suite asserts /collab/healthz through the reverse proxy.
- deployment.md/stages.md: proxy routing, per-stage COLLAB_PORT, checklist.

Closes #33

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-08 14:53:44 +02:00
b16d23297e Scaffold pnpm monorepo with lint, format, and test tooling
pnpm workspace with apps/web, apps/api, apps/collab, and
packages/shared; strict TypeScript base config, repo-wide ESLint (flat)
+ Prettier, Vitest per package, and root scripts lint/typecheck/test/
build. @dorfteich/shared ships a first health-response helper consumed
by apps/api to prove workspace linking. Existing markdown docs are
reformatted once by the new Prettier setup.

Closes #1

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:06:27 +02:00
6517c8fb71 Update topology: non-prod stages on the VPS, Prod host decided at go-live
Kickoff update by the project owner: test.dorfteich.cloud and
int.dorfteich.cloud run on the VPS 188.245.116.44 (DNS for
*.dorfteich.cloud and *.dorfteich.online already points there); the
Prod host for dorfteich.online is chosen at go-live. Dev runs locally
via Docker. Affects deployment.md, roadmap.md, and ADR 0014; the
matching Gitea issues (#8, #9, #87, #89) were updated in place.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 15:20:58 +02:00
0629411966 Add architecture documentation, ADRs, and operations concept
Initial deliverable of the architecture phase: 16 ADRs (stack, CRDT
collaboration, plugin sandbox, import/export, backups, CI/CD), data
model, permission model, real-time collaboration and plugin concepts,
deployment/operations/security documentation, and the milestone roadmap
that the implementation issues are derived from.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 14:36:16 +02:00