13 Commits

Author	SHA1	Message	Date
Claude Fable 5	5cef359b8f	Nextcloud backup target: admin-configured, manual + scheduled uploads, in-app restore (#103) ... Off-host backups for every self-hoster, configured entirely in the admin UI — supersedes the host-specific mirror plan behind #84. shared: - webdav.ts (new package entry like token-crypto): minimal WebDAV client with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT (streamed), GET, DELETE; Nextcloud DAV path derived from the plain server URL, explicit DAV bases pass through - backup-status.ts: additive remote-upload status in status.json, the restore-status.json contract (running/succeeded/failed + staleness bound), the backup_command/backup_maintenance NOTIFY channels, and the one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz) - backup-set.ts moved here from apps/backup (api lists local sets) backup sidecar: - reads the backup.* instance settings directly from the database (admin changes apply next run; local retention row overrides the env) and the app password from the secret store - after each successful set: bundle dump + files archive + manifest into ONE self-contained tar.gz, upload via WebDAV per schedule (off/daily/weekly; manual runs always upload), prune remote bundles — never the newest — and record the outcome in status.json; upload failures alert via a new backupUploadFailed mail (de+en) - command listener on backup_command (run / restore) with a serial queue against the nightly timer - restore orchestrator: restore-status.json → maintenance NOTIFY → grace → (remote: download + manifest-verify bundle) → terminate other DB connections → shared perform-restore path (same code as restore.sh) → final status + maintenance exit api: - MaintenanceGuard (global, registered before the setup gate): 503 maintenance_mode while restore-status says running; health endpoints and the new public GET /backup/restore-status stay exempt; a stale running state (crashed sidecar) unblocks after 30 min - MaintenanceStateService watches the file and restarts the api after a successful restore (fresh caches, migrate-on-start for older dumps); main.ts refuses to touch the database while a restore runs — a container restarting mid-restore must not race pg_restore with migrate deploy - worker sweeps (conversion, mail outbox, scheduler) catch transient database failures instead of dying on an unhandled rejection — the restore's connection termination crashed the api in verification - backup admin endpoints under /admin/system/backup: settings (live connection test before save, password write-only into the secret store), nextcloud/test, sets (local via the ro backups mount + remote via WebDAV), run + restore (type-to-confirm backstop, source validation) — commands travel as NOTIFY payloads; audit actions backup.settings_changed/run_triggered/restore_requested - readyz: new warning-level backup_remote check while a target is configured (26 h daily / 170 h weekly bound) collab: - maintenance listener: on enter, persist + close every live session and refuse new connections until exit (failsafe timeout 30 min) — no in-memory document may write pre-restore content back afterwards web: - Admin → System backup section: status card with remote facts and a "Back up now" button, the Nextcloud settings form with test button, and the restore picker (local + remote sets, type-to-confirm) - global maintenance screen: any 503 maintenance_mode flips the SPA to a status page polling the exempt endpoint, reloading when the instance returns Verified end-to-end against a live stack (fresh DB, native api + sidecar, fake WebDAV server): configure → test → manual backup → bundle upload → readyz/sets/status surfaces → remote restore with maintenance gate, marker rollback and api restart; suites: shared 21, backup 9, collab 11, api 58 files green, lint + i18n:check + typecheck clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-12 10:39:18 +02:00
Claude Fable 5	67fb01fe2b	Notify watchers about page changes and comments, with an in-app center (#94) ... New notifications table (payload denormalized for join-free rendering; mailed_at already prepares the #95 digests). Generation fans page events out to page and pond watchers, excluding the actors, and re-checks page read permission per watcher at delivery time — a revoked watcher gets nothing. Sources: named version snapshots (api), new comments (api), and the collab server's automatic session-close snapshots — announced over a new pg NOTIFY channel (the reverse of the established api→collab bus) consumed by a dedicated LISTEN client in the api, since the collab server has no permission resolution of its own. API: paginated list (unread first via nulls-first ordering), mark read, mark all read. UI: bell with unread badge in the top bar (30 s polling, no push in v1) and a dropdown whose entries navigate and mark themselves read; comment notifications deep-link with ?comments=1, which now opens the comments panel on load. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-11 23:20:25 +02:00
Claude Opus 4.8	9d288b2ad0	Wire real permissions into collab tokens and revocation (#53) ... Live editing now obeys the same rules as REST: the collab-token mode comes from the shared grant resolution, anonymous visitors can join public pages, and revoking write access flips a running session to read-only within seconds. - Anonymous public tokens: `GET /pages/:id/collab-token` is `@Public()` but still permission-guarded, so a logged-out visitor gets an `ro` token where a `public` grant makes the page readable (404 otherwise). The token's `userId` is nullable (shared schema + collab context) for anonymous subjects. - Prompt revocation: the pond-level NOTIFY (#39) now also fires on label tree/assignment changes (LabelsService move/remove/assign/unassign), and the collab server closes the *actual* WebSocket instead of only sending an application-level close message. Hocuspocus' `closeConnections` leaves the socket open so the client only re-checks on its ~30s message timeout; `closeDocumentConnections` drops the socket so the client reconnects and re-authenticates with a freshly-resolved token at once — the "within seconds" downgrade the milestone promises. - Tests: the #52 fixture matrix gains anonymous cases (public grant → `ro`, none → 404); a collab db test proves an editor downgraded to reader goes read-only on reconnect (its post-downgrade edits no longer reach a peer); a new browser `collab-permissions` pack covers the read-only participant and the live downgrade end to end (new plain `fixture-editor` account). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-09 18:48:42 +02:00
Claude Opus 4.8	91dfccf226	Add SearchProvider interface with PostgreSQL FTS (#49) ... Full-text search behind a swappable interface (ADR 0010). - prisma: `page_content_cache.search_vector tsvector` (Unsupported column); migration adds it plus a GIN index (raw SQL — the index is a production perf optimization; correctness holds without it, so schema-pushed test DBs work unchanged). - shared: `normalizeForSearch` (NFKD + strip diacritics + lowercase) folds both the indexed text and the query, so 'Baume' finds 'Bäume' without the Postgres `unaccent` extension; search query schema + result view + highlight sentinels. - api search module: - abstract `SearchProvider` (DI token: indexPage / removePage / search / reindexAll) so an external engine can replace the binding — a fake proves the seam in a test. - `PostgresSearchProvider`: weighted vector (title A, labels B, body C), `websearch_to_tsquery`, `ts_headline` snippets, results filtered to the ponds the user may read; `GET /search?q=&pondId=&labels=`. - `search:reindex` CLI (rebuilds from the content cache, idempotent). - reindex hooks: page create/rename (title) and label assign/unassign/ rename/delete (labels are weight-B). - collab: the persistence hook maintains `search_vector` in the same transaction as the content cache (same weighting, normalized). - tests: shared normalize/schema; api db (title ranks above body, highlight, diacritic-insensitive match, permission filter, idempotent reindex) and the fake-provider DI test; collab persistence already covers the write path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY	2026-07-09 13:25:05 +02:00
Claude Opus 4.8	14e69b399c	Add wikilink index, backlinks API, and phantom resolution (#47) ... Maintain a server-side `page_links` index on every content change so backlinks and missing-target ("phantom") links can be queried. - prisma: `PageLink` (from_page_id, nullable to_page_id, target_slug; unique per (from, slug); cascade on source purge, set-null on target purge); migration. - shared: `extractWikilinkSlugs(doc)` (distinct target slugs) and the `BacklinkView` / `PhantomLinkView` read shapes. - collab: the persistence hook (#35) now rewrites the source page's outgoing links in the same transaction as the content cache — one row per distinct wikilink slug, resolved to a page in the same pond (null = phantom). - api: `GET /pages/:id/backlinks` (permission-filtered — wikilinks resolve within a pond, so seeing the pond is the read right) and `GET /ponds/:id/phantom-links` (missing targets grouped with their referrers). Creating or renaming a page to a slug that pages already link to resolves those phantom rows; because links store the target's id, backlinks survive a later rename of the target's slug. - tests: shared extraction unit test; collab persistence db test (store writes resolved + phantom rows and rewrites the index); api LinksService db test (backlinks, permission filter, phantom aggregation, create/rename resolution, id-based backlinks survive target rename). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY	2026-07-09 12:54:10 +02:00
Claude Opus 4.8	1bda137ca4	Add version history UI: list, view, diff, restore (#42) ... Users can see who changed what and restore old states (ADR 0013). - shared: dependency-free word-level Markdown diff (diffMarkdown) with a unit test; PageVersionContentView; PAGE_RESTORE_CHANNEL + PageRestoreRequest. - api: GET /pages/:id/versions (list), GET .../:versionId (read-only HTML + Markdown for diffing), POST .../:versionId/restore. Every route requires write access — viewing history is gated like editing (permissions.md). Restore checks permission, then emits the page_restore NOTIFY; history is append-only (the api never deletes a version). - collab: a page_restore listener applies the restore on the live document via openDirectConnection — it snapshots the current state as a PRE_RESTORE version, then replaces the content in one transaction, so every connected client converges and the change persists like a normal edit. - web: HistoryPanel (version list with time/trigger/label/contributors, a read-only render of a selected version, a Markdown diff against the current page, and a restore action), toggled from the page menu. de+en strings. Tests: shared diff (added/removed/round-trip/edges); collab restore DB test (a connected client converges on the restored content; a pre-restore snapshot is appended alongside the original — append-only); api list/get/restore (newest-first, rendered content, write-permission gate, restore returns the target without mutating history). This completes M3 (real-time collaboration & history, #33–#42). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY	2026-07-09 08:53:42 +02:00
Claude Opus 4.8	6fb6f6fce7	Add version snapshots: automatic, named, thinning (#41) ... Version history is a core kickoff decision (ADR 0013). Snapshots are full, self-contained encoded Yjs states, so restore never depends on the update log and compaction (#40) cannot lose history. (The page_versions / page_pending_contributors tables and base schema landed a commit early, bundled into 3583a04; this commit completes #41.) - schema: page_versions gains created_by (editor of manual/pre-restore versions; null for automatic snapshots). shared: PageVersionView, CreateVersionInput, PageVersionTrigger. - collab: PostgresVersionStore tracks contributors per open doc (onChange), flushes them to the shared page_pending_contributors accumulator on store, creates an automatic snapshot on last-participant disconnect (only if something changed — no duplicate on a quick reconnect) and every 30 active-editing minutes. Contributors and snapshot are consumed atomically. - api: POST /pages/:id/versions creates a named version (write permission, label + creator, snapshot reconstructed from persisted state, consumes the same contributor accumulator). Daily version-thinning scheduler job keeps all versions for 90 days, then the newest auto snapshot per day; manual and pre-restore versions are never thinned. pre_restore trigger reserved for #42. Tests: collab (one auto version on session end with the full two-author contributor set, none when unchanged, no duplicate on reconnect, interval snapshot); api (named version stores label+creator, contributor set consumed, non-owner refused, thinning time-travel keeps newest-per-day beyond window). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY	2026-07-09 08:37:07 +02:00
Claude Opus 4.8	acd1cc32b7	Add Yjs update-log compaction job (#40) ... Update logs grow with every edit; compaction bounds storage and load time. - prisma: `collab_open_sessions` table (page_id, heartbeat_at) — the live-session registry that lets the compaction job avoid pages being edited, decoupled from collab (no api↔collab network call) and self- healing (a crashed collab's rows age out of the freshness window). - collab: `PostgresSessionRegistry` marks a page open on document load and closed on unload, and refreshes an every-30s heartbeat for all open docs; wired into the server hooks and started/stopped in index.ts. - api: `CompactionService` runs hourly via the shared scheduler (#31). For pages with > 500 log rows and no fresh session it merges `page_updates` into `ydoc_state` and deletes the merged rows in one FOR UPDATE transaction — atomic, so a mid-run crash leaves the page untouched and the next run resumes. Content is unchanged (merged state = base + all updates), so the content cache is left as-is; updated_at is deliberately not bumped. Metric log line with pages compacted / rows / bytes removed. Tests: DB-backed compaction test (content hash unchanged, below-threshold skipped, active session skipped then picked up next run, stale heartbeat ignored, idempotent); session-registry DB test (open/close, heartbeat refresh). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY	2026-07-09 08:15:55 +02:00
Claude Opus 4.8	fa7ae033b5	Add permission-revocation handling for live and offline sessions (#39) ... Revoking write access must terminate live sessions and let a user with pending offline edits export them rather than lose them silently. Backend (generic, reused by M5 grants #53): - packages/shared: POND_ACCESS_CHANGED_CHANNEL, the LISTEN/NOTIFY channel shared by api and collab. - api: PondAccessNotifier emits pg_notify(pond_access_changed, pondId) on a permission-relevant change; the single generic seam for revocation. Wired into pond soft-delete as the interim trigger (see==modify until #53). - collab: a dedicated-connection LISTEN listener (LISTEN is connection- bound, not pooled) that, on a notification, closes every open connection to the pond's open pages. Clients then reconnect and the api re-issues a token reflecting current access (downgrade to ro, or 403/404). Reconnects and re-LISTENs if its connection drops. Frontend: - use-collab-provider: a refused token (403/404) on (re)connect sets accessRevoked and stops the reconnect loop; exposes discardLocal. - AccessRevokedDialog: keeps local content visible and offers Markdown copy/download (derived from the live editor doc, so offline edits are included) and an explicit discard that clears IndexedDB. de+en strings. Tests: collab DB-backed integration test proves a direct NOTIFY closes a live session within seconds (AC1) and leaves unrelated ponds untouched; listener unit tests; api test asserts soft-delete fires the notifier; web test for the export Markdown derivation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY	2026-07-09 07:01:46 +02:00
Claude Opus 4.8	7d8f331870	Add collab document persistence hooks and content-cache refresh (#35) ... The collaboration server becomes the writer of page state (ADR 0003, realtime-collaboration.md §lifecycle): - onLoadDocument reconstructs a page's Y.Doc from PostgreSQL by applying `pages.ydoc_state` and then every `page_updates` row in order, so a page with a long update log loads correctly. - onStoreDocument persists debounced (2 s, max 30 s): it appends the delta since the last flush to `page_updates`, periodically merges the log back into `ydoc_state` (inline threshold; the session-aware compaction of idle pages remains the separate job, #40), refreshes `page_content_cache` (plain text / Markdown / HTML / outline via the shared derivation, #24), bumps `pages.updated_at`, and keeps `Attachment.pageId` pointed at the embedding page (#31). Each flush runs in one transaction and its duration is logged. - The document size ceiling (MAX_PAGE_DOCUMENT_BYTES) is enforced on store: an oversize document is not persisted and the clients are notified with a stateless error so they can revert. Persistence is an injected port (PagePersistence): the Postgres implementation is covered by a DB-backed test (store/load round-trip, content-cache refresh, a 1000-entry update log, size-ceiling rejection, not-found), and the hook wiring — two-client sync, survival across a server restart, and the size-ceiling stateless notification — by an integration test using an in-memory fake. The collab package gains its own vitest setup that provisions an isolated `_collab` test database. The REST `PUT /pages/:id/state` write path stays in place for now and is retired (410) together with switching the editor to live collaboration in #36, so the deployed editor is never left unable to save between the two deploys. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY	2026-07-08 17:06:41 +02:00
Claude Opus 4.8	d4ebcfcfbe	Add collaboration token issuance and connection authentication (#34) ... The api mints a short-lived (60 s) HS256 JWT per page open after an interim permission check; the collab server authenticates every connection with it (ADR 0003/0007 — the only JWTs in the system). - packages/shared: browser-safe token schema/types in `collab-token`, and the Node `crypto` sign/verify in `token-crypto` behind its own subpath export (`@dorfteich/shared/token-crypto`) so the web bundle never pulls in `node:crypto`. Only HS256 is produced/accepted; the signature is checked in constant time before any untrusted field is read. - api: `GET /pages/:id/collab-token` (auth-required) returns {token, mode, expiresInSeconds}; `mode` is rw/ro via the interim access service; issuance is logged at debug level without the token value. - collab: `onAuthenticate` verifies the token, checks the pageId matches the document name, stores {userId, mode} context, and enforces `ro` via Hocuspocus' read-only connection flag. Hocuspocus' own signal handling is disabled so index.ts remains the single shutdown owner. - Shared COLLAB_TOKEN_SECRET env for api + collab (compose, dev overlay, .env.example, stage docs); a dev default keeps native dev/test/CI running. Tests: shared token round-trip/rejection; api endpoint e2e (auth required, claims, 404 for non-members/unknown ids); collab integration via HocuspocusProvider (valid token connects; expired/tampered/mismatched-page/ wrong-secret rejected; read-only writes dropped, verified with two clients). Closes #34 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-08 15:52:19 +02:00
Claude Opus 4.8	8316c617d2	Add collaboration server skeleton (Hocuspocus) with health, container, and CI/CD (#33) ... Bootstrap apps/collab as a Hocuspocus WebSocket server (ADR 0003): - pino JSON logging (service=collab) and shared Zod env validation (collabEnvSchema); structured connection open/close logs. - /healthz endpoint (process liveness + PostgreSQL ping) served via the onRequest hook, matching the container-internal path and the proxied /collab/healthz path; any WebSocket handshake is accepted for now (authentication arrives with #34, persistence with #35). - Dockerfile (ESM workspace build) and a compose service on the frontend and internal networks with a healthcheck; dev overlay service and a new COLLAB_PORT variable. - CD builds, pushes, and promotes the collab image; CI builds it on PRs; the smoke suite asserts /collab/healthz through the reverse proxy. - deployment.md/stages.md: proxy routing, per-stage COLLAB_PORT, checklist. Closes #33 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-08 14:53:44 +02:00
Claude Fable 5	b16d23297e	Scaffold pnpm monorepo with lint, format, and test tooling ... pnpm workspace with apps/web, apps/api, apps/collab, and packages/shared; strict TypeScript base config, repo-wide ESLint (flat) + Prettier, Vitest per package, and root scripts lint/typecheck/test/ build. @dorfteich/shared ships a first health-response helper consumed by apps/api to prove workspace linking. Existing markdown docs are reformatted once by the new Prettier setup. Closes #1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-04 19:06:27 +02:00