25 Commits

Author	SHA1	Message	Date
Claude Opus 5	f8c241b11a	#304: custom fonts in the pickers, an admin screen, and the licence page ... The backend from #303 could store an operator's font but nothing could choose one: no list endpoint outside the Site-Admin routes, no @font-face rules for a family that only exists at runtime, and no management UI. Found while wiring it up — a real defect in #303, invisible to its tests: `fontStack` cannot tell an uploaded family from a deleted one, so the PDF exporter embedded the face and then never named it. Every export of a pond using an operator font rendered in the system font while the job reported success. Both `fontStack` call sites now take the uploaded families (`buildPdfHtml`, `pondFontVariables`); `pdf-html.test.ts` pins the regression from both sides. Verified against a real Gotenberg: with the families the PDF embeds PlayfairDisplay-Bold, without them NotoSans-Bold — that was the whole bug, in one diff of two PDFs. - `GET /fonts/custom` is readable by any signed-in user, not Site Admins only: the pickers, the licence page and the injected `@font-face` rules all need it, and gating it would have forced a second, admin-only UI. - Bundled and uploaded families are told apart by their `<optgroup>`, not by a badge — the grouping is then part of the control's semantics, so a screen reader announces it and the native mobile select keeps it. Within each source the catalog's category grouping is preserved. - The delete confirmation names how many ponds use the family and what happens to them; focus moves to it and back on cancel. Deletion stays unblocked (the api's decision, #303) — the ponds degrade, they do not break. - The licence page grew a second table. That is what makes an attribution obligation satisfiable: a commercial licence that requires naming the foundry needs a page to name it on. Verified in the browser end to end (upload two weights → listed and rendered in its own font → chosen in a pond → page renders in it → deleted → pond falls back): api suite for fonts/export 77 passed, a11y pack 11/11 locally in both schemes, lint/typecheck/i18n:check green.	2026-08-01 18:32:46 +02:00
Claude Opus 5	b96997501a	#303: operator-uploaded fonts — storage, API, PDF embedding, backup ... An operator holding a font licence could only use it by baking the file into a custom image, which tied every change to a rebuild and left the file out of the backup. ADR 0016 said there is no runtime font management. It also listed this exact case under "Alternatives considered" — *may become a Site-Admin- level feature later*. The amendment takes that option and answers the two objections it raised: licensing risk (Site Admins only, licence recorded with the family) and file-format attack surface (magic-byte check and a size cap, never a parse). - `CUSTOM_FONTS_DIR` (default `./data/fonts`) — a sibling of uploads and plugins, NOT inside the image-baked `FONTS_DIR`, where a deploy would overwrite it and no backup would ever see it. - One list of data directories (`apps/backup/src/data-dirs.ts`) now feeds both the nightly archive and the restore, so they cannot drift. #306 and #307 add one line each instead of a second mechanism. - Both Dockerfiles bake the path. The backup image sets its volume paths itself ("self-sufficient without compose env" — #71's lesson) and reads no *_DIR from compose; without the ENV entry the archive would have skipped the directory silently. - The PDF path already read WOFF2 from disk at request time, so it only had to pick the other base directory for a custom family. - `fontStack`/`fontEntry` take the instance's uploaded families as an argument — they are runtime data. The catalog is searched first, and a colliding family name is rejected at upload, so a custom font can never shadow a catalog one. - Deletion is never blocked by usage: an unknown family already falls back to the system stack, so affected ponds degrade instead of breaking. The count of affected ponds travels into the audit entry. - Audit catalogue v1.6 (`font.uploaded`, `font.deleted`). Verified: api full suite against a fresh database, 102 files / 571 tests. The upload suite writes into a real temp directory and reads the bytes back off disk, so the storage layer is exercised rather than mocked.	2026-08-01 14:49:13 +02:00
Claude Opus 5	45f1925917	#302: configurable pond start page, created with every new pond ... Opening a pond landed on whatever sorted first in the sidebar — stable, but a rule nobody could see, and one whose target moved as soon as someone added a page ahead of it. New ponds landed on the empty-pond hint instead of anything useful. - `startPageId` joins the pond settings. No migration: `Pond.settings` is already jsonb. It stores an id, not a slug, so renaming or moving the page keeps it working. - `PondHomePage` prefers it, but only when the page is in this user's page list. That list already holds just what they may see, so a start page hidden by a page-scoped grant — or trashed — falls back silently instead of landing them on a 404, and it costs no extra request. - Both creation paths give the pond a start page, titled from the creator's stored locale. It happens after the creating transaction commits: the owner's grant is written inside it and permissions cache per pond, so creating the page any earlier would ask about rights the grant has not published yet. A failure is logged, not fatal — a pond without a start page still works. `PagesModule` imported `PondsModule` without using it. Removing that vestigial edge let PondsModule depend on PagesModule in the honest direction instead of tying the two together with forwardRef. Every pond created through the api now owns a page, which broke eight suites whose teardown deleted ponds directly — `Page.pond` deliberately has no cascade, because a real purge removes contents explicitly and audits it. A shared `deletePondsWhere` helper deletes pages first. Two tests that counted pages now account for the start page rather than pretending the pond began empty.	2026-08-01 08:06:35 +02:00
Claude Fable 5	05a979bac3	#222: read-access trail for classified pages ... Instrument every full-content read channel for pages with classification = vs_nfd (ADR 0023, variant A): SPA state fetch and read rendering, public JSON content, no-JS shell, expanded embeds, public API GET (incl. the MCP read_page path and write echoes), attachment download under the #212 effective classification, all export shapes (markdown, pond ZIP, account data export, queued docx/odt/pdf at enqueue), and collab-token issuance as the api-side proxy for the WS join. Events land in the new read_events table (no FKs — evidence survives page purges and hard user deletions) with actor, session key (session:/token:/job:/anon), page, pond, channel and the classification at read time. Recording failures are NOT swallowed: a failed write aborts the read (hard failure, the deliberate contrast to AuditService — decision recorded in ADR 0023 and security.md §Logging, together with the recorded residuals: content fragments and feeds). One e2e test per channel proves both the event and its absence for unclassified pages, plus the hard-failure semantics. Refs #222. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8	2026-07-31 12:12:55 +02:00
Claude Fable 5	e505fc74dc	#212: mark attachment downloads by filename prefix and companion file ... Downloads whose effective classification is vs_nfd carry the documented VS-NfD_ filename prefix (single source classificationFilenamePrefix() in shared; ADR 0022 records the short form for file names). Effective classification: the linked page's level; an attachment with unset pageId (paste-then-insert, pond-level) FAILS CLOSED to the highest level of any live page in its pond. The pond export ZIP adds a sibling <file>.classification.txt companion with the full marking for classified media, next to the manifest entry (#210). Documented in operations.md, incl. the deliberate residual risk: the file's own content carries no marking (recorded on #231, not hidden). Tests: prefixed classified download, unchanged open download, fail-closed orphan both ways, ZIP companion + manifest level. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>	2026-07-31 07:29:16 +02:00
Claude Fable 5	68497046e9	#210: mark the Markdown ZIP export with frontmatter, imprint and manifest ... A classified page's .md carries the level in YAML frontmatter AND the marking line at top and bottom; unclassified files are byte-identical to before. Every pond archive (incl. the per-pond folders of the account data export) ships a manifest.json listing each file with its level and stating the highest level once at archive level — media inherits the highest classification among the readable pages referencing it (fail-closed). Round trip: the importer recognizes exactly our frontmatter block, strips it plus the imprint lines, and creates the page at least at the imported level (content must not escape its marking by traveling through a ZIP) — pinned by unit and e2e round-trip tests. Foreign frontmatter passes through unchanged; the Obsidian vault import keeps its own frontmatter modes. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>	2026-07-31 07:13:53 +02:00
Claude Fable 5	74a9e495e4	#209: pandoc reference documents carry the VS-NfD marking for DOCX/ODT ... reference-vs-nfd.docx/.odt ship as derived binaries: the pinned pandoc's default reference documents plus a header and footer with the marking — part of the document's page setup, so it repeats on every page in Word and LibreOffice and is not deletable body text. Source of truth is scripts/gen-classified-reference-docs.mjs (wording from shared classificationMarking(); maintenance documented in assets/README.md). The converter passes reference docs to pandoc-server via in-request files + reference-doc; the worker attaches them for marked docx/odt jobs (job option {marking}, as in #208). Unclassified exports pass nothing and are unchanged (pinned by fake-converter test). Fidelity suite asserts against real pandoc 3.6 that marked outputs carry the header/footer parts and unmarked ones do not; per-page repetition verified via LibreOffice 25.8 headless PDF (5/5 pages, 2 markings each, both formats). Word: quick manual look pending (sample files in the workspace), procedure documented in assets/README.md. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>	2026-07-31 07:09:47 +02:00
Claude Fable 5	c2df7c0c23	#208: VS-NfD marking in the Gotenberg per-page header and footer ... A classified page's PDF export carries its marking as a job option; the renderer hands it to Gotenberg's Chromium header/footer templates, so it repeats on every page — bold centered in the running header and next to the existing page numbers in the footer. Unclassified pages send exactly the pre-#208 forms (unchanged PDF, asserted by the fidelity smoke and a lastMarking=null check). New real-Gotenberg fidelity test asserts the marking appears twice on EVERY page of a multi-page render while the document-level header keeps working. Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>	2026-07-31 06:55:52 +02:00
Claude Fable 5	ff505bc752	#233: prune conversion job payloads for every job kind ... The raw input/result bytes of import/export conversion jobs were kept forever; a deleted classified page could live on inside its last export. A new daily conversion-payload-prune job nulls both once a finished (succeeded or failed) job passes conversion.payloadRetentionDays (instance setting, default 30) — the row survives for status/audit. PENDING and RUNNING rows keep their payload, so the worker's stale-lock recovery path is untouched; a hand-requeued pruned job fails finally via conversionInputOf instead of crashing the worker. The input column becomes nullable; the migration backfills by clearing payloads of jobs already finished longer ago than the default period (recent results stay downloadable until they age out). Job-count fence in system.spec: 7 -> 8 (new scheduler registration). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ	2026-07-30 22:12:32 +02:00
Claude Fable 5	0bc36aa58c	#194: orphan-file sweep, drop the unused Attachment.deletedAt ... Nightly sweep with two directions: attachments still unclaimed (pageId null) after a 24 h grace period - claimed by no collab persist, page upload, or import - are reclaimed (row, file, quota released); files on the uploads volume without a database row (drift after a crashed upload) are removed once older than the grace period. The grace period protects the paste-then-insert window. Deliberate deviation from the issue's content-reference idea, documented in schema comment and operations.md: claimed attachments whose page content no longer embeds them are NOT auto-deleted. The page attachments panel lists claimed files as user-managed objects (inserting into the document is optional there), so 'not embedded' is not 'unused' - an auto-delete would destroy panel assets. Humans clean those up in the panel or the pond file manager, which flags orphans already. Attachment.deletedAt is removed by migration - deletion is hard everywhere (sweep, purge, manual), there is no soft-delete state; the never-true deletedAt:null filters in files/export queries went with it. Refs #194 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ	2026-07-30 13:19:58 +02:00
Claude Fable 5	78a913c47b	Vault import: decode ZIP names as UTF-8 and NFC-normalize them ... Real vault ZIPs broke umlauts in page titles ("Fußball zum Götzen" → mojibake, slug fua-ball…goi-tzen): fflate honors only the ZIP UTF-8 flag, which common archivers omit, and decodes unflagged names as Latin-1. That decoding is byte-lossless, so parseVaultZip now re-reads any name whose chars all fit one byte as UTF-8 (a strict decoder — genuine Latin-1 and flag-decoded precomposed chars fall back unchanged), then NFC-normalizes: macOS zips store umlauts decomposed, which silently broke slugify's ä→ae digraphs, wikilink matching, and duplicate-basename detection. slugify itself also precomposes first as defense in depth for NFD input from other paths. Unit tests pin both cases: a hand-patched ZIP whose UTF-8 name bytes carry no UTF-8 flag, and an NFD-named note that must come out precomposed with an ueber- slug. Pages already imported with garbled titles stay as they are — delete the imported subtree and re-import after this lands (or rename by hand). Fixes #127 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fb2VzvcoBPHkjh8bZ6PzQn	2026-07-15 13:24:58 +02:00
Claude Fable 5	8ae010218e	Obsidian vault import: endpoint, job orchestration, rollback (#117) ... POST /ponds/:pondId/import/vault (pond-admin-gated; a vault import creates a subtree, uploads files, and creates labels — administration, not everyday editing) takes the ZIP plus a JSON options field {parentPageId?, labelIds?, frontmatterMode}. The archive is parsed at enqueue for fast 400s; the job (new kind import_vault, riding the existing isImportKind worker routing) re-parses and runs the #116 transform, then: containers top-down → notes (asset placeholders → uploaded pond files; non-images become page attachments) → tags to labels (nested tags build a label hierarchy via LabelsService, so locking and cache invalidation apply) plus the dialog labels. All-or-nothing: any failure hard-deletes the created pages (children first) and removes the stored files (quota restored), then surfaces as import_vault_invalid_zip / import_vault_too_large / quota_exceeded / conversion_failed — and makes the worker's retry policy safe. Supporting changes: - conversion_jobs gains a nullable options jsonb column; enqueue takes kind-specific options and a maxInputBytes override (the 25 MiB default protects the pandoc sidecar, which a vault never touches — vaults use the 64 MiB upload limit). - insertPage accepts a pre-reserved slug (the batch reserves all slugs up front against pond ∪ batch). - NEW: pages born with content seed their outgoing page_links rows (deriveContent now returns wikilinkSlugs) — imported pages would otherwise stay invisible to backlinks and the graph until their first collab save. Collab still rewrites the rows on every save, and the existing phantom resolution heals batch creation order. import-vault.e2e.db.test.ts (4 tests, real worker drained): gating + input rejection, the full fixture import (tree under a mount page, collision suffixes, link rows incl. phantom, nested tag labels, extra label everywhere, frontmatter stripped, image embedded + PDF attached), complete quota rollback, and a clean re-import with fresh suffixes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 17:13:11 +02:00
Claude Fable 5	269b36c761	Fix the gates the #116 commit skipped past ... The asset type lost its name field during the split into path+extension (tsc error), and the fixture's .obsidian/app.json needed Prettier's formatting. Lesson from the /srv move repeated within one day: the gate chain must gate — never '; echo' past a failing typecheck. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 16:53:11 +02:00
Claude Fable 5	09abda3ada	Obsidian vault transform module (#116) ... Pure functions from vault ZIP to import plan — no DB, no DI: - parseVaultZip: fflate unzip with the plugin-package protections (zip-slip rejection, incremental unpacked ceiling 256 MiB, parameterized for tests); dot-directories like .obsidian/ skipped; deterministic ordering. - extractFrontmatter: leading --- block, tags:/tag: in scalar, inline- array, and block-list forms; strip mode drops the block, preserve re-emits it as a fenced yaml code block. - extractInlineTags: fence- and inline-code-aware #tag / #nested/tag extraction and removal (headings and pure numbers untouched). - rewriteLinks: [[Name]], [[Name|Display]], [[Name#Heading]] (fragment stripped), [[folder/Name]] (path match beats basename) → the FINAL slug with the human name as display; unresolvable → slugified phantom; ![[img]] and relative  → vault-asset: placeholders the uploader resolves (#117); non-image embeds → italic filename + page attachment; SVG deliberately stays an attachment (never inline, security.md); note embeds degrade to plain wikilinks. - planFolders: folder chains merged at the deepest levels to fit MAX_PAGE_DEPTH below the mount page (merged titles read c/d). - planSlugs: -n suffixing against existing ∪ batch; duplicate basenames resolve to the lexicographically first vault path. - planVaultImport ties it together into containers + notes + the referenced-asset set. Fixture vault under fixtures/import/obsidian-vault/ (umlauts, duplicate basenames, nested tags, deep folders, embeds, code traps); 13 unit tests colocated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-14 16:51:31 +02:00
Claude Fable 5	0c85293830	Public REST API v1: personal access tokens, instance switch + per-pond opt-in (#104) ... Token-authenticated machine access at /api/public/v1 — the foundation for the built-in MCP endpoint (#105). Personal access tokens: - api_tokens table (SHA-256 hash, scope read|write, optional pond restriction, expiry, revocation, throttled last-used) + migration; secrets are dt_pat_<random>, shown exactly once - lifecycle endpoints under /users/me/api-tokens (session-only — a leaked token can never mint more tokens) with audit entries api.token_created/api.token_revoked - settings UI section (create with scope/expiry/pond restriction, one-time reveal with copy, list with status + revoke), de+en Activation (404 semantics per #60 on both levels): - instance setting api.enabled (default off, admin settings switch) - pond setting apiEnabled (default off, pond settings toggle; the PondsService settings-merge learned the key — the #92 lesson) Surface (/api/public/v1, excluded from the SPA's global prefix): - me, ponds, pages (list/read as Markdown+HTML, create from Markdown via the shared pipeline, PATCH title/content, DELETE to trash), search (permission-filtered + narrowed to exposed ponds, highlights as **…**), markdown ZIP export, labels (tree, create/rename/recolour/move/delete, assign/unassign), comments (threads, create, resolve/reopen) - content replacement travels the collab-owned document path: the new state lands as a MANUAL version "API update", then the established restore NOTIFY applies it — open editors converge, history stays append-only, no second lineage (VersionsService.replaceContent) - hand-maintained OpenAPI 3.1 document at /openapi.json, pinned to the controller by a route-coverage test in both directions Enforcement: - PublicApiGuard: instance switch → bearer PAT auth (request.user is the token's user) → per-token rate limit (429 + Retry-After) → scope (403 scope_required) → pond opt-in + token restriction - the shared PermissionGuard then applies the unchanged permission model; PageParamSource gained pondSlugParam for the slug+slug routes - no cookies anywhere → no CSRF surface (pinned by a hostile-Origin test) - every write audit-logged as api.write with the token attributed Tests/verification: - 12-test e2e pack: lifecycle, switches, permission matrix (reader/editor/outsider × scopes), restriction, page roundtrip incl. restore-NOTIFY assertion, labels, comments incl. policy, search narrowing, ZIP export, rate limit; full api suite 60/60 green (quota fixture via per-user override — never the instance default) - new collab-pack test proves an open editor converges onto an API content replacement (green against a local seeded stack) - UI smoke against the built SPA: token create/reveal/revoke, pond opt-in persists, admin switch persists (10/10) - docs/self-hosting/public-api.md + README link Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-12 11:17:03 +02:00
Claude Fable 5	5cef359b8f	Nextcloud backup target: admin-configured, manual + scheduled uploads, in-app restore (#103) ... Off-host backups for every self-hoster, configured entirely in the admin UI — supersedes the host-specific mirror plan behind #84. shared: - webdav.ts (new package entry like token-crypto): minimal WebDAV client with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT (streamed), GET, DELETE; Nextcloud DAV path derived from the plain server URL, explicit DAV bases pass through - backup-status.ts: additive remote-upload status in status.json, the restore-status.json contract (running/succeeded/failed + staleness bound), the backup_command/backup_maintenance NOTIFY channels, and the one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz) - backup-set.ts moved here from apps/backup (api lists local sets) backup sidecar: - reads the backup.* instance settings directly from the database (admin changes apply next run; local retention row overrides the env) and the app password from the secret store - after each successful set: bundle dump + files archive + manifest into ONE self-contained tar.gz, upload via WebDAV per schedule (off/daily/weekly; manual runs always upload), prune remote bundles — never the newest — and record the outcome in status.json; upload failures alert via a new backupUploadFailed mail (de+en) - command listener on backup_command (run / restore) with a serial queue against the nightly timer - restore orchestrator: restore-status.json → maintenance NOTIFY → grace → (remote: download + manifest-verify bundle) → terminate other DB connections → shared perform-restore path (same code as restore.sh) → final status + maintenance exit api: - MaintenanceGuard (global, registered before the setup gate): 503 maintenance_mode while restore-status says running; health endpoints and the new public GET /backup/restore-status stay exempt; a stale running state (crashed sidecar) unblocks after 30 min - MaintenanceStateService watches the file and restarts the api after a successful restore (fresh caches, migrate-on-start for older dumps); main.ts refuses to touch the database while a restore runs — a container restarting mid-restore must not race pg_restore with migrate deploy - worker sweeps (conversion, mail outbox, scheduler) catch transient database failures instead of dying on an unhandled rejection — the restore's connection termination crashed the api in verification - backup admin endpoints under /admin/system/backup: settings (live connection test before save, password write-only into the secret store), nextcloud/test, sets (local via the ro backups mount + remote via WebDAV), run + restore (type-to-confirm backstop, source validation) — commands travel as NOTIFY payloads; audit actions backup.settings_changed/run_triggered/restore_requested - readyz: new warning-level backup_remote check while a target is configured (26 h daily / 170 h weekly bound) collab: - maintenance listener: on enter, persist + close every live session and refuse new connections until exit (failsafe timeout 30 min) — no in-memory document may write pre-restore content back afterwards web: - Admin → System backup section: status card with remote facts and a "Back up now" button, the Nextcloud settings form with test button, and the restore picker (local + remote sets, type-to-confirm) - global maintenance screen: any 503 maintenance_mode flips the SPA to a status page polling the exempt endpoint, reloading when the instance returns Verified end-to-end against a live stack (fresh DB, native api + sidecar, fake WebDAV server): configure → test → manual backup → bundle upload → readyz/sets/status surfaces → remote restore with maintenance gate, marker rollback and api restart; suites: shared 21, backup 9, collab 11, api 58 files green, lint + i18n:check + typecheck clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-12 10:39:18 +02:00
Claude Fable 5	9e8ebfe49c	Degrade plugin content gracefully in HTML, PDF, and office exports (#79) ... Completes M7: exports and the public read view no longer show raw plugin placeholders (ADR 0008/0009). - PluginFallbackRenderer (api): replaces each plugin-block placeholder in content-cache HTML with its best static form — the block's stored SVG snapshot (block data is author-controlled, so it passes the same DOMPurify sanitizer as uploaded SVG files before entering host HTML), else the manifest fallback from the stored snapshot (text, or an image inlined as a data URI so network-isolated renderers work; tombstone-safe for uninstalled plugins), else the literal '[plugin content]' marker. - Office exports (docx/odt): the export markdown is degraded before pandoc — GFM knows neither the dorfteich-plugin fence nor the section fenced div, so blocks become their fallback text and sections plain quoted blocks (shared replacePluginNodesForExport, AST-level so nesting and embedded blocks inside sections survive). - PDF export applies the HTML fallback pass before building the Gotenberg document — resolving the TODO left in #67. - Public read view: the same fallback pass plus the pond's active section-style CSS inlined as a <style> block, so public pages show styled sections and static plugin content without any plugin runtime. - Covered in export.service.db.test (snapshot SVG sanitized — hostile <script> stripped; manifest text; tombstone text; quoted sections and no fence artifacts in the pandoc input) and shared export-fallbacks tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-11 14:13:09 +02:00
Claude Fable 5	e32f961047	Complete section-style plugins: CSS gate, injection, picker, export (#75) ... Second half of #75 on top of the section node (2e96173/784f21d): - Install gate for section_style CSS (plugin-css.ts): every rule must be scoped under one of the plugin's own .dt-style-<pluginId>-<styleId> classes (enforced, not rewritten — grouping at-rules checked inside, @font-face/@keyframes exempt, statement at-rules rejected); positioning out of the content flow (anything but static/relative) is rejected as an overlay vector; "</style" is rejected as a breakout vector for inlined embedding. Hostile fixtures from the acceptance list are pinned in plugin-css.test.ts. - Web: usePondPlugins loads the pond's active plugins once per visit; SectionStyleSheets links each active style plugin's immutable styles.css; SectionStyleMenu (toolbar) wraps/restyles/unwraps with a picker fed from the plugins' i18n titles. Sections show a faint dashed hint while editing so unstyled (plugin-disabled) sections stay findable. - PDF export: PluginsService.sectionStyleCssForPond inlines the pond's active section-style CSS into the Gotenberg HTML, so styled sections survive the network-isolated render; covered in export.service.db.test. - Reference plugin packages/plugins/section-styles-basic (callout, info, warning, colored-box; theme-neutral semi-transparent backgrounds), a workspace package whose tests validate it against the SDK schema and whose real files run through the api install gate. - e2e section-styles.spec.ts: install → wrap → computed background in edit and read mode → unwrap → neutral fallback after disabling the plugin. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-11 11:43:36 +02:00
Claude Opus 4.8	aaa9a253ae	Add import/export fidelity gate to CI (#69) ... Make the "structure-true best effort" fidelity contract (ADR 0009) an objective, pipeline-gated suite so "best effort" cannot erode silently. - New CI job "Import/export fidelity gate" (.gitea/workflows/ci.yml) runs the corpus suites against the pinned sidecar images the stages use (pandoc/core:3.6, gotenberg/gotenberg:8), started via docker run and reached over the host gateway. Small and separate so it stays well under five minutes; the suites self-skip in the main checks job (no sidecars). - Export fidelity: fixtures/export corpus + gen-export-fixtures.mjs + export.fidelity.test.ts — exports Markdown to docx/odt through the real pinned pandoc and reads it back, snapshotting the round trip so a writer drift (ours or a version bump) fails the gate. - PDF smoke: pdf.fidelity.test.ts renders a page through real Gotenberg and asserts the extracted text and a sane page count (pdf-parse, dev-only). - Fidelity contract doc: fixtures/README.md defines "corpus green = fidelity acceptable" and the fixture-first bug process; per-corpus READMEs updated. Because the snapshots are byte-exact and generated with the pinned tools, bumping a sidecar without regenerating shifts the output and fails the suite (AC3). The import corpus (#63) is folded into the same gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-10 13:59:10 +02:00
Claude Opus 4.8	462eca9699	Add self-service GDPR data export (#68) ... A signed-in account can export all of its own data — profile, a list of its memberships/grants, and the Markdown of its personal pond plus the shared ponds it owns — as one ZIP. Foreign content never appears: only owned ponds are bundled and the per-page read filter (reused from #65) runs for each. - Reuse the conversion-job queue as the async carrier: a `data_export` job whose worker branch resolves DataExportService via a token (no DI cycle), builds the ZIP, and stores it with an `expiresAt`. The download link 404s past expiry and an hourly scheduled purge drops the bytes (data minimization, security.md §Privacy). - Extract ExportService.appendPondMarkdown so the pond ZIP (#65) and the data export share one read-filtered pond archiver. - Rate-limit requests per account (RateLimitService); POST /users/me/data-export enqueues, GET /jobs/:id(/result) poll/download. - Settings UI "Export my data" (de+en); web share pollJob/downloadJobResult between the document and data export hooks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-10 13:01:51 +02:00
Claude Opus 4.8	8a68ef68e7	Add PDF export via Gotenberg (#67) ... Server-side PDF export for reading/sharing (ADR 0009), rendered by a new internal Gotenberg (headless Chromium) sidecar. - Sidecar: `gotenberg/gotenberg:8` in the compose stack (internal, pinned, healthcheck); api `GOTENBERG_URL` env; a `renderer` readyz check at warning-level (mirrors the converter) so PDF export degrades gracefully when Gotenberg is down without failing readyz. - Export HTML: `buildPdfHtml` renders a self-contained document (no app chrome) — the page's content with images inlined as data URIs, the pond's fonts inlined as base64 `@font-face` + applied via CSS variables (ADR 0016), print CSS (A4, page-break rules, a title header), and page numbers from Gotenberg's footer. Plugin-block fallbacks are a marked TODO(#79) for M7. - Fonts in the api image: the api Dockerfile now bakes the font catalog in (`build-fonts.mjs` with FONTS_OUT) so the exporter can read a pond's chosen WOFF2 and inline them; a missing file falls back to the system stack. - Job flow: `POST /pages/:id/export {format: pdf}` builds the HTML (read permission checked by the guard) and enqueues an `export_pdf` job on the #62 queue with the HTML as input; the worker branches `to === 'pdf'` to the `GotenbergRenderer` (html → pdf) instead of pandoc, retrying an unreachable sidecar and failing a refused render (`renderer_unavailable`/`render_failed`, de+en). The client polls and downloads `GET /jobs/:id/result`. - Frontend: the page-menu PDF button is now a real export (PDF added to EXPORT_FORMATS; the disabled placeholder removed). - Tests: export.service.db PDF cases (HTML has title/font-variable/inlined image; renderer-down fails with `render_failed`); e2e PDF export self-skips without a Gotenberg sidecar (like the .docx case). Verified locally against real Gotenberg — a valid PDF with the pond font embedded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-10 12:11:12 +02:00
Claude Opus 4.8	699c003d04	Add pond ZIP + per-page docx/odt export (#65) ... Two export paths, both permission-aware (permissions.md): - `GET /ponds/:id/export/markdown` streams a ZIP of the pond's readable pages as Markdown (one `<slug>.md` per page, a `media/` directory, wikilinks rewritten to relative `[text](slug.md)` links, image sources to `media/<id>.<ext>`). The `reader` guard is "may see the pond"; the service filters to the pages the requester may actually read, so a label-restricted reader gets only their slice. Media is appended as read streams and pages as small strings, so memory stays bounded for a large pond (500-page test). - `POST /pages/:id/export {format: docx|odt}` enqueues a `markdown → pandoc → file` conversion job (the #62 queue): embedded images are inlined as data URIs so the sidecar embeds them, wikilinks flatten to text. The client polls `GET /jobs/:id` and downloads `GET /jobs/:id/result`. Frontend: office-export buttons in the page menu (`.docx`/`.odt` run the job and download the result; PDF is a disabled placeholder for Gotenberg, #67) and a "Download pond as ZIP" link in pond settings. New `export` i18n namespace (de+en). Markdown copy/download stay as-is (#30). Robustness: the pond ZIP skips an attachment whose bytes are missing on disk (data drift) rather than letting an unhandled read-stream error crash the api; `FileStorageService.exists` gates inclusion, with a defensive stream error handler. The per-page export drops an unreadable image the same way. - shared: EXPORT_FORMATS + pageExportInputSchema; export-markdown transform helpers (image/wikilink rewrites, MIME→extension). - deps: archiver (streaming ZIP; v7 for CommonJS compat), fflate (dev, reads ZIPs in tests). - tests: export-markdown unit + export.service.db (ZIP contents & relative links, label-restricted omission, docx job with inlined images, 500-page streaming, missing-media skip); e2e export pack (ZIP download; `.docx` self-skips without a pandoc sidecar, as in the import pack, #64). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-10 10:31:19 +02:00
Claude Opus 4.8	e2f942c0ff	Add document import UI in the sidebar (#64) ... An "Import document" action in the pond sidebar: pick a .docx/.odt/.md file (or several), upload with per-file progress, and open the new page. A .docx/.odt polls the conversion job (queued → converting → done); a .md imports directly and comes back already succeeded. Failures stay listed with the localized error and a retry; concurrent imports all complete and appear. - web apps/web/src/import/: useImport hook (upload via apiUploadFile → poll GET /jobs/:id → resolve the page slug → navigate; first success of a batch navigates, every success refreshes the sidebar) and ImportControl (hidden file input, accept from shared IMPORT_EXTENSIONS, per-file status list). Wired into Sidebar next to "new page"; `import` i18n namespace (de+en). - api: ImportService accepts .md/.markdown and imports in-process (no job), returning a succeeded ConversionJobView with the created resultPageId ("Markdown imports directly"); the media+parse+create tail is now shared between the job path and the sync path (createPageFromMarkdown), and a conversion error on the sync path maps to an HTTP status. shared IMPORT_EXTENSIONS gains md/markdown. - e2e apps/web/e2e/import.spec.ts + CI step: .docx corpus fixture opens the converted page (self-skips without a reachable pandoc sidecar — CI's e2e stack has none, same as #63; verified locally + on stage), .md opens directly, an unsupported .txt shows the localized error with no page created, and two concurrent .md imports both complete. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-10 09:38:31 +02:00
Claude Opus 4.8	546e8279ac	Import .docx and .odt documents as new pages (#63) ... Uploading a Word/OpenOffice document to POST /ponds/:id/import enqueues a conversion job (the #62 queue) that produces a new page in the pond; the client polls GET /jobs/:id for the created resultPageId. Pipeline (ImportService, ADR 0009): pandoc-server is stateless and hands back a document's media no other way, so we convert in two passes — docx/odt → html with embed-resources inlines every image as a data: URI, then html → gfm produces clean structural Markdown with those data URIs still inline. Embedded images are stored as pond files (with quota accounting) and their references rewritten to file ids on the Markdown text before parsing (the editor parser only admits png/jpeg/gif/webp data URIs); an image whose bytes the upload pipeline rejects is dropped, not fatal. The title comes from a leading top-level heading (removed from the body) else the file name. The page is created from the resulting Yjs state. The shared conversion worker routes import-kind jobs to the pipeline via a token (breaking a module cycle), so import inherits the queue's locking, retry, and restart-survival. Media stored during a failed attempt is rolled back; a pond that runs out of storage fails the job with quota_exceeded. - schema: ConversionJob gains pond_id / source_name / result_page_id (migration 20260710041215_import_pages_conversion); ConversionJobView gains resultPageId. - PagesService.createWithState / yjs-content docToState build a page from a prepared document; FilesService.linkAttachmentsToPage links import media. - fixtures/import/: representative .docx/.odt corpus (headings, lists, nested lists, tables, images, links, bold/italic) with expected-Markdown snapshots; scripts/gen-import-fixtures.mjs regenerates them. - tests: import.service.db.test.ts drives the full pipeline with a fake converter (CI); import.fixtures.test.ts runs the real two-pass conversion over the corpus and a 50-page timing check against a reachable sidecar. - i18n: import_unsupported_format (de+en). Limits documented (25 MiB input, 60 s per pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-10 07:34:43 +02:00
Claude Opus 4.8	4755c18ef5	Add conversion job queue and pandoc sidecar integration (#62) ... Import/export conversions run asynchronously against an internal pandoc-server sidecar with limits and graceful failure (ADR 0009). This is the plumbing; the import (#63) and export (#65) features enqueue jobs onto it. Sidecar & config: - pandoc/core:3.6 in HTTP server mode added to the Compose stack, internal network only, with a wget healthcheck on /version; the api depends on it healthy and reaches it via the new PANDOC_URL env (default http://pandoc:3030). - readyz gains a warning-level `converter` check: an unreachable sidecar degrades import/export but never flips the instance to unready (new `warn` status on ReadinessCheck). Conversion flow (apps/api/src/import-export/): - ConversionJob table (per-request work queue, distinct from the name-keyed maintenance Job table): owner, formats, input/result bytes, status, attempts, lockedAt. Migration + owner cascade. - PandocConverter (abstract) + PandocServerConverter: POST / with {text,from,to,standalone}; binary input formats (docx/odt/…) are base64-encoded in `text`; 60 s AbortController timeout; input/output size caps. Failures map to distinct localized codes — converter_unavailable / converter_timeout (retryable) and conversion_failed (final). - ConversionWorker: claims one job at a time with `FOR UPDATE SKIP LOCKED` (safe against overlapping sweeps and a second process), recovers a stale RUNNING lock, retries transient failures up to 3 attempts then fails. A 2 s sweep plus wake-on-enqueue means a queued job survives an API restart. - ConversionJobService.enqueue (size-limited) + owner-scoped GET /jobs/:id (poll) and GET /jobs/:id/result (stream the output); a foreign/unknown id is 404. ConversionJobView in @dorfteich/shared. Tests: - conversion-job.e2e.db.test.ts (fake converter injected via a new createTestApp override hook): enqueue→convert→poll→result; foreign/unknown job 404; a persisted PENDING job picked up by a fresh app's worker (restart survival); sidecar-down fails after 3 retries while the API stays healthy. - pandoc.converter.test.ts: success, non-200→conversion_failed, refused→ converter_unavailable, and a delay-injecting server→converter_timeout. - Verified locally against a real pandoc/core:3.6 container: markdown→html, markdown→docx (valid PK/OOXML bytes), and a docx→markdown round-trip. Local: typecheck, lint, i18n:check, build all green; api 193 tests (9 new), shared 121, web 50. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1	2026-07-10 04:06:27 +02:00