Compare commits

...

88 Commits

Author SHA1 Message Date
cc9c70287c Ship third-party license texts in plugin ZIPs (#345)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m51s
CI / Build container images (pull_request) Successful in 1m13s
CI / Auth e2e pack (pull_request) Successful in 9m28s
CI / Import/export fidelity gate (pull_request) Successful in 54s
CD / Build and push images (push) Successful in 15s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m57s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 9m7s
CI / Import/export fidelity gate (push) Successful in 57s
Restore drill / Restore the latest backup into a scratch stack (push) Failing after 17s
The drawio, excalidraw, and mermaid plugin packages redistribute
third-party material (the draw.io webapp, the Excalidraw editor and its
fonts, mermaid and its dependency tree) without the license texts their
licenses require. Every affected ZIP now carries a licenses/ directory:

- licenses/THIRD-PARTY-NOTICES.txt is generated from the esbuild
  metafile (packages/plugins/third-party-licenses.mjs), so the notice
  list is derived from what actually lands in plugin.js and cannot
  drift the way a hand-maintained list would.
- drawio additionally extracts the upstream LICENSE from the pinned
  release tarball (Apache-2.0 requires the text with redistribution);
  the extraction guard also heals vendor/ caches from before this
  change. The CI fast path (no vendor fetch, no ZIP) is unchanged.
- excalidraw additionally commits curated texts (MIT for Excalidraw,
  per-font OFL-1.1/MIT with each font's own copyright statement, plus
  a FONT-NOTICES.md attribution table), because neither the npm
  package nor upstream ships any license files for them.

The api-side package validator accepts additional ZIP entries, so
installed plugins are unaffected beyond the new files.

Closes #345

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aoPvnakfBP28nAfijgUY9
2026-08-16 19:00:26 +02:00
f142289813 Recognize Markdown tables on paste and while typing (#339)
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 7m40s
CI / Build container images (pull_request) Successful in 1m21s
CI / Auth e2e pack (pull_request) Successful in 9m30s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 16s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m21s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 6m55s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 9m19s
CI / Import/export fidelity gate (push) Successful in 57s
Release / Build release images and notes (push) Successful in 2m44s
Release / Release-candidate operations QA (push) Successful in 42s
Prod deploy / Deploy the released images to Prod (push) Successful in 15s
The paste conversion (issue #30) already handled tables, but any
text/html flavor on the clipboard bypassed it. Code editors (VS Code
with copyWithSyntaxHighlighting) ship the plain text a second time as
styled div/span HTML, so a Markdown table copied there arrived verbatim
while the same text from a plain editor converted fine. Clipboard HTML
without a single structural element (table/list/heading/link/emphasis/
code...) is now treated as equivalent to the plain text; anything from a
rich-text source keeps going through ProseMirror's HTML paste.

Pasting inside a code block never converts anymore -- text is code
there, and the conversion would have split the block around rich nodes.

Hand-typed tables: pressing Enter at the end of a GFM separator row
whose previous sibling is a pipe row replaces the two paragraphs with a
real table (input rules cannot express this -- they see only one
textblock). Conversion is refused inside existing tables; body rows are
then typed cell-wise, with Tab appending rows (#338).

Closes #339

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aoPvnakfBP28nAfijgUY9
2026-08-15 21:32:25 +02:00
c17ab41a33 Word-style Tab navigation in tables with an accessible exit (#338)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 7m39s
CI / Build container images (pull_request) Successful in 4m32s
CI / Auth e2e pack (pull_request) Successful in 10m11s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Successful in 24s
CD / Smoke tests against Test (push) Has been cancelled
CD / Promote to Int (push) Has been skipped
CD / Deploy to Test (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
Tab used to fall through to the browser's focus navigation everywhere.
Inside tables it now moves cell-wise (Shift-Tab backwards) and appends a
new row from the last cell, Word-style. Outside tables every branch
returns false, so Tab keeps leaving the editor.

Capturing Tab inside tables needs a documented way out (WCAG 2.1.2):
Escape places the cursor after the table -- unlike the arrow keys, which
reach the gap cursor (#335) only from the table's edge cells, it works
from every cell, including from a cell selection. When no textblock
follows the table it falls back to the gap cursor position. The
mechanism is announced to assistive tech via an aria-describedby hint
on the editor surface (visually hidden, de+en).

e2e: cell round trip per Tab/Shift-Tab with typed markers, row append
from the last cell, and the full keyboard-only exit (Escape, then Tab
leaves the editor). The table specs now settle briefly after the insert
-- right after it the collab sync can swallow a click's selection
update, which had the markers landing in stale selections.

Closes #338

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aoPvnakfBP28nAfijgUY9
2026-08-15 21:32:24 +02:00
3bf9363c34 Merge and split table cells (#337)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 7m18s
CI / Build container images (pull_request) Successful in 4m41s
CI / Auth e2e pack (pull_request) Successful in 10m4s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Successful in 33s
CI / Lint, typecheck, test (push) Has been cancelled
prosemirror-tables already ships mergeCells/splitCell and the schema
(tableNodes) already carries colspan/rowspan -- only the controls were
missing. Adds the two commands, toolbar buttons whose enabled state
follows the selection (merge needs a multi-cell selection, split a
merged cell), and de+en labels.

Both render paths now carry the spans: docToHtml emits colspan/rowspan
(read mode, exports via the HTML path), and the markdown serializer pads
a colspan with empty cells so every row keeps the table's column count
-- rowspan stays lossy there, GFM cannot express it.

e2e drives merge and split through the toolbar; the cell selection is
made per Shift+Click because a keypress in the same tick as the
preceding click races the editor's post-click rendering (keyboard cell
selection itself works, verified interactively with a settled editor).

Closes #337

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aoPvnakfBP28nAfijgUY9
2026-08-15 21:32:14 +02:00
b1165a37e6 Unambiguous delete row/column toolbar icons (#336)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m52s
CI / Build container images (pull_request) Successful in 1m12s
CI / Auth e2e pack (pull_request) Successful in 9m24s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
The delete buttons paired the minus-box with a double arrow (bidirectional
arrows next to the symbol) which reads as "resize/expand", not "delete".
Replace them with axis stripes plus the x delete marker that deleteTable
already established: vertical stripes with x for delete column, horizontal
stripes with x for delete row. Labels/tooltips were correct all along and
stay unchanged.

Closes #336

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aoPvnakfBP28nAfijgUY9
2026-08-15 20:44:37 +02:00
69563348ca Gap cursor for block-edge positions (#335)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m52s
CI / Build container images (pull_request) Successful in 1m13s
CI / Auth e2e pack (pull_request) Successful in 9m25s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CI / Lint, typecheck, test (push) Has been cancelled
CD / Build and push images (push) Has been cancelled
A table (or any other block node without a text position of its own) as
the page's first, last, or only block was unreachable from before/after:
neither mouse nor arrow keys could place the cursor there, so no
paragraph could be created around it.

- add the prosemirror-gapcursor plugin as a TipTap extension (via
  @tiptap/pm, no new dependency; schema-neutral, so the editorSchema
  drift fence is unaffected)
- style the gap cursor bar in base.css -- the upstream package does not
  ship its stylesheet through our import path; the blink animation
  honors prefers-reduced-motion
- e2e: keyboard-only round trip that creates paragraphs before and
  after a lone table

Closes #335

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012aoPvnakfBP28nAfijgUY9
2026-08-15 20:41:23 +02:00
7e17a2dba6 settings-nav fence: 10 sections since the invitations section (#332)
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m52s
CI / Auth e2e pack (pull_request) Successful in 9m36s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CI / Build container images (pull_request) Successful in 1m14s
CD / Build and push images (push) Successful in 16s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m26s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 7m0s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 9m19s
CI / Import/export fidelity gate (push) Successful in 59s
Release / Build release images and notes (push) Successful in 2m54s
Release / Release-candidate operations QA (push) Successful in 49s
Prod deploy / Deploy the released images to Prod (push) Successful in 19s
2026-08-05 13:06:32 +02:00
c2a4dde5cc Invitation flow with per-user quota (#332)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m50s
CI / Build container images (pull_request) Successful in 3m57s
CI / Auth e2e pack (pull_request) Failing after 6m18s
CI / Import/export fidelity gate (pull_request) Has been skipped
Any authenticated user can invite an e-mail address; the mailed
single-use token lets exactly one signup through even while
registration is closed. Open (pending, unexpired) invitations count
against the new instance setting invitations.maxOpenPerUser (default 5,
0 disables inviting) — plus a 20/day per-user rate limit so a
revoke-and-recreate loop cannot become a mail cannon. Only the SHA-256
token hash is stored (auth-tokens pattern); a failed signup (taken
username) un-redeems the token so the invitee can retry.

Surfaces: invitations section in the user settings (list, invite,
revoke, quota line; wide table in a focusable .table-scroll region),
signup page reads ?invitation=<token> (preview banner, e-mail prefill,
closed-mode gate opens only for a previewed-valid token), admin general
card gets the quota field (flat RHF name per #322; VS-NfD marked and
hideable).

Governance: audit actions invitation.created/revoked/accepted
(catalogue 1.10), VS-NfD profile entry (compliant: 0) + hardening-guide
row, i18n de+en including the invitation mail template.

Tests: api e2e-db (mail link, closed-mode single-use signup with
un-redeem on failure, quota + revoke frees slot, quota 0 = 403, auth
matrix), new web e2e pack invitations.spec.ts (full UI loop through
Mailpit, wired into ci.yml with its own rate-limit reset), a11y scan
waits for the new section. Full api suite (107 files / 607 tests),
auth/admin-settings/a11y packs green against a fresh local stack.

Closes #332
2026-08-05 12:44:20 +02:00
9cf7b85b93 Admin can create user accounts directly (#331)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m47s
CI / Build container images (pull_request) Successful in 3m59s
CI / Auth e2e pack (pull_request) Successful in 9m7s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CI / Lint, typecheck, test (push) Has been cancelled
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
POST /admin/users (Site-Admin guard) creates an account with the same
field rules as self-registration, but active immediately: the admin
vouches for the address, so the e-mail is marked verified and the
personal pond is provisioned exactly like the verify-email path does
(markEmailVerified alone would skip the pond).

The user manager gains a create dialog (useModalFocus/useDismissable,
Field wiring, flat RHF field names per the #322 lesson). New audit
action user.created_by_admin, catalogue bumped to 1.9.

Tests: api e2e-db (create + immediate login + personal pond, duplicate
username 409, non-admin 403), web e2e through the dialog, and the
admin a11y scan now opens the dialog too. Both packs verified locally
against a fresh stack.

Closes #331
2026-08-05 12:23:01 +02:00
64f2deb40f Quota override cell stays a table cell, flex on an inner wrapper (#329)
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 7m54s
CI / Build container images (pull_request) Successful in 1m25s
CI / Auth e2e pack (pull_request) Successful in 9m27s
CI / Import/export fidelity gate (pull_request) Successful in 55s
CD / Build and push images (push) Successful in 25s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m20s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m53s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 9m0s
CI / Import/export fidelity gate (push) Successful in 59s
Same defect and same fix as the user list's actions cell (#177):
display:flex directly on the override td removed its table-cell
behaviour, so the cell stopped growing to row height and its bottom
border no longer met the row's — visibly uneven separator lines
(Stefan's screenshot from the self-hosting walkthrough). The flex
layout now lives on .quota-row__override-inner.

Measured locally like #177: bottom-delta across all cells of every
quota row was 24–49 px before, 0 px after (override set, so the cell
carries input + two link buttons); admin-quotas e2e pack green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aviRTgWCcAHUh1SBoxf6P
2026-08-04 12:44:17 +02:00
20677ea247 Self-hosting findings: URL-safe password advice, operator-readable pre-seed errors (#324, #325)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 7m58s
CI / Build container images (pull_request) Successful in 1m11s
CI / Auth e2e pack (pull_request) Successful in 9m12s
CI / Import/export fidelity gate (pull_request) Successful in 59s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
CD / Promote to Int (push) Blocked by required conditions
Two findings from Stefan's manual clean install per the guide, both
ending in an api restart loop that was hard to diagnose:

- #324: the guide recommended `openssl rand -base64 32` for
  POSTGRES_PASSWORD, but the compose interpolates the password unescaped
  into DATABASE_URL — base64's `/`, `+`, `=` break the URL. Misleadingly,
  db stays healthy (it gets the password as a plain env var) while
  api/collab/backup crash. Guide and .env.example now recommend
  `openssl rand -hex 24` for both secrets and say why; Troubleshooting
  gained the symptom line.
- #325: SETUP_ADMIN_PASSWORD's minimum (10 chars,
  packages/shared/src/auth.ts) was undocumented, and a violation crashed
  the boot with a raw ZodError naming schema fields and i18n keys.
  Failing the boot stays — deliberately, no half-seeded instance — but
  preseedFromEnv now translates validation errors into operator terms
  ("Pre-seeding failed: SETUP_ADMIN_PASSWORD must be at least 10
  characters. Fix .env and recreate the api container."). Documented in
  the guide's first-run section, .env.example, and Troubleshooting; new
  test pins the message and that nothing is half-seeded afterwards.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aviRTgWCcAHUh1SBoxf6P
2026-08-04 12:44:16 +02:00
6999b3dd73 Document title follows the configured instance name (#323)
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m58s
CI / Build container images (pull_request) Successful in 1m26s
CI / Auth e2e pack (pull_request) Successful in 9m7s
CI / Import/export fidelity gate (pull_request) Successful in 1m2s
CD / Build and push images (push) Successful in 13s
CD / Deploy to Test (push) Successful in 14s
CI / Lint, typecheck, test (push) Successful in 7m31s
CI / Build container images (push) Has been skipped
CD / Smoke tests against Test (push) Successful in 1m24s
CD / Promote to Int (push) Successful in 13s
CI / Auth e2e pack (push) Successful in 9m25s
CI / Import/export fidelity gate (push) Successful in 55s
useDocumentTitle pinned APP_NAME = 'Dorfteich', so every route title —
tab, bookmarks, the window title a screen reader announces (WCAG 2.4.2)
— named the product instead of the operator's instance. The trailing
name now comes from the public branding query, exactly like the TopBar
brand (#306); until the query resolves (or when it cannot, e.g.
maintenance mode) the shipped default keeps the title stable, so an
untouched instance reads exactly as before. The static index.html title
stays the pre-JS placeholder — server-rendering it is #179's territory,
deliberately out of scope (recorded in the issue).

The admin-settings e2e now also asserts the title carries the new name
right after saving, without a reload.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aviRTgWCcAHUh1SBoxf6P
2026-08-04 11:43:05 +02:00
4d6a27194f Follow the field rename in vs-nfd-marking locators
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 7m0s
CI / Build container images (pull_request) Successful in 1m23s
CI / Auth e2e pack (pull_request) Successful in 9m0s
CI / Import/export fidelity gate (pull_request) Successful in 1m1s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CI / Lint, typecheck, test (push) Has been cancelled
CD / Build and push images (push) Has been cancelled
The pack addresses the registration-mode select by its DOM name
attribute, which react-hook-form derives from the field name — now
`registrationMode` (dot-free, see admin-settings-form.ts). Caught by CI
run 713; the pack needs VS_NFD_MODE stages and was not part of the local
verification set.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aviRTgWCcAHUh1SBoxf6P
2026-08-04 11:42:55 +02:00
9f754649d4 Fix admin general settings form: dot-free field names, flat PATCH keys (#322)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m42s
CI / Build container images (pull_request) Successful in 1m20s
CI / Auth e2e pack (pull_request) Failing after 10m3s
CI / Import/export fidelity gate (pull_request) Has been skipped
The general and quota cards registered their react-hook-form fields under
the dotted settings keys. RHF treats dots as nested-path separators, so
the form DISPLAYED fine (its getter falls back to the literal flat key)
but typing nested the value ({ instance: { name } }) and the api's strict
PATCH schema rejected the body — none of these fields ever saved through
the UI, on any instance. Found by Stefan on a fresh self-hosted install.

- admin-settings-form.ts: dot-free form model with one explicit mapping
  to the dotted settings keys and converters in both directions; the
  submit now also carries ONLY the settings these cards edit, so the
  internal branding metadata keys never ride along.
- Saving invalidates the branding query too — the TopBar reads the
  instance name from it and kept the old name until its staleTime ran out.
- admin-settings.spec.ts (new e2e pack, registered in ci.yml): drives the
  rename THROUGH THE FORM — success message, TopBar update without
  reload, value survives reload, api returns it. Verified locally to fail
  against the unfixed page and pass against the fix. Every existing
  admin-settings test patched the api directly, which is why this bug was
  invisible to CI.
- admin-settings-form.test.ts pins that no form field name contains a dot
  and the mapping round-trips.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aviRTgWCcAHUh1SBoxf6P
2026-08-04 11:18:26 +02:00
d2be1116bc Refresh the self-hosting guide for the first public release (#320)
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m43s
CI / Build container images (pull_request) Successful in 1m13s
CI / Auth e2e pack (pull_request) Successful in 8m52s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Build and push images (push) Successful in 16s
CD / Deploy to Test (push) Successful in 17s
CD / Smoke tests against Test (push) Successful in 1m25s
CD / Promote to Int (push) Successful in 13s
CI / Lint, typecheck, test (push) Successful in 6m54s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m42s
CI / Import/export fidelity gate (push) Successful in 58s
- TAG guidance points at pinned release tags (e.g. v0.14.0) instead of
  the pre-release `test` tag; concrete curl commands fetch the three
  reference files.
- Backup wording (guide + .env.example) names all four data volumes in
  the restore set (uploads, plugins, custom fonts, branding).
- Updating section states the back-up-first step and links the update
  runbook.
- Pass the external-authentication variables (OIDC_*, AUTH_LOCAL_ENABLED,
  AUTH_PROXY_*) through the reference compose and document them in
  .env.example: they were documented in security.md but unreachable from
  .env. Empty values count as unset (app-config.service.ts), so the block
  is inert until configured.
- New guide section "External authentication (optional)"; neutral
  APP_BASE_URL example.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aviRTgWCcAHUh1SBoxf6P
2026-08-03 12:34:30 +02:00
78258c4f9b test: give the 10k-iteration sort-key property test its own timeout
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m45s
CI / Build container images (pull_request) Successful in 2m51s
CI / Auth e2e pack (pull_request) Successful in 8m54s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Build and push images (push) Successful in 15s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 13s
CI / Lint, typecheck, test (push) Successful in 6m52s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m43s
Release / Build release images and notes (push) Successful in 2m48s
CI / Import/export fidelity gate (push) Successful in 56s
Release / Release-candidate operations QA (push) Successful in 56s
Prod deploy / Deploy the released images to Prod (push) Successful in 17s
Under parallel CI load the test repeatedly exceeded the default 5000 ms
per-test timeout (run 685 on main, run 699 on an unrelated PR); the
identical test passed on rerun. Locally it finishes in about 1.3 s, so
30 s is generous headroom, not a mask for a regression.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017aviRTgWCcAHUh1SBoxf6P
2026-08-02 14:01:14 +02:00
a327126fac #307: pond-level branding overrides the instance logo and favicon
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 7m8s
CI / Build container images (pull_request) Successful in 4m3s
CI / Auth e2e pack (pull_request) Successful in 9m1s
CI / Import/export fidelity gate (pull_request) Successful in 1m3s
CD / Build and push images (push) Successful in 16s
CD / Deploy to Test (push) Successful in 17s
CD / Smoke tests against Test (push) Successful in 1m21s
CD / Promote to Int (push) Successful in 13s
CI / Lint, typecheck, test (push) Successful in 6m49s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m42s
CI / Import/export fidelity gate (push) Successful in 58s
Built on #306's storage, serving and crop control — a layer, not a parallel
implementation. `resolveBranding` in shared is the ONE place that answers
"which asset applies here?", and both the sidebar logo and the favicon swap
read it.

The decision most likely to be "fixed" by accident, so it is pinned by name
in `branding.test.ts`: **a logo set belongs to one level and variants are
never mixed across levels.** A pond that uploaded only a light logo shows THAT
logo in dark mode; it does not borrow the instance's dark variant. Decided
2026-08-01 — a logo silently swapping to a different image when the viewer
switches theme is a change nobody ordered, and a design that looks wrong is
more honest than one that is quietly substituted. Only a pond with no logo at
all inherits the instance's set, again as a set. The settings screen warns
about a missing dark variant; it never blocks.

Consequences that fall out of that rule and are easy to get wrong:

- The serving route does NOT fall back when given a pond scope. The caller
  already decided which level applies; a "helpful" fallback in the route
  would mix variants across levels behind the resolver's back.
- The logo link's accessible name follows the LEVEL: a pond logo is named by
  the pond, an instance logo by the instance. It is the link home, and a
  link's name has to say where it goes.

- **Charged to the pond's storage quota**, before the write, like attachments.
  Without it branding would be a way around the quota, and replacing a logo
  repeatedly would consume disk with no ceiling. The replaced asset's bytes
  are released FIRST, so re-uploading the same logo costs nothing — and a
  refused upload puts the released reservation back, so a rejection cannot
  leave the pond with more room than it had.
- **Purge removes the branding files.** The purge standard is absolute: after
  it nothing referencing the pond survives, rows or files. Asserted against
  the real purge path, not the new code alone.
- Security unchanged from #306 and not relaxed because the uploader is now an
  ordinary Pond Admin: SVG refused, magic bytes and IHDR checked server-side,
  size caps, content type pinned, no image parsing.
- The favicon swap is driven by the RESOLVED pond, never the raw route
  parameter — an unreadable or unknown slug must not leave a stale icon in
  the tab. That it happens after first paint is accepted and stated in the
  code and the UI: avoiding it would mean server-rendering index.html, which
  is #179's territory.

Same audit id as #306 (`branding.changed`) with `scope: 'pond'` — the catalogue
already carries the field, so no version bump.

Verified: api suite 105 files / 592 tests green; 5 pond-branding e2e tests
(pond scope serves the pond's bytes while the instance level still 404s, the
quota is charged and released exactly, SVG refused at pond level, a reader may
read but not change, purge deletes the files); 9 shared unit tests on the
resolution order including both mixing directions.
2026-08-01 20:44:02 +02:00
3310ae3926 #305: a full pond archive before deletion and before purge
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 7m19s
CI / Build container images (pull_request) Successful in 1m23s
CI / Auth e2e pack (pull_request) Successful in 8m55s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Build and push images (push) Successful in 14s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m28s
CD / Promote to Int (push) Successful in 13s
CI / Lint, typecheck, test (push) Successful in 6m52s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 9m6s
CI / Import/export fidelity gate (push) Successful in 58s
Deleting a pond already had a strict prompt — typing the pond name, stricter
than a confirm dialog. That was never the gap. The gap is that the person who
deletes it loses access the moment they do: the pond leaves their view, only a
Site Admin can bring it back, and the export is no longer reachable for them.
So the archive is offered INSIDE the deletion flow, before the button.

What it contains, and why it is not the existing export:

- Every page the requester may read, as Markdown, as before.
- **Every attachment of the pond**, not only the embedded ones. An
  attachment nobody put on a page would otherwise vanish unnoticed — which
  is the whole reason this issue exists.
- `manifest.json`: pond settings (EFFECTIVE, defaults filled in — a
  preservation format must not require its reader to know Dorfteich's
  defaults), labels, the page hierarchy and sort keys, comments, and
  attachment metadata including the #199 hash so a reader can verify bytes.
  It extends the #210 manifest rather than adding a second descriptor, and
  carries an explicit `formatVersion`.
- `README.txt`, because the manifest is for machines: whoever unpacks a
  folder of Markdown a year from now must not believe they hold a one-click
  restore.

Decisions worth naming:

- **"Complete" describes the RESULT, not the route.** A pond admin who may
  read every page gets `complete: true`; only an archive that actually
  leaves pages out is incomplete. The Site-Admin route skips the read filter
  (an archive taken before an irreversible purge must not depend on which
  ponds the operator happens to be a member of) — those are two different
  questions and the first version of this conflated them.
- **The omission is named before the download**, with its number, in the UI
  and in the manifest. An archive silently missing content is worse than no
  archive, because it ends the search.
- **Not downloading stays allowed.** A pond of test pages should not require
  one, and the server cannot tell whether a file arrived anyway — so the
  finality is stated in text instead of enforced.
- **A plain link, not fetch-into-a-blob.** The api streams the ZIP; buffering
  a whole pond in the tab to draw a progress bar would trade memory for
  cosmetics. The browser reports progress and completion; what it cannot say
  — that the archive is being BUILT — is announced in a live region.
- Read trail unchanged in kind (ADR 0023): one `export` event per classified
  page before any classified byte enters the stream. Attachments never travel
  without their page, so the same events cover them.
- New audit action `pond.archived` (catalogue v1.7) with page and attachment
  counts, omitted pages, and completeness.

Format documented in `docs/architecture/pond-archive-format.md`, including
what is deliberately NOT in it (history, permissions, trash).

Verified by hand, not only asserted: a real pond's archive downloaded and
unpacked — README, manifest, three page files, the media file; the manifest's
effective settings, per-page classification, the VS-NfD frontmatter and
marking preserved in the classified page's Markdown, and the attachment's
sha256 present. Plus six api tests (including that an unembedded attachment
travels and that a Site Admin gets a complete archive without membership) and
the a11y pack 11/11 in both schemes, which now also scans the pond settings
screen.

Not done, because there is nothing to attach it to: the Site Admin's purge
dialog (#193) exists only as an api endpoint — there is no pond-trash UI in
the web app. The api half is here and tested, so it becomes a link when that
screen is built.
2026-08-01 20:24:35 +02:00
8752cf0c5a #179: negotiate the SPA shell's lang attribute in nginx
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 7m13s
CI / Build container images (pull_request) Successful in 1m21s
CI / Auth e2e pack (pull_request) Successful in 9m25s
CI / Import/export fidelity gate (pull_request) Successful in 54s
CD / Build and push images (push) Successful in 21s
CD / Deploy to Test (push) Successful in 15s
CD / Smoke tests against Test (push) Successful in 1m40s
CD / Promote to Int (push) Successful in 15s
CI / Lint, typecheck, test (push) Successful in 7m9s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m45s
CI / Import/export fidelity gate (push) Successful in 55s
`apps/web/index.html` carries a hard `lang="en"`. The app corrects it at
runtime (#163), but nginx answers every SPA route with that same file, so a
crawler or a no-JS visit — `/public/...` on prod is exactly that — saw `en`
for German content, permanently. WCAG 3.1.1 is about the delivered document,
not the one JavaScript later fixes.

nginx-only, no backend involved: a `map` on `Accept-Language` and a
`sub_filter` in the index.html path. Only the FIRST tag decides, which is
what "the browser's preferred language" means and mirrors #163 — `de-CH`
counts as German, `en-US,de` does not.

`Vary: Accept-Language` is new. The response now genuinely depends on a
request header, and without it a shared cache could hand one language's copy
to the other. Everything else in the location is untouched: same CSP, same
`Cache-Control: no-cache`, same `nosniff`.

The known limit is documented in the config rather than worked around: nginx
cannot read `instance.defaultLocale`, so an unlisted or absent
Accept-Language yields `en` even on a German instance. For public content
that is not the authoritative rendering anyway — the api's server shell
(`/api/v1/public/...`) already renders those with the instance locale.

Verified against a real nginx 1.27 (the image the stage runs) with the
config mounted as-is: `de-DE,de;q=0.9,en;q=0.8` and `de` yield `lang="de"`;
`en-US,en;q=0.9`, `en-US,en;q=0.9,de;q=0.8`, `fr-FR,fr` and a request with no
header yield `lang="en"`; an SPA route (`/public/teich/seite`) negotiates the
same way; the gzipped response is rewritten too, and a JavaScript asset comes
through byte-identical.
2026-08-01 20:01:53 +02:00
6377faf332 #306: instance branding — logo and favicon, cropped in the browser
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 7m28s
CI / Build container images (pull_request) Successful in 2m7s
CI / Auth e2e pack (pull_request) Successful in 9m37s
CI / Import/export fidelity gate (pull_request) Successful in 1m7s
CD / Build and push images (push) Successful in 23s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m47s
CD / Promote to Int (push) Successful in 16s
CI / Lint, typecheck, test (push) Successful in 7m25s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 9m41s
CI / Import/export fidelity gate (push) Successful in 1m12s
An instance had no way to look like itself: the top bar said "Dorfteich"
whatever the operator called their instance, `instance.name` was never
rendered in the running app at all, and there was no favicon anywhere —
`index.html` had no `<link rel="icon">` and `public/` held only fonts and
theme-init.js.

Where the line is drawn, and why:

- **The api never decodes an image.** Cropping, scaling and the conversion
  to PNG happen on a canvas in the browser; the api checks the PNG
  signature, reads the IHDR dimensions at their fixed offsets and enforces
  the caps. An image library would put a decoder in front of
  attacker-supplied bytes AND would have to be carried through the
  `--network none` offline build. Reading two big-endian integers is not
  decoding.
- **SVG is refused**, with its own error message rather than a generic
  "not a PNG": it can carry script, and serving it from our own origin
  would be a cross-site-scripting vector. An operator who tried one should
  learn that it is deliberate.
- **The crop is driven by number inputs, not by dragging.** A drag-only
  cropper excludes keyboard and switch users outright; a number input is
  arrow-key operable and screen-reader readable without any custom aria.
  The resulting pixel size is stated in text, not only drawn as a frame.
- **The variant is chosen by CSS, not JavaScript.** `theme-init.js` has
  already resolved `data-theme` before first paint, so the correct logo is
  the one painted rather than the one that appears after a flash. Without a
  dark variant the LIGHT logo carries both themes — the operator's own
  asset shown unchanged beats one they did not choose (the rule #307
  extends to ponds). The settings screen warns; it never blocks.
- **The favicon link is static, its resource dynamic.** index.html stays a
  static file and the api answers with the uploaded icon or a shipped
  default — that route must never 404, or the browser keeps its generic
  icon for good. The default is generated by a script from Node's own zlib
  (`gen-default-favicon.mjs`), for the same offline-build reason.
- Both favicon sizes are uploaded together: one source, one crop, so the
  tab icon and the home-screen icon can never disagree.
- Branding is served WITHOUT a session, because the login screen carries it
  and the browser fetches the favicon before anyone signs in. The admin
  screen says so — an operator may not expect their logo to be public.
- The metadata is not writable through the settings endpoint: it describes
  bytes on disk, and hand-writing it would claim an asset that is not
  there.

`./data/branding` follows the three-step rule #303 paid for: env default +
`data-dirs.ts` entry, compose volume (repo AND the stages on ONE), and the
`mkdir`/`chown` line in the api Dockerfile. `data-dirs.test.ts` is new and
closes the hole that made #303's variant invisible: the nightly archive
skips a missing directory WORDLESSLY, so the fence now demands that every
`*_DIR` the backup env declares actually travels in the archive. Verified
against the real defect — removing the line fails it by name.

Audit catalogue v1.7 (`branding.changed`), carrying `scope` from the start
so #307 is the same event with a different scope, not a second id.

Verified: api suite 103 files green (a lone `public-api` ECONNRESET under
local parallel load, green in isolation — the documented local flake);
branding suite 12 tests against a real directory; crop arithmetic unit
tests; a11y pack 11/11 in both schemes; /admin measured at 320px with the
new section (overflow 0); and the whole flow walked in the browser: upload
→ crop 780×180 → stored as 512×118 → logo in the sidebar linking home with
the instance name as its accessible name → topbar wordmark following
`instance.name` → light logo still shown under `data-theme="dark"`.
2026-08-01 19:30:52 +02:00
942f7b13d3 #304: scope the legal spec's status locator to its own form
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m31s
CI / Build container images (pull_request) Successful in 1m15s
CI / Auth e2e pack (pull_request) Successful in 8m57s
CI / Import/export fidelity gate (pull_request) Successful in 53s
CD / Build and push images (push) Successful in 36s
CD / Deploy to Test (push) Successful in 11s
CD / Smoke tests against Test (push) Successful in 1m24s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Failing after 7m12s
CI / Auth e2e pack (push) Has been skipped
CI / Import/export fidelity gate (push) Has been skipped
CI / Build container images (push) Has been skipped
The font manager's upload live regions made `getByRole('status')` ambiguous
on /admin, and legal.spec.ts — which asserts the legal form's success message
— started failing in the e2e pack. That is the documented trap in CLAUDE.md:
a new label or region makes an existing page-wide locator ambiguous, and the
fix is to scope the SPEC, not to drop the region a screen reader needs.

The section gets a named class for exactly that purpose.

Verified locally against the running stack: legal, fonts, admin-users,
admin-quotas and the a11y pack all pass.
2026-08-01 19:05:18 +02:00
ee6a11f9b0 #304: declare the font-list route's access rule explicitly
Some checks failed
CI / Build container images (pull_request) Successful in 3m51s
CI / Lint, typecheck, test (pull_request) Successful in 6m35s
CI / Auth e2e pack (pull_request) Failing after 3m6s
CI / Import/export fidelity gate (pull_request) Has been skipped
The route-permission fence (#52) failed in CI, not locally: I had run the
fonts and import-export suites, not the full api suite, and that fence needs
a database. `@AuthenticatedOnly()` is the rule the route always meant — a
session, no further permission.

Re-verified with the FULL api suite against a fresh database: 103 files /
575 tests passed.
2026-08-01 18:46:47 +02:00
f8c241b11a #304: custom fonts in the pickers, an admin screen, and the licence page
Some checks failed
CI / Lint, typecheck, test (pull_request) Failing after 6m26s
CI / Auth e2e pack (pull_request) Has been skipped
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Build container images (pull_request) Has been skipped
The backend from #303 could store an operator's font but nothing could
choose one: no list endpoint outside the Site-Admin routes, no @font-face
rules for a family that only exists at runtime, and no management UI.

Found while wiring it up — a real defect in #303, invisible to its tests:
`fontStack` cannot tell an uploaded family from a deleted one, so the PDF
exporter embedded the face and then never named it. Every export of a pond
using an operator font rendered in the system font while the job reported
success. Both `fontStack` call sites now take the uploaded families
(`buildPdfHtml`, `pondFontVariables`); `pdf-html.test.ts` pins the
regression from both sides. Verified against a real Gotenberg: with the
families the PDF embeds PlayfairDisplay-Bold, without them NotoSans-Bold —
that was the whole bug, in one diff of two PDFs.

- `GET /fonts/custom` is readable by any signed-in user, not Site Admins
  only: the pickers, the licence page and the injected `@font-face` rules
  all need it, and gating it would have forced a second, admin-only UI.
- Bundled and uploaded families are told apart by their `<optgroup>`, not
  by a badge — the grouping is then part of the control's semantics, so a
  screen reader announces it and the native mobile select keeps it. Within
  each source the catalog's category grouping is preserved.
- The delete confirmation names how many ponds use the family and what
  happens to them; focus moves to it and back on cancel. Deletion stays
  unblocked (the api's decision, #303) — the ponds degrade, they do not
  break.
- The licence page grew a second table. That is what makes an attribution
  obligation satisfiable: a commercial licence that requires naming the
  foundry needs a page to name it on.

Verified in the browser end to end (upload two weights → listed and
rendered in its own font → chosen in a pond → page renders in it → deleted
→ pond falls back): api suite for fonts/export 77 passed, a11y pack 11/11
locally in both schemes, lint/typecheck/i18n:check green.
2026-08-01 18:32:46 +02:00
485c8fa538 #303 follow-up: the fonts volume must mount node-owned
All checks were successful
CI / Auth e2e pack (pull_request) Successful in 8m49s
CD / Build and push images (push) Successful in 14s
CD / Deploy to Test (push) Successful in 17s
CD / Smoke tests against Test (push) Successful in 1m21s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m35s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m30s
CI / Import/export fidelity gate (push) Successful in 57s
CI / Build container images (pull_request) Successful in 2m52s
CI / Lint, typecheck, test (pull_request) Successful in 6m28s
CI / Import/export fidelity gate (pull_request) Successful in 57s
Found on the real deploy, not in any test: `/data/fonts` in the running
api container was `root:root` and the non-root `node` user could not
write to it. Every upload would have failed with EACCES at runtime while
the api reported ready.

The api Dockerfile already explains the mechanism for uploads and
plugins — Docker copies an image directory's ownership into a fresh named
volume on first mount — and pre-creates them chowned. #303 added
`CUSTOM_FONTS_DIR` to the ENV but not to that mkdir/chown line.

Adds a CI fence so it cannot recur: every `/data/…` path the api image
defaults to must also appear in the mkdir AND the chown. Verified against
the actual defect — removing `/data/fonts` from the chown makes it fail.
2026-08-01 15:12:06 +02:00
b96997501a #303: operator-uploaded fonts — storage, API, PDF embedding, backup
All checks were successful
CI / Build container images (pull_request) Successful in 3m53s
CI / Auth e2e pack (pull_request) Successful in 8m42s
CI / Auth e2e pack (push) Successful in 8m41s
CI / Lint, typecheck, test (pull_request) Successful in 6m30s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Build and push images (push) Successful in 18s
CD / Smoke tests against Test (push) Successful in 1m19s
CD / Deploy to Test (push) Successful in 16s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m41s
CI / Build container images (push) Has been skipped
CI / Import/export fidelity gate (push) Successful in 52s
An operator holding a font licence could only use it by baking the file
into a custom image, which tied every change to a rebuild and left the
file out of the backup.

ADR 0016 said there is no runtime font management. It also listed this
exact case under "Alternatives considered" — *may become a Site-Admin-
level feature later*. The amendment takes that option and answers the two
objections it raised: licensing risk (Site Admins only, licence recorded
with the family) and file-format attack surface (magic-byte check and a
size cap, never a parse).

- `CUSTOM_FONTS_DIR` (default `./data/fonts`) — a sibling of uploads and
  plugins, NOT inside the image-baked `FONTS_DIR`, where a deploy would
  overwrite it and no backup would ever see it.
- One list of data directories (`apps/backup/src/data-dirs.ts`) now feeds
  both the nightly archive and the restore, so they cannot drift. #306 and
  #307 add one line each instead of a second mechanism.
- Both Dockerfiles bake the path. The backup image sets its volume paths
  itself ("self-sufficient without compose env" — #71's lesson) and reads
  no *_DIR from compose; without the ENV entry the archive would have
  skipped the directory silently.
- The PDF path already read WOFF2 from disk at request time, so it only
  had to pick the other base directory for a custom family.
- `fontStack`/`fontEntry` take the instance's uploaded families as an
  argument — they are runtime data. The catalog is searched first, and a
  colliding family name is rejected at upload, so a custom font can never
  shadow a catalog one.
- Deletion is never blocked by usage: an unknown family already falls back
  to the system stack, so affected ponds degrade instead of breaking. The
  count of affected ponds travels into the audit entry.
- Audit catalogue v1.6 (`font.uploaded`, `font.deleted`).

Verified: api full suite against a fresh database, 102 files / 571 tests.
The upload suite writes into a real temp directory and reads the bytes
back off disk, so the storage layer is exercised rather than mocked.
2026-08-01 14:49:13 +02:00
5164801676 #301: reset the login rate limit before the VS-NfD packs
All checks were successful
CD / Promote to Int (push) Successful in 12s
CI / Build container images (push) Has been skipped
CI / Import/export fidelity gate (push) Successful in 58s
CI / Lint, typecheck, test (push) Successful in 6m32s
CI / Auth e2e pack (push) Successful in 8m30s
CI / Build container images (pull_request) Successful in 1m13s
CI / Auth e2e pack (pull_request) Successful in 8m42s
CI / Import/export fidelity gate (pull_request) Successful in 1m6s
CI / Lint, typecheck, test (pull_request) Successful in 6m24s
CD / Build and push images (push) Successful in 18s
CD / Smoke tests against Test (push) Successful in 1m15s
CD / Deploy to Test (push) Successful in 14s
CI 665: the reflow guard itself passed; the run died two packs later on
`fixture login for fixture-admin failed: 429`.

The a11y pack costs one more login since this branch added the reflow
test, and that was enough to exhaust the budget before the VS-NfD packs.
Same trap the workflow already documents for the content and collab
packs — it just needed one more reset, in the place the extra login
pushed it over.
2026-08-01 12:31:30 +02:00
69882ecbea #301: the token tables need the same scroll wrapper
The sorted report finally named it: `table.api-tokens__table` at 833px
wide, with its `.visually-hidden` heading reaching right=737 — exactly
the document's scrollWidth. Same mechanism as the sessions table, a
second table I had not wrapped.

Locally the API-tokens table was empty and therefore narrow, which is why
this only ever appeared in CI. With a token present it reproduces:
without the wrapper 345px of page overflow, with it none.

The feed-token table gets the same treatment — it is built the same way
and would fail as soon as someone holds a feed token with a long name.

The "[in fitting scroller]" marker in the report is misleading for these:
`main.main` is a scroller, but it is `position: static`, so it never
clipped the absolutely positioned heading. Only a positioned ancestor
does — which is what `.table-scroll` now is.

Verified locally against a real stack, with a wide token table present:
reflow guard green, whole a11y pack green in both colour schemes.
2026-08-01 12:31:30 +02:00
2422f3a28f #301: sort the reflow report so the culprit cannot be buried
CI still reports 737 while the local stack is now clean, and the box list
was capped at 15 entries — all of them nav links clipped by their own
scroller. Whatever pushes the page in CI sits past that cap.

The list is now sorted by reach, marks each entry as either clipped by a
fitting scroller or actually pushing the page, and shows 40.
2026-08-01 12:31:30 +02:00
b65339ae13 #301: the overflow was an escaping visually-hidden heading
Found by standing up the local stack instead of guessing through CI.
The DOM tree under `.app-body` shows it in one line:

  span.visually-hidden rect=[342,343] pos=absolute

Its right edge is 343, and `.app-body` reports scrollWidth 343 against a
320 client. The table's actions column carries a `.visually-hidden`
heading, which is `position: absolute`. `.table-scroll` was `position:
static`, so it was NOT that span's containing block — the span escaped
the scroller's clipping, kept its static position out at the table's
right edge, and pushed the page.

`position: relative` on the wrapper makes it the containing block, and
the span is clipped like the rest of the table.

This is one cause behind both numbers: 23px locally, matching the
original report, and 417px in CI, where different font metrics make the
table wider and carry the span further out. Chasing them as separate
problems is what cost three CI rounds.

Verified locally against a real stack: the reflow guard passes and the
whole a11y pack is green, 11 tests in both colour schemes.
2026-08-01 12:31:30 +02:00
194f144797 #301: dump raw box metrics from the reflow guard
Two rounds now reported no element past the viewport edge while the
document still claimed 417px of overflow — a combination that rules out
every hypothesis I had, including my own filter.

So stop inferring. The guard now prints the html/body metrics, every
element whose own content is wider than its box (with its overflow-x, so
the intentional scrollers are distinguishable), and every box reaching
past the edge with no filtering at all. Diagnostics ride in the assertion
message, not the compared value, so they show up even when they match.
2026-08-01 12:31:30 +02:00
9fce824a8e #301: make the reflow guard report the ancestor chain
The previous run came back with an empty offender list and an unchanged
417px overflow: the filter treated everything under a scroll container as
innocent, including the container that was itself too wide. A scroller
only absolves its children when the scroller fits.

It now reports the chain from body down to the widest offender with each
box's width, so the first element wider than the viewport is visible
instead of inferred.
2026-08-01 12:31:30 +02:00
18c2ed0bfe #301: the real culprit was the jump nav, not the wide content
The first attempt fixed plausible suspects. CI measured the actual page
and named something else: six `.settings-nav__link` buttons, 417px of
page-level overflow at 320px.

`.settings-nav` already had `overflow-x: auto`, but as a flex child it
also had the default `min-width: auto` — the min-content width of the
whole jump strip. That forced the column wider than the viewport, so its
own overflow rule never had anything to scroll. `min-width: 0` is exactly
the case CLAUDE.md warns about under Reflow.

The guard now ignores elements that sit inside a scroll container. Such
content is *meant* to be wider than the viewport — reporting it buried
the one finding that mattered under twelve lines of noise, and the cap
truncated the list before it could show anything else.

The table wrapper and the wrapping settings rows from the first commit
stay. Neither was the cause here, but a table cannot shrink below its
min-content width and those rows cannot wrap on their own, so both are
hardening that holds regardless of content.
2026-08-01 12:31:30 +02:00
f938ee9880 #301: stop /settings scrolling horizontally at 320px
WCAG 2.1 SC 1.4.10 asks for no two-dimensional scrolling down to 320px,
which is also what 400% zoom on a 1280px screen produces. The layout
skeleton was already hardened for this in #165; the overflow came from
content inside the sections.

- The sessions table cannot shrink below its min-content width — four
  columns, one of them the full user-agent string. It now scrolls inside
  its own container rather than pushing the page. The container is
  focusable with a role and a name, because a scroll area that only a
  mouse can reach trades one barrier for another.
- `.settings-checkbox` rows may wrap. The accent swatches have a fixed
  size and cannot shrink, so an unwrappable row set a floor for the whole
  page width.

Adds a reflow guard to the a11y pack. axe does not cover 1.4.10 — the
criterion is not derivable from the DOM — so this is a separate check,
and it names the overflowing elements when it trips instead of only
reporting that something overflows.
2026-08-01 12:31:30 +02:00
f9149eba13 #302: the vault import test reaches its page through the sidebar
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m30s
CI / Build container images (pull_request) Successful in 1m21s
CI / Auth e2e pack (pull_request) Successful in 8m49s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 35s
CD / Smoke tests against Test (push) Successful in 1m25s
CD / Deploy to Test (push) Successful in 14s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m49s
CI / Import/export fidelity gate (push) Successful in 1m0s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m30s
The Obsidian fixture vault contains a note called "Startseite", and the
pond now creates one too — the seeded fixtures use locale `de`. Two
consequences, and the second is the one that mattered:

- the unscoped title locator matched two sidebar entries;
- `/p/<pond>/startseite` no longer belongs to the imported note. The
  pond's own start page took that slug, so the import landed on a
  suffixed one and the test was about to assert against the wrong page.

Both are fixed by scoping to the mount page and navigating through the
sidebar instead of guessing a slug. The test stays meaningful: it then
clicks a wikilink inside the page content, which the empty auto-created
start page would not have.

CI caught this; the local run passed it. Worth remembering that a
title-based locator can go green by luck.
2026-08-01 11:29:54 +02:00
30fd1ff53b #302: the permission matrix counts the start page
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m26s
CI / Auth e2e pack (pull_request) Failing after 6m12s
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Build container images (pull_request) Successful in 1m20s
Every pond created through the api now carries one, and the matrix pond
is created that way. The start page is an ordinary page with no grant of
its own, so it follows the pond-wide permissions: the three member
subjects each see one more, the label-restricted editor too, and the
outsider — who reaches only the explicitly public page — still sees one.

The 429 in the same run was the login rate limit, reached through the
retries of this failure rather than on its own.
2026-08-01 08:25:51 +02:00
45f1925917 #302: configurable pond start page, created with every new pond
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m28s
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Auth e2e pack (pull_request) Failing after 4m1s
CI / Build container images (pull_request) Successful in 4m3s
Opening a pond landed on whatever sorted first in the sidebar — stable,
but a rule nobody could see, and one whose target moved as soon as
someone added a page ahead of it. New ponds landed on the empty-pond hint
instead of anything useful.

- `startPageId` joins the pond settings. No migration: `Pond.settings` is
  already jsonb. It stores an id, not a slug, so renaming or moving the
  page keeps it working.
- `PondHomePage` prefers it, but only when the page is in this user's
  page list. That list already holds just what they may see, so a start
  page hidden by a page-scoped grant — or trashed — falls back silently
  instead of landing them on a 404, and it costs no extra request.
- Both creation paths give the pond a start page, titled from the
  creator's stored locale. It happens after the creating transaction
  commits: the owner's grant is written inside it and permissions cache
  per pond, so creating the page any earlier would ask about rights the
  grant has not published yet. A failure is logged, not fatal — a pond
  without a start page still works.

`PagesModule` imported `PondsModule` without using it. Removing that
vestigial edge let PondsModule depend on PagesModule in the honest
direction instead of tying the two together with forwardRef.

Every pond created through the api now owns a page, which broke eight
suites whose teardown deleted ponds directly — `Page.pond` deliberately
has no cascade, because a real purge removes contents explicitly and
audits it. A shared `deletePondsWhere` helper deletes pages first. Two
tests that counted pages now account for the start page rather than
pretending the pond began empty.
2026-08-01 08:06:35 +02:00
5a4a99196e #300: route icon-only controls through IconButton/IconLink
All checks were successful
CI / Auth e2e pack (pull_request) Successful in 8m36s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CI / Lint, typecheck, test (pull_request) Successful in 6m22s
CI / Build container images (pull_request) Successful in 3m51s
CD / Build and push images (push) Successful in 15s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m16s
CD / Promote to Int (push) Successful in 13s
CI / Lint, typecheck, test (push) Successful in 6m32s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m25s
CI / Import/export fidelity gate (push) Successful in 58s
The notification bell sat higher and larger than search and the theme
toggle next to it. The cause was not the glyph: `.notifications-bell__button`
carried its own rules with neither flex centring nor an icon size, so the
svg was laid out inline on the text baseline and rendered at lucide's
24px default instead of the 1.15rem the shared `.icon-button` enforces.

Route every icon-only control through the shared components instead:

- `IconLink` joins `IconButton`, sharing one class helper. Three controls
  navigate (pond settings, graph, trash) and are links, not buttons —
  without a link twin they would have stayed the one group gluing the
  class on by hand.
- 17 hand-applied `className="icon-button …"` usages across nine files
  now go through the components, which is what enforces the accessible
  name on a control that shows only an icon.
- The bell's unread count reaches assistive technology. The badge sits
  inside the control, so `aria-label` hid it and a screen reader
  announced "Notifications" without ever saying how many.

An ESLint rule keeps it that way: `icon-button` on a raw button, anchor
or Link is now an error, in both string and template-literal form.

The plugin uninstall button keeps a title that differs from its name (it
explains why a required plugin is locked); IconButton spreads rest last,
so the explicit title still wins.

Also drops the graphify block from CLAUDE.md — it duplicates the
workspace-level instructions.
2026-08-01 06:56:13 +02:00
1f56f34113 #296: remove the unsubscribe-token dual-verify window early
All checks were successful
CD / Smoke tests against Test (push) Successful in 1m25s
CD / Promote to Int (push) Successful in 12s
Release / Build release images and notes (push) Successful in 3m31s
Release / Release-candidate operations QA (push) Successful in 46s
CI / Build container images (push) Has been skipped
Prod deploy / Deploy the released images to Prod (push) Successful in 58s
CI / Import/export fidelity gate (push) Successful in 59s
CI / Lint, typecheck, test (push) Successful in 6m40s
CI / Auth e2e pack (push) Successful in 8m21s
Restore drill / Restore the latest backup into a scratch stack (push) Successful in 1m18s
CI / Build container images (pull_request) Successful in 2m53s
CI / Auth e2e pack (pull_request) Successful in 8m34s
CI / Lint, typecheck, test (pull_request) Successful in 6m22s
CI / Import/export fidelity gate (pull_request) Successful in 59s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 14s
Operator decision at the ADR 0020 acceptance: verification is
subkey-only now instead of waiting for the stated 2026-11-01 expiry.
Links in digest mails sent before the #188 key separation stop working;
recipients use the in-app notification settings. A regression test pins
that the legacy derivation (root key + purpose prefix) can never verify
again; security.md records the removal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 23:03:03 +02:00
9b7acab294 #232: plugin allowlist with SHA-256 hash pinning
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m21s
CI / Build container images (pull_request) Successful in 3m59s
CI / Auth e2e pack (pull_request) Successful in 8m35s
CI / Import/export fidelity gate (pull_request) Successful in 1m1s
CD / Build and push images (push) Successful in 17s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 12s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m27s
CI / Import/export fidelity gate (push) Successful in 58s
CI / Lint, typecheck, test (push) Successful in 6m30s
The install path records the SHA-256 of the delivered bundle ZIP
(plugins.bundle_hash; pre-#232 installs show it as unknown until
reinstalled). plugins.allowlist in instance_settings names permitted
ids with their pinned hashes: empty (default) = not enforced, existing
instances unchanged; non-empty = installs of unlisted or deviating
bundles are rejected (plugin_not_pinned / plugin_hash_mismatch, 403),
and an installed plugin outside the list or with a deviating hash does
not load — absent from pond mount lists, frame/assets 404. Every
rejection is audited (plugin.rejected, catalogue v1.5). A version bump
changes the hash and therefore requires an explicit re-pin — the
intended friction (ADR 0025). Admin UI shows observed vs pinned hash
per plugin with pin/re-pin/unpin. Scope stated honestly in
plugin-architecture.md: the pin answers "is this the reviewed bundle";
post-install disk tampering is platform integrity (ADR 0019), sandbox
containment stays the sandbox's job. Hardening guide row + catalog
advisory triage; residual risk R-03 resolved. e2e: empty-allowlist
compatibility, pinned load, unpinned and tampered installs rejected and
audited, pin drift blocks loading while the admin still sees the
mismatch, version bump needs re-pin. Full api suite 101 files / 561
green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 21:26:36 +02:00
404a3741c8 ADRs 0019-0027: accepted after explicit operator review (2026-07-31)
All checks were successful
CI / Auth e2e pack (pull_request) Successful in 8m34s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CI / Lint, typecheck, test (pull_request) Successful in 6m19s
CI / Build container images (pull_request) Successful in 1m14s
CD / Build and push images (push) Successful in 17s
CD / Deploy to Test (push) Successful in 15s
CD / Smoke tests against Test (push) Successful in 1m16s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m25s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m24s
CI / Import/export fidelity gate (push) Successful in 59s
Stefan reviewed and accepted all nine VS-NfD ADRs one by one. Two
adjustments from the review: ADR 0021 decision 3 now states the #216
refinement in the decision itself (PAT/feed-token issuance stays
available to IdP-authenticated sessions — API authorization under its
own switches, not interactive sign-in) instead of contradicting the
later Decisions section; and the ADR 0020 dual-verify window will be
removed early (issue #296) rather than waiting for its stated expiry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 20:47:27 +02:00
4d9f913845 #246: mode enforced — reject profile-violating configuration writes
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m18s
CI / Build container images (pull_request) Successful in 4m3s
CI / Auth e2e pack (pull_request) Successful in 8m43s
CI / Import/export fidelity gate (pull_request) Successful in 1m0s
CD / Build and push images (push) Successful in 23s
CD / Smoke tests against Test (push) Successful in 1m22s
CD / Deploy to Test (push) Successful in 12s
CD / Promote to Int (push) Successful in 13s
CI / Lint, typecheck, test (push) Successful in 6m27s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m24s
CI / Import/export fidelity gate (push) Successful in 58s
In enforced mode the ONE settings write path every caller uses rejects
catalog-violating values with the stable code vs_nfd_profile_violation
(403 — the request is well-formed, the policy says no). Existing
violating values are reported at startup (log line, database-less boots
must not fail) and on the admin card, never auto-changed. The UI
renders as in hidden (#245 already keys on hidden|enforced). The
hardening guide now names enforced as the recommended mode for VS-NfD
reference operation. Tests: violating write rejected with the stable
code and nothing stored; compliant writes pass; the same violating
write passes in marked and hidden (own app boots); pre-existing
violation reported and untouched. Full api suite 100 files / 555 green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 19:48:37 +02:00
0d95e1304e #245: mode hidden — hide profile-violating options, mark the hiding
All checks were successful
CI / Lint, typecheck, test (push) Successful in 6m24s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m34s
CI / Import/export fidelity gate (push) Successful in 1m1s
CI / Build container images (pull_request) Successful in 1m13s
CI / Lint, typecheck, test (pull_request) Successful in 6m14s
CI / Auth e2e pack (pull_request) Successful in 8m31s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
In hidden (and later enforced) mode, catalog-listed controls whose only
purpose is enabling a violation are not rendered while their saved value
is compliant (the four master switches, the Nextcloud backup block);
value-listed selects keep only their compliant choices (registration
mode, new-page classification, upload policy, SVG policy). Every
affected section shows one accessible policy note (i18n de+en) so
policy is distinguishable from missing features. A value that was
already violating is surfaced exactly like in marked — never silently
hidden. The API stays unchanged; enforcement is #246. e2e: hidden half
of the marking pack (rows disappear, note visible, already-violating
row stays marked, axe WCAG A/AA clean) — verified live locally; CI runs
it against a second api (VS_NFD_MODE=hidden, same database) behind its
own static server.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 19:27:17 +02:00
5fdef95f67 #244: mode marked — flag profile-violating configuration in the UI
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m34s
CI / Build container images (pull_request) Successful in 1m19s
CI / Auth e2e pack (pull_request) Successful in 8m23s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 23s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m33s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m49s
CI / Import/export fidelity gate (push) Successful in 58s
Every catalog-listed control on the admin surfaces carries an accessible
deviation marking in mode marked: text + icon under the control (never
colour alone), part of the control's accessible description
(aria-describedby), i18n de+en. The check runs against the CURRENT
control value, so a violating choice is marked before saving. Covered
controls: registration mode, new-page classification, upload policy,
SVG policy, the four master switches (api/mcp/feeds/plugins), the legal
texts (violating while empty), and the Nextcloud backup toggle on the
system panel. The profile card (#243) gains the warning summary and the
hardening-guide reference. e2e: new vs-nfd-marking pack (marked half in
CI — the e2e api now runs VS_NFD_MODE=marked, which also puts the
marked state into the a11y admin scan; off half in local default runs;
both halves verified live). hidden/enforced follow in #245/#246.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 18:49:26 +02:00
da5fd7c770 #243: VS_NFD_MODE and the machine-readable hardening-profile catalog
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m12s
CI / Build container images (pull_request) Successful in 4m2s
CI / Auth e2e pack (pull_request) Successful in 8m29s
CI / Import/export fidelity gate (pull_request) Successful in 54s
CD / Build and push images (push) Successful in 31s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m29s
CD / Promote to Int (push) Successful in 14s
CI / Build container images (push) Has been skipped
CI / Lint, typecheck, test (push) Successful in 6m30s
CI / Auth e2e pack (push) Successful in 8m6s
CI / Import/export fidelity gate (push) Successful in 57s
The deployment declares through VS_NFD_MODE (off | marked | hidden |
enforced, default off) how the application treats configuration that
violates the VS-NfD reference profile — deploy-level like
BACKUP_ALLOWED_TARGETS, so a compromised Site Admin cannot widen it.
The catalog in shared (vs-nfd-profile.ts) is the single source of
truth: every profile-relevant setting with a decidable compliant value,
judgement calls in an explicit advisory list, and a fence test parsing
the hardening guide's reference tables so neither can drift (pattern
#201). The api evaluates the catalog against the typed settings
registry and validated env and exposes mode + verdict on
GET /admin/system/vs-nfd-profile; the admin settings view shows the
card whenever the mode is not off. Display only — the treatments land
with #244–#246 (ADR 0027, proposed).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 18:24:04 +02:00
18239e2fa9 #221: offline update path incl. migrations, rehearsed with rollback
All checks were successful
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m15s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m19s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m13s
CI / Import/export fidelity gate (push) Successful in 54s
CI / Lint, typecheck, test (pull_request) Successful in 6m20s
CI / Build container images (pull_request) Successful in 1m12s
CI / Auth e2e pack (pull_request) Successful in 8m24s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Build and push images (push) Successful in 22s
Adds docs/operations/update-runbook.md (obtain, verify by digest, back
up, apply, verify, roll back) with the migration behaviour stated
explicitly: a failed migration rolls back its own transaction but is
recorded in _prisma_migrations and blocks every further migrate deploy
(P3009) — including a re-deployed old image — until migrate resolve
--rolled-back; semantically irreversible migrations have exactly one way
back, the pre-update backup set. No rolling updates on a compose stage.
Rehearsed in the isolated environment of #220: regular update to a v2
image set, then a deliberate failed-update (P3018 division by zero,
schema change proven rolled back) with image-rollback-alone shown
insufficient and the documented recovery executed. Protocol:
docs/vs-nfd/98-update-rollback-protokoll.md. ADR 0024 decisions 5+6
recorded as executed; operations handbook and restore runbook updated;
plan checkbox P1-3 ticked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 17:26:50 +02:00
ccffcaadd6 #220: protocol of the isolated deployment run
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m16s
CI / Build container images (pull_request) Successful in 1m26s
CI / Auth e2e pack (pull_request) Successful in 8m27s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Build and push images (push) Successful in 21s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m22s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m12s
CI / Import/export fidelity gate (push) Successful in 57s
Full deployment exercised in a compose stack whose networks are all
internal: true — setup, login, live collaboration, search, upload, all
export formats, backup and restore. tcpdump full capture on both
bridges: zero packets leave the isolated subnets; the only outbound
attempt the application makes is SMTP, which fails contained in the
outbox (5 retries, then FAILED) while the instance stays fully
functional. The restore finding became #288, fixed earlier in this
chain and re-verified in the same stack. Plan checkbox P1-3 and the
I-28 open question ticked; operations handbook airgap section updated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 17:05:08 +02:00
4f6596e8a2 #288: reset schema before pg_restore — partitioned tables broke --clean
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m48s
CI / Build container images (pull_request) Successful in 1m46s
CI / Auth e2e pack (pull_request) Successful in 8m22s
CI / Import/export fidelity gate (pull_request) Successful in 1m8s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
Since #224 read_events is partitioned; the dump carries per-partition
primary keys as own entries, and pg_restore --clean emitted DROP
CONSTRAINT against inherited constraints, which PostgreSQL refuses. The
restore then reported FAILED although the content was restored. Dropping
and recreating the public schema first makes every --clean drop a no-op
and the restore faithful: objects created after the backup no longer
survive. Verified in the isolated environment of #220 (set
20260731-132200, exit 0, readyz green).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 17:00:05 +02:00
a758c9d78b #219: verified reproducible build without network access
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m45s
CI / Build container images (pull_request) Successful in 1m14s
CI / Auth e2e pack (pull_request) Successful in 8m37s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Build and push images (push) Successful in 21s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m16s
CD / Promote to Int (push) Successful in 28s
CI / Lint, typecheck, test (push) Successful in 6m19s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m12s
CI / Import/export fidelity gate (push) Successful in 58s
The ADR 0024 §4 decision, taken explicitly and both ways: customers
OPERATE prebuilt digest-pinned images (no customer-side build), and
ADDITIONALLY the workspace build is verified to work with networking
disabled - so site-local patching stays possible without internet.

Evidence (docs/vs-nfd/96-offline-build-protokoll.md): pnpm install
--offline --frozen-lockfile plus pnpm build under docker run
--network none (node:22.15.1-alpine + pnpm 11.9.0, the pinned
toolchain), reproduced twice from clean checkouts with identical
results. The offline kit is the pnpm store (~870 MB) plus the build
user's ~/.cache (~460 MB - the prisma engines live there; without the
cache the prisma postinstall fails offline).

The one network dependency found and bounded: the drawio plugin's
installable ZIP fetches its pinned vendor tarball on first build.
Deploy images contain no plugin ZIPs, so the delivery-relevant build is
fully offline (CI=1 skips the fetch, as in CI); an offline ZIP build
pre-seeds the tarball into packages/plugins/drawio/vendor/.

Also catches up the operations manual's scheduler-job table to 10
(read-trail-maintenance was added in #224 without the row here).

Refs #219.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 14:39:35 +02:00
2f7ba65eef #218: mirror procedure into an internal registry
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m11s
CI / Build container images (pull_request) Successful in 1m24s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Auth e2e pack (pull_request) Successful in 8m49s
CD / Build and push images (push) Successful in 20s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Successful in 1m25s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m24s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Has been cancelled
Airgapped sites pull from their own registry (ADR 0024). The image list
is GENERATED (deploy/scripts/list-images.sh resolves the compose file
incl. the caddy profile) so a mirror can never silently miss a service;
third-party images gain a configurable ${REGISTRY_PREFIX:-} in the
compose file (digest pins unchanged - Docker verifies the same sha256
regardless of which registry serves it), own images keep IMAGE_PREFIX;
no image reference is ever edited per site.

Step-by-step procedure in deploy/stages.md 5b: generate list, copy
digest-preservingly (docker buildx imagetools create; plain
pull/tag/push as the documented fallback - the digest comparison closes
the loop either way), verify the digest in the mirror against the pin,
point the deployment via REGISTRY_PREFIX/IMAGE_PREFIX.

Executed once end-to-end and recorded as assessor-facing evidence
(docs/vs-nfd/95-mirror-protokoll.md): all four third-party images
mirrored digest-identically into a local registry:2, plus
dorfteich-api:v0.12.0 (sha256:576f1646... identical on both sides; the
imagetools stall against the Gitea registry is recorded with its
workaround). Operations manual's airgap section now lists the mirror
part as available.

Refs #218.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 14:33:04 +02:00
6aac785841 #217: map IdP groups and roles onto the permission model
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m55s
CI / Build container images (pull_request) Successful in 3m0s
CI / Auth e2e pack (pull_request) Successful in 8m49s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Build and push images (push) Successful in 21s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m19s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m28s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m15s
CI / Import/export fidelity gate (push) Successful in 59s
Declarative instance setting idpMapping.rules turns ID-token claims into
pond roles and the site-admin flag on every OIDC login — configuration,
not code. Mapped grants travel through the SAME GrantsService path as
manual ones (permission cache invalidated, collab access notify fires so
live sessions revalidate — asserted by test), never raw rows.

Ownership makes precedence explicit: role_grants.origin marks mapped
rows, users.is_site_admin_managed marks a mapping-set admin flag. The
mapping only creates and revokes what it owns — manual wins: hand-made
grants and hand-promoted admins are never revoked by a missing claim (a
manual toggle clears the marker and takes ownership). Removal of a claim
revokes the mapped grant and the managed flag on the next login. Every
mapping-driven change is audited with origin idp_mapping.

Failure containment: unknown pond slugs and the last-Pond-Admin
protection log-and-skip — a mapping problem must never become a login
lockout. Tests drive real OIDC logins against the fake IdP with group
claims: grant + working access, revocation incl. notify, manual-wins,
managed site-admin promote/demote/hands-off.

Documented in permissions.md (own section), ADR 0021, data-model.md and
the hardening guide (care rule: same PR).

Refs #217.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 13:09:11 +02:00
13f0311d8e #216: hard AUTH_LOCAL_ENABLED switch over every local credential flow
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m55s
CI / Build container images (pull_request) Successful in 4m43s
CI / Auth e2e pack (pull_request) Successful in 9m13s
CI / Import/export fidelity gate (pull_request) Successful in 1m4s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CI / Lint, typecheck, test (push) Has been cancelled
CD / Build and push images (push) Has been cancelled
The deploy-level realization of auth.local.enabled (ADR 0021): FALSE
answers 404 on every local credential flow — login, signup, e-mail
verification, resend, password forgot/reset/change — enforced centrally
in the auth guard via the @LocalCredentialFlow() marker before any
session or CSRF logic runs. Deploy-level on purpose: a compromised Site
Admin cannot reopen the local path, so the runtime-flip residual risk
from ADR 0021 does not materialize (R-02 closed in the risk list).

An enumeration fence fails when an auth route is neither marked nor on
the reviewed allowlist, so a new credential flow cannot ship unswitched.
Stated decisions, each tested: sessions/logout keep working for
externally authenticated users; PAT and feed-token issuance stays
available (API authorization under its own switches, not interactive
sign-in). Bootstrap: complete setup (or SETUP_ADMIN_* pre-seed) before
flipping; the api warns at boot when local auth is off with neither OIDC
nor proxy auth configured. GET /auth/methods reports local:false and the
login page hides the local form and credential links.

Hardening guide: the planned auth.local.enabled row moves from 1.3 into
the live deploy table with the bootstrap ordering, and the verification
checklist gains the login-404 probe.

Refs #216.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 12:58:07 +02:00
4c7f001cab #215: trusted reverse-proxy header / mTLS client-certificate path
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m44s
CI / Build container images (pull_request) Successful in 4m42s
CI / Auth e2e pack (pull_request) Successful in 9m15s
CI / Import/export fidelity gate (pull_request) Successful in 59s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
For perimeters that authenticate before the application (ADR 0021 §4).
Off unless BOTH AUTH_PROXY_HEADER and AUTH_PROXY_TRUSTED_PEERS are set —
nothing about the header is guessed. The peer check runs against the TCP
peer address only (a forwarded header is attacker-influenced): a request
carrying the header from any other peer is rejected outright and audited
as auth.proxy_rejected (catalogue v1.4) — that is a spoof attempt, not a
misconfiguration — even when a valid session cookie rides along. From a
trusted peer the header IS the identity; a session cookie never
escalates beyond it; with the feature off the header is inert.

Mapping is explicit (AUTH_PROXY_MAP: username or e-mail); deliberately
no just-in-time creation — the header carries no verified address. The
mTLS variant (AUTH_PROXY_MODE=mtls-dn) maps the configured attribute
(default CN) out of the certificate subject DN the TLS terminator
forwards, under the same peer rules. Session-less proxy requests key the
read trail per user (user:<id>).

The trust boundary is stated in security.md (the section an assessor
reads closest), the VS-NfD security documentation and the hardening
guide's deploy table. Tests cover all four decisions: off = inert,
trusted peer authenticates (username and DN mapping), untrusted peer
rejected + audited, no escalation past a session cookie.

Refs #215.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 12:50:09 +02:00
5796b7a5dd #214: OIDC Authorization Code with PKCE, Keycloak as reference IdP
Some checks failed
CI / Lint, typecheck, test (pull_request) Failing after 14s
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Build container images (pull_request) Has been skipped
CI / Auth e2e pack (pull_request) Has been skipped
External authentication (ADR 0021) built on jose (#188's vetted library)
plus fetch — no new dependency enters the supply chain for a security
base function. Discovery-configured; ID tokens validate against the
IdP's JWKS under an explicit RS256/ES256 allowlist with issuer,
audience, expiry and nonce binding. State, nonce and the PKCE verifier
travel in a signed HttpOnly Lax cookie keyed by a dedicated HKDF
purpose (oidc-state, ADR 0020).

Deploy-level configuration (OIDC_ISSUER/CLIENT_ID/CLIENT_SECRET/SCOPES/
PROVIDER_LABEL): who authenticates users is a platform decision. The
login page discovers the provider via GET /auth/methods and renders the
SSO button (i18n de+en).

Identities use the existing slot (provider oidc:<issuer>, subject from
the token). First login creates the account just-in-time — ACTIVE and
mail-verified only when the IdP asserts a verified address. An existing
local account is NEVER adopted silently by e-mail (account-takeover
path): login refuses with oidc_link_required and the owner links
explicitly via GET /auth/oidc/link (audited auth.identity_linked,
catalogue v1.3). Sessions come from the one existing session service.

Tests run the full flow against a protocol-faithful fake IdP: PKCE
verifier at the token endpoint, JIT creation incl. personal pond,
invalid state/nonce/signature/issuer/audience/expiry each rejected, the
linking refusal and the explicit link flow. Verified end-to-end against
a real Keycloak 26.0 (repeatable procedure documented in security.md
§External authentication).

Refs #214.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 12:44:52 +02:00
4af5e6e81f #225: read-trail master switch and written purpose limitation
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m2s
CI / Build container images (pull_request) Successful in 4m1s
CI / Auth e2e pack (pull_request) Successful in 8m26s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Build and push images (push) Successful in 18s
CD / Deploy to Test (push) Successful in 15s
CD / Smoke tests against Test (push) Successful in 1m23s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m6s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m25s
CI / Import/export fidelity gate (push) Successful in 1m0s
New instance switch readTrail.enabled, default OFF: read logging is
employee monitoring in a works council's eyes — an ordinary instance
must not surveil reads. Off means no event is written ANYWHERE (no row,
no stdout line, verified by test); the api announces the switch position
once per boot, so an eventless trail is never ambiguous — a gap reads
as "was off", never "was lost".

The written purpose limitation ships as section 7 of the VS-NfD
security documentation (#228): what is recorded (no content, no titles,
no IPs, no fingerprinting), why (evidence for reads of marked content
only — variant A is the technical anchor of the promise), who may read
it (Site Admin, API-only), for how long (readTrail.retentionDays,
audited pruning), and what it may NOT be used for (no performance or
behaviour monitoring). The hardening guide's reference configuration
turns the trail on (reference value true) and points to that text; the
existing trail suites now enable the switch explicitly.

Refs #225.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 12:35:18 +02:00
2bdb0ec2cf #224: read-trail storage — partitioning, retention, admin query path
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m30s
CI / Auth e2e pack (pull_request) Failing after 5s
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Build container images (pull_request) Failing after 2s
Convert read_events to monthly RANGE partitions on occurred_at, with a
DEFAULT partition as safety net: a lagging maintenance job must never
turn the trail's hard-failure semantics into an outage for classified
reads. The dedup unique pair (#223) moves to per-partition indexes
(PostgreSQL cannot carry it on the parent); a bucket spanning a month
boundary may record one duplicate — over-recording is acceptable, gaps
are not.

New daily job read-trail-maintenance (job-count fence 9 -> 10) creates
months ahead — each with its dedup index — and applies the trail's own
retention readTrail.retentionDays (default 365, deliberately independent
of audit.retentionDays): whole expired months are DROPped without
scanning, remainders deleted by range, every run audited as
read_trail.pruned (catalogue v1.2; the fence regex now admits an
underscore namespace).

Site-Admin query path GET /admin/system/read-events answers "who read
page X" and "what did user Y read" within a period — API-only by
design, documented. Growth measured and documented in data-model.md:
~1 MB per 1000 events including indexes.

Tests: retention pruning + audited deletion + admin queries on the
shared database; the partitioned shape, per-partition P2002 dedup,
months-ahead creation and DROP-based pruning against a fresh database
built by the real migration chain.

Refs #224.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 12:21:45 +02:00
fd4fd60c99 #223: dedup window for the read trail
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m29s
CI / Auth e2e pack (pull_request) Failing after 5s
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Build container images (pull_request) Successful in 2m55s
One event per (session, page, channel) within an aligned window of
readTrail.dedupWindowMinutes (default 5): buckets are
floor(epoch / windowSeconds), and a unique (dedup_key, window_bucket)
pair collapses concurrent duplicates race-free at insert time — the
first access in a window is always recorded, a later duplicate lands on
the unique violation and is skipped quietly (a skipped duplicate is not
a gap; only real write failures still abort the read). Each row carries
windowSeconds, so the evidence states it represents a window, never a
request count.

Reconnects within a window stay one event; a new session records again
even for the same user; channels never collapse into each other; the
page-less attachment key uses the documented `-` placeholder. Load
evidence: 30 collab-token renewals inside one window produce exactly one
event (test), bounding a live editing session at ~12 events/hour/page.

Window semantics documented in ADR 0023, the VS-NfD security
documentation (#228) and as a hardening-guide line for the new setting
(care rule: same PR).

Refs #223.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 12:13:21 +02:00
05a979bac3 #222: read-access trail for classified pages
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m25s
CI / Build container images (pull_request) Successful in 2m58s
CI / Auth e2e pack (pull_request) Successful in 8m35s
CI / Import/export fidelity gate (pull_request) Successful in 1m7s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CI / Lint, typecheck, test (push) Has been cancelled
CD / Build and push images (push) Has been cancelled
Instrument every full-content read channel for pages with
classification = vs_nfd (ADR 0023, variant A): SPA state fetch and read
rendering, public JSON content, no-JS shell, expanded embeds, public API
GET (incl. the MCP read_page path and write echoes), attachment download
under the #212 effective classification, all export shapes (markdown,
pond ZIP, account data export, queued docx/odt/pdf at enqueue), and
collab-token issuance as the api-side proxy for the WS join.

Events land in the new read_events table (no FKs — evidence survives
page purges and hard user deletions) with actor, session key
(session:/token:/job:/anon), page, pond, channel and the classification
at read time. Recording failures are NOT swallowed: a failed write
aborts the read (hard failure, the deliberate contrast to AuditService —
decision recorded in ADR 0023 and security.md §Logging, together with
the recorded residuals: content fragments and feeds).

One e2e test per channel proves both the event and its absence for
unclassified pages, plus the hard-failure semantics.

Refs #222.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
2026-07-31 12:12:55 +02:00
919201d9b0 #230: IT-Grundschutz mapping for APP.3.1 and CON.11.1
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m56s
CI / Build container images (pull_request) Successful in 1m23s
CI / Auth e2e pack (pull_request) Successful in 8m57s
CI / Import/export fidelity gate (pull_request) Successful in 1m1s
CD / Build and push images (push) Successful in 22s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 5m48s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m13s
CI / Import/export fidelity gate (push) Successful in 58s
docs/vs-nfd/80-grundschutz-mapping.md against the Edition 2023 texts
of both building blocks (fetched from the BSI single PDFs, edition and
retrieval date stated; the dropped requirements of APP.3.1 are listed
as such, CON.11.1's 18 requirements are all Basis). Every requirement
classified as product / operator / n.a.: product rows point at code,
configuration and tests (auth+rate limits, upload controls, security
headers with the honest HSTS-at-the-proxy split, marking = the whole of
M26 under CON.11.1.A7 incl. the answered does-the-marking-carry-a-
security-function question); operator rows say what we hand over
(copy list, procedures, network plan, SBOMs); n.a. rows are argued via
the delimitation statement (no §52 security functions, no built-in
remote maintenance). Open requirements point at their closing issues
(M27/M28/M29/M32), so the document doubles as the gap list; the
never-scheduled external pentest is stated honestly.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 10:35:26 +02:00
87c1c5ee88 #231: residual-risk list
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 7m1s
CI / Build container images (pull_request) Successful in 1m16s
CI / Auth e2e pack (pull_request) Successful in 8m48s
CI / Import/export fidelity gate (pull_request) Successful in 59s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
docs/vs-nfd/90-restrisiken.md: nine entries, each with risk, why it is
accepted, compensating control and decider — unmarked attachment
content (#212), local auth not yet switchable incl. the open runtime-
flippability question (#216), deferred plugin hash pinning (#232), the
one-time git-history secret check with its pattern caveat (#198),
digest-mail titles (I-23, revisit M32), page_links slug residue (I-24),
the IndexedDB endpoint copy (I-25), deliberately unscheduled features,
and the Site-Admin read bypass. Binding same-PR maintenance rule
stated; referenced from the delimitation statement and consumed by the
Grundschutz mapping.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 10:35:26 +02:00
2c6eff85f3 #227: hardening guide with the VS-NfD reference configuration
docs/vs-nfd/50-haertungsleitfaden.md: one adoptable profile — every
entry with the exact switch name, value, default and the reason, split
into instance settings (registration closed, api/mcp off, feeds off,
plugins off, classification defaults vs_nfd + upload block, svg reject,
minimal extension list) and deploy-level configuration (empty
BACKUP_ALLOWED_TARGETS enforces backup-local-only outside Site-Admin
reach; tightened session hours; SMTP deliberately unconfigured with the
consequence stated honestly). auth.local.enabled is listed as the one
pending row (#216) with its compensation until then; the guide states
the binding updated-in-same-PR rule for every future switch. Includes
an operator verification checklist (four unauthenticated 404 curls +
readyz + admin spot checks). Cross-referenced from the delimitation
statement (file names made concrete) and consumed by the Grundschutz
mapping (#230).

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 10:35:26 +02:00
040f3fbeae #229: operations manual (install, update, backup/restore, deletion, roles)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 7m1s
CI / Build container images (pull_request) Successful in 1m21s
CI / Auth e2e pack (pull_request) Successful in 8m47s
CI / Import/export fidelity gate (pull_request) Successful in 58s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
docs/vs-nfd/70-betriebshandbuch.md: installation as run on the real
stages (airgap variant explicitly pending #218-#221 with what already
exists as groundwork), update/rollback incl. the no-down-migrations
caveat, backup/restore with the ADR-0026 target allowlist and the
rehearsed monthly restore drill (evidence: logs on #98), the full
scheduler-job table (cadences verified against code), the deletion-and-
destruction chapter built on the #228 copy list (per content type:
what deletion reaches, what remains, immediate-destruction path,
decommissioning), and role separation incl. the deliberate limits of a
Site Admin and the honest note that Site Admin read-bypass makes the
content/platform split non-absolute app-side. Every procedure carries
its evidence level (erprobt / nicht geprobt / offen) — nothing claimed
above what was actually executed.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 10:35:26 +02:00
f0c6af4412 #228: security documentation (architecture, data flows, network plan)
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 7m0s
CI / Build container images (pull_request) Successful in 1m22s
CI / Auth e2e pack (pull_request) Successful in 9m2s
CI / Import/export fidelity gate (pull_request) Successful in 1m2s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
docs/vs-nfd/60-sicherheitsdokumentation.md: component diagram with per-
service purpose and privileges, network plan digit-exact against the
deploy compose (127.0.0.1-only app bindings, internal-only data zone),
data-flow diagrams (auth, realtime editing incl. LISTEN/NOTIFY and the
60s collab token, export via the pinned sidecars, backup incl. the
ADR-0026 allowlist, and every read channel), named trust boundaries
(reverse proxy, plugin sandbox, outbound SMTP/mirror), and the complete
list of content copies — in-database, on-volume and outside the
instance — that the deletion concept in #229 builds on. Mermaid only,
German (assessor audience), with the maintained-in-same-PR rule stated.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 10:35:26 +02:00
868b79c8bc #213: warn on uploads to classified pages; instance policy can block
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m15s
CI / Build container images (pull_request) Successful in 4m27s
CI / Auth e2e pack (pull_request) Successful in 9m10s
CI / Import/export fidelity gate (pull_request) Successful in 53s
CD / Build and push images (push) Successful in 17s
CD / Deploy to Test (push) Successful in 15s
CD / Smoke tests against Test (push) Successful in 1m16s
CD / Promote to Int (push) Successful in 20s
CI / Lint, typecheck, test (push) Successful in 5m47s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m26s
CI / Import/export fidelity gate (push) Successful in 1m0s
The attachments panel of a classified page shows a persistent notice
naming the consequence (de+en): the file inherits the page's
classification but its content carries no marking (#212). The new
instance setting classification.uploadPolicy (default warn, documented;
the VS-NfD reference configuration blocks, #227) hardens the warning
into a server-side rejection (403 classified_upload_blocked) — enforced
in the upload service, not only in the UI. Tests: warning visible in the
local attachments pack; block enforced server-side with warn/block both
ways and open pages unaffected.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 07:33:34 +02:00
e505fc74dc #212: mark attachment downloads by filename prefix and companion file
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m34s
CI / Build container images (pull_request) Successful in 14s
CI / Auth e2e pack (pull_request) Successful in 9m36s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
Downloads whose effective classification is vs_nfd carry the documented
VS-NfD_ filename prefix (single source classificationFilenamePrefix() in
shared; ADR 0022 records the short form for file names). Effective
classification: the linked page's level; an attachment with unset pageId
(paste-then-insert, pond-level) FAILS CLOSED to the highest level of any
live page in its pond. The pond export ZIP adds a sibling
<file>.classification.txt companion with the full marking for classified
media, next to the manifest entry (#210). Documented in operations.md,
incl. the deliberate residual risk: the file's own content carries no
marking (recorded on #231, not hidden). Tests: prefixed classified
download, unchanged open download, fail-closed orphan both ways, ZIP
companion + manifest level.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 07:29:16 +02:00
521ea514b4 #211: classification through feeds, public API, search and the no-JS shell
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m38s
CI / Build container images (pull_request) Successful in 4m14s
CI / Auth e2e pack (pull_request) Successful in 9m7s
CI / Import/export fidelity gate (pull_request) Successful in 1m6s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
Feeds: classified entries carry a standard Atom <category>
(term=level, scheme=urn:dorfteich:classification, label=the fixed
wording); the feed document states the highest contained level once;
all-open feeds carry none. Public API: page representations (list+get)
gain the classification field, OpenAPI + public-api.md documented.
Search: every hit carries the level and the palette renders the marking
with the snippet (compact form of the banner, text token only). No-JS
shell: banner above and below the content, own markup for the separate
render path; unclassified pages unchanged everywhere. One test per
channel (feed categories + count, public API list/get with the switch
on, search hit levels, shell top+bottom).

Also: fidelity CI sidecars get per-job container names — the fixed
names collided across parallel runs on the shared host (run 547's red
fidelity job; a fixed-name cleanup could even kill a sibling's live
sidecars).

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 07:23:53 +02:00
68497046e9 #210: mark the Markdown ZIP export with frontmatter, imprint and manifest
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m1s
CI / Build container images (pull_request) Successful in 2m58s
CI / Auth e2e pack (pull_request) Successful in 9m6s
CI / Import/export fidelity gate (pull_request) Successful in 1m8s
CI / Import/export fidelity gate (push) Blocked by required conditions
CD / Build and push images (push) Successful in 24s
CI / Lint, typecheck, test (push) Successful in 6m37s
CD / Deploy to Test (push) Successful in 12s
CI / Build container images (push) Has been skipped
CD / Smoke tests against Test (push) Successful in 1m25s
CD / Promote to Int (push) Successful in 13s
CI / Auth e2e pack (push) Has been cancelled
A classified page's .md carries the level in YAML frontmatter AND the
marking line at top and bottom; unclassified files are byte-identical to
before. Every pond archive (incl. the per-pond folders of the account
data export) ships a manifest.json listing each file with its level and
stating the highest level once at archive level — media inherits the
highest classification among the readable pages referencing it
(fail-closed). Round trip: the importer recognizes exactly our
frontmatter block, strips it plus the imprint lines, and creates the
page at least at the imported level (content must not escape its marking
by traveling through a ZIP) — pinned by unit and e2e round-trip tests.
Foreign frontmatter passes through unchanged; the Obsidian vault import
keeps its own frontmatter modes.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 07:13:53 +02:00
74a9e495e4 #209: pandoc reference documents carry the VS-NfD marking for DOCX/ODT
Some checks failed
CI / Lint, typecheck, test (pull_request) Failing after 6m34s
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Build container images (pull_request) Has been skipped
CI / Auth e2e pack (pull_request) Has been skipped
reference-vs-nfd.docx/.odt ship as derived binaries: the pinned pandoc's
default reference documents plus a header and footer with the marking —
part of the document's page setup, so it repeats on every page in Word
and LibreOffice and is not deletable body text. Source of truth is
scripts/gen-classified-reference-docs.mjs (wording from shared
classificationMarking(); maintenance documented in assets/README.md).
The converter passes reference docs to pandoc-server via in-request
files + reference-doc; the worker attaches them for marked docx/odt jobs
(job option {marking}, as in #208). Unclassified exports pass nothing
and are unchanged (pinned by fake-converter test). Fidelity suite
asserts against real pandoc 3.6 that marked outputs carry the
header/footer parts and unmarked ones do not; per-page repetition
verified via LibreOffice 25.8 headless PDF (5/5 pages, 2 markings each,
both formats). Word: quick manual look pending (sample files in the
workspace), procedure documented in assets/README.md.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 07:09:47 +02:00
c2df7c0c23 #208: VS-NfD marking in the Gotenberg per-page header and footer
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m16s
CI / Build container images (pull_request) Successful in 3m0s
CI / Auth e2e pack (pull_request) Successful in 8m54s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Promote to Int (push) Blocked by required conditions
CD / Build and push images (push) Successful in 24s
CD / Deploy to Test (push) Successful in 11s
CD / Smoke tests against Test (push) Successful in 1m27s
CI / Lint, typecheck, test (push) Failing after 6m24s
CI / Auth e2e pack (push) Has been skipped
CI / Import/export fidelity gate (push) Has been skipped
CI / Build container images (push) Has been skipped
A classified page's PDF export carries its marking as a job option; the
renderer hands it to Gotenberg's Chromium header/footer templates, so it
repeats on every page — bold centered in the running header and next to
the existing page numbers in the footer. Unclassified pages send exactly
the pre-#208 forms (unchanged PDF, asserted by the fidelity smoke and a
lastMarking=null check). New real-Gotenberg fidelity test asserts the
marking appears twice on EVERY page of a multi-page render while the
document-level header keeps working.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 06:55:52 +02:00
809e071f14 #207: print stylesheet with the classification on every printed sheet
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m56s
CI / Build container images (pull_request) Successful in 1m27s
CI / Auth e2e pack (pull_request) Successful in 9m18s
CI / Import/export fidelity gate (pull_request) Successful in 1m6s
CI / Import/export fidelity gate (push) Blocked by required conditions
CD / Build and push images (push) Successful in 20s
CD / Deploy to Test (push) Successful in 11s
CD / Smoke tests against Test (push) Successful in 1m25s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m5s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Has been cancelled
First @media print support at all: page size/margins, navigation and
interactive chrome suppressed, break behaviour for headings, tables,
code blocks, figures and plugin blocks. The VS-NfD marking runs as
header AND footer on every sheet via a real-table PrintFrame whose
thead/tfoot browsers repeat per page — @page margin boxes are
unimplemented and position:fixed places unreliably in both engines
(verified empirically); on screen the table chain renders as plain
blocks, so nothing changes visually. Verified as PDF-from-browser in
Chromium 140 and Firefox 153 (2 markings on every page of a multi-page
document); the repeatable procedure is documented in
apps/web/e2e/README.md. Unclassified pages print without a marking.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 06:48:44 +02:00
adceca7358 #206: show the VS-NfD marking in web view header and footer
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m24s
CI / Build container images (pull_request) Successful in 4m24s
CI / Auth e2e pack (pull_request) Successful in 8m44s
CI / Import/export fidelity gate (pull_request) Successful in 59s
CD / Build and push images (push) Successful in 26s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Successful in 1m30s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 6m10s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m55s
CI / Import/export fidelity gate (push) Failing after 50s
ClassificationBanner renders the fixed ADR-0022 wording above and below
the content in reading view, editor and public page view; unclassified
pages show nothing. Announced to assistive tech via a localized hidden
prefix (de+en); styled from the plain text token only, so contrast holds
in both themes and under every accent with no new color pair. Public
content endpoint now carries the classification. New seed fixture
classified-note; a11y pack asserts banner top+bottom and axe-clean in
light and dark.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 06:18:47 +02:00
488d0d06f1 #205: classification inherits down the tree; lowering is a guarded, audited act
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m40s
CI / Build container images (pull_request) Successful in 4m34s
CI / Auth e2e pack (pull_request) Successful in 9m7s
CI / Import/export fidelity gate (pull_request) Successful in 1m0s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
New pages take max(instance default, parent level); moving a subtree
under a higher-classified parent raises every member below that level.
No move-like path (reposition, trash-promote, purge-promote) lowers a
level as a side effect — pinned by test. Raising is ordinary editorial
work; lowering requires the dedicated capability canLowerClassification
(pond-wide Pond Admin) in the central permission model. Both directions
are audited (page.classification_raised/_lowered, catalogue v1.1) with
old value, new value, actor and page.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 06:12:26 +02:00
183faf7710 #204: classification as first-class page metadata (ADR 0022)
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m42s
CI / Build container images (pull_request) Successful in 3m56s
CI / Auth e2e pack (pull_request) Successful in 8m17s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 24s
CD / Deploy to Test (push) Successful in 10s
CI / Lint, typecheck, test (push) Successful in 6m17s
CD / Smoke tests against Test (push) Successful in 3m32s
CI / Build container images (push) Has been skipped
CD / Promote to Int (push) Successful in 13s
CI / Auth e2e pack (push) Successful in 8m20s
CI / Import/export fidelity gate (push) Successful in 55s
Enum field on Page (UNCLASSIFIED default, VS_NFD), migration backfills
existing pages. New pages take the instance-wide default from
classification.newPageDefault (admin-visible, de+en). The value rides in
every PageView, so no channel needs an extra request. The field is a
marking, not a protection mechanism: a test pins that permission
decisions are unchanged by it. The marking wording is fixed in ADR 0022
and sourced solely from classificationMarking() in @dorfteich/shared.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
2026-07-31 06:01:50 +02:00
db4f517e44 #203: pin all third-party deploy images by digest
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m32s
CI / Build container images (pull_request) Successful in 1m13s
CI / Auth e2e pack (pull_request) Successful in 8m22s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Build and push images (push) Successful in 16s
CD / Deploy to Test (push) Successful in 56s
CD / Smoke tests against Test (push) Successful in 1m24s
CD / Promote to Int (push) Successful in 52s
CI / Lint, typecheck, test (push) Successful in 5m36s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m3s
CI / Import/export fidelity gate (push) Successful in 57s
The four third-party images in the deploy compose (postgres, pandoc,
gotenberg — previously a floating MAJOR tag —, caddy) are now
name:tag@sha256 pins; the tag stays for readability, the digest decides
what runs. The pinned digests are exactly what the stages already run
(verified against the live containers' RepoDigests on ONE), so the next
recreation is byte-identical. A new early CI step fails on any
third-party compose image without a digest; compose.dev.yml is a local
convenience and deliberately exempt (its node helpers now follow the
#236 pin). Update + rollout procedure in deploy/stages.md — CD does not
sync stage composes, so the hand rollout to test/int/prod is part of
this issue's definition of done.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-31 05:13:13 +02:00
000d110727 #201: stable audit event catalogue for syslog/SIEM export
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 6m9s
CI / Build container images (pull_request) Successful in 3m5s
CI / Auth e2e pack (pull_request) Successful in 8m40s
CI / Import/export fidelity gate (pull_request) Successful in 1m2s
CD / Build and push images (push) Successful in 20s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m40s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m10s
CI / Import/export fidelity gate (push) Successful in 57s
The 36 audit action ids become a typed union (AUDIT_EVENTS in
audit-actions.ts) — an uncatalogued id is now a compile error; every
existing id keeps its name. The published, versioned catalogue
(docs/architecture/audit-events.md, v1.0) documents per event: trigger,
severity, actor and target semantics, and every field, plus the
compatibility promise (ids are never repurposed; retiring keeps the row
forever) and the stable stdout field set. audit-catalogue.test.ts is
the fence: it parses the document's event tables and fails when ids or
severities drift from the code (negative case verified). Audit stdout
lines now carry the catalogue severity as a routing hint — pino level
stays 30 so transport is unaffected; no DB migration.

Forwarding path documented: container stdout -> operator's collector;
deliberately no application-side syslog client.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-31 04:51:09 +02:00
c4c84b33f9 #200: hard instance-wide plugins.enabled kill switch
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m38s
CI / Build container images (pull_request) Successful in 4m11s
CI / Auth e2e pack (pull_request) Successful in 8m55s
CI / Import/export fidelity gate (pull_request) Successful in 1m9s
CI / Import/export fidelity gate (push) Blocked by required conditions
CD / Build and push images (push) Successful in 20s
CD / Deploy to Test (push) Failing after 51s
CD / Smoke tests against Test (push) Has been skipped
CD / Promote to Int (push) Has been skipped
CI / Lint, typecheck, test (push) Successful in 5m37s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Has been cancelled
plugins.enabled (instance setting, default on — plugins predate the
switch; the VS-NfD reference configuration turns it off) makes every
plugin surface answer 404 via a shared guard: Site-Admin
install/list/mode, pond activation and plugin list, the sandbox frame
and asset routes. The dropzone watcher quarantines drops instead of
installing. Deliberately NOT guarded: the authenticated
fallback-metadata route — it serves no plugin code and existing
plugin_block nodes need it to render their declared fallback (an image
fallback degrades to the neutral placeholder while off, because its
bytes live on the disabled asset surface). The editor offers no plugin
blocks because the pond plugin list is one of the 404ing surfaces.
Admin settings panel gets the toggle (i18n de+en) with the documented
api-restart note (in-process settings cache).

Answers "code execution inside the zone?" with one verifiable
off-switch instead of per-plugin trust machinery (#232, ADR 0025).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-31 04:42:42 +02:00
74970f6073 #199: SHA-256 integrity hashes for attachments
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 6m12s
CI / Build container images (pull_request) Successful in 3m4s
CI / Auth e2e pack (pull_request) Successful in 8m35s
CI / Import/export fidelity gate (pull_request) Successful in 1m2s
CI / Import/export fidelity gate (push) Blocked by required conditions
CD / Build and push images (push) Successful in 29s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m35s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 6m10s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Has been cancelled
Every upload stores the SHA-256 of its bytes, computed from the
in-memory buffer that is written — never by re-reading disk. Every
download re-hashes the stored object BEFORE the first byte leaves
(memory bounded by the max_file_bytes quota that gated the upload) and
fails closed on mismatch with attachment_integrity_failure; the
mismatch lands in the audit trail as file.integrity_failed with both
hashes. Detection of payload manipulation is the one integrity duty
par. 52 VSA leaves with the application — only it knows what the file
should be.

Pre-#199 rows are hashed by a bounded, idempotent backfill that rides
the existing nightly orphan-file-sweep job (no new scheduler job, job
fence untouched); unreadable files are logged and retried, never
silently skipped, and null-hash rows are served unverified only until
the backfill reaches them. Operator runbook note in security.md
(restore from backup, re-download, audit entry carries both hashes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-31 04:32:29 +02:00
d3289b2167 #202: SBOM and license report in CI
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m33s
CI / Build container images (pull_request) Successful in 4m38s
CI / Auth e2e pack (pull_request) Successful in 9m14s
CI / Import/export fidelity gate (pull_request) Successful in 1m12s
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Deploy to Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CI / Lint, typecheck, test (push) Waiting to run
CI / Import/export fidelity gate (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
The release run now generates CycloneDX 1.6 SBOMs with a pinned
anchore/syft container — one per released image (scanned from the
freshly built image tar, OS packages included) and one for the pnpm
workspace (from the lockfile) — plus the full pnpm licenses report, and
attaches everything as build artefacts BEFORE publishing the release,
so a red gate stops the release. Runner constraints dictated the
mechanics (documented in the workflow): the job talks to the HOST
daemon, so files travel into the syft container via docker cp and
images via docker save to a tar copied the same way (syft cannot read
a tar from stdin — verified).

scripts/check-licenses.mjs is the documented license policy: permissive
allowlist, MPL-2.0/CC-BY-4.0 with recorded reasoning, per-package
exception table (khroma: MIT text shipped, metadata missing). CI runs
the gate on every PR (pnpm licenses:check); positive and negative case
tested locally, both SBOM paths tested against real images/lockfile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-31 04:21:58 +02:00
9326177534 #236: also pin the node helper images in workflows
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m41s
CI / Build container images (pull_request) Successful in 3m9s
CI / Auth e2e pack (pull_request) Successful in 8m32s
CI / Import/export fidelity gate (pull_request) Successful in 1m7s
CD / Build and push images (push) Successful in 21s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 6m0s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 8m32s
CI / Import/export fidelity gate (push) Successful in 1m3s
release.yml and drill.yml ran throwaway `docker run node:22.15-alpine`
helpers outside the pin; the drift check now also fails on any
node:<other>-alpine reference in .gitea/workflows.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-31 04:16:07 +02:00
6a520e27b1 #236: pin the Node version
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m40s
CI / Build container images (pull_request) Successful in 4m15s
CI / Auth e2e pack (pull_request) Successful in 8m33s
CI / Import/export fidelity gate (pull_request) Successful in 59s
.node-version (22.15.1) becomes the single authoritative Node version:
CI/CD select Node only via node-version-file, every Dockerfile pins
node:22.15.1-alpine, and the engines floor in package.json states the
same version (open-ended upwards so a newer local Node keeps working —
reproducibility rests on images and CI). An early CI step fails on any
drift between those places; update procedure in operations.md
(Update strategy). Precondition for the reproducibility claim in #219.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-31 04:14:55 +02:00
9a43a2f6bb #235: keep page_links rows pointing at purged pages — recorded decision
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m53s
CI / Build container images (pull_request) Successful in 1m11s
CI / Auth e2e pack (pull_request) Successful in 8m6s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 14s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m16s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m19s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m39s
CI / Import/export fidelity gate (push) Successful in 58s
The row is only the index of a wikilink whose text (slug = title)
remains visible in the linking page's own content either way; deleting
the index would remove nothing the system still shows while breaking
phantom-link re-resolution. Kept as an accepted residue, reasoning in
operations.md (deletion/purge section) and recorded on #231.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 22:18:39 +02:00
69d9072d2c #234: retention for mail_outbox
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m51s
CI / Build container images (pull_request) Successful in 3m4s
CI / Auth e2e pack (pull_request) Successful in 8m4s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
Sent mails were kept forever, and digest bodies name page titles and
actors — an unbounded copy of content-adjacent data. A new daily
mail-outbox-retention job deletes SENT rows (by sentAt) and permanently
FAILED rows (by nextAttemptAt, the last attempt's stamp) once they pass
mail.outboxRetentionDays (instance setting, default 30). PENDING rows —
including failed-but-retryable ones — stay the retry loop's alone.

Decision recorded (security.md §Privacy, residual-risk note for #231):
digest mails keep carrying page titles for now — there is no per-page
classification marking yet to key a suppression on (ADR 0022 / M32
revisits), and a VS-NfD reference configuration can leave SMTP
unconfigured entirely.

Job-count fence in system.spec: 8 -> 9.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 22:17:28 +02:00
ff505bc752 #233: prune conversion job payloads for every job kind
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m12s
CI / Build container images (pull_request) Successful in 3m28s
CI / Auth e2e pack (pull_request) Successful in 8m33s
CI / Import/export fidelity gate (pull_request) Successful in 1m2s
CD / Build and push images (push) Successful in 29s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m22s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Failing after 5m9s
CI / Auth e2e pack (push) Has been skipped
CI / Import/export fidelity gate (push) Has been skipped
CI / Build container images (push) Has been skipped
The raw input/result bytes of import/export conversion jobs were kept
forever; a deleted classified page could live on inside its last export.
A new daily conversion-payload-prune job nulls both once a finished
(succeeded or failed) job passes conversion.payloadRetentionDays
(instance setting, default 30) — the row survives for status/audit.
PENDING and RUNNING rows keep their payload, so the worker's stale-lock
recovery path is untouched; a hand-requeued pruned job fails finally
via conversionInputOf instead of crashing the worker.

The input column becomes nullable; the migration backfills by clearing
payloads of jobs already finished longer ago than the default period
(recent results stay downloadable until they age out).

Job-count fence in system.spec: 7 -> 8 (new scheduler registration).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 22:12:32 +02:00
ff842f97e2 #198: CI fence — no tracked .env or secret material, example is authoritative
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m27s
CI / Build container images (pull_request) Successful in 1m16s
CI / Auth e2e pack (pull_request) Successful in 7m49s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 28s
CD / Smoke tests against Test (push) Successful in 1m32s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 5m14s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m41s
CI / Import/export fidelity gate (push) Failing after 10s
Verification result: only deploy/compose/.env.example was ever tracked
(full-history check), zero hits for obvious secret patterns across all
added lines in history — recorded on issue #231 (residual-risk list).

The new CI step in the checks job fails if any .env other than
.env.example is tracked or a tracked file matches an obvious secret
pattern (private key blocks, AWS/GitHub/GitLab/Slack token shapes).
.env.example already documents every variable the compose files
reference (verified: comm of compose ${VAR} refs vs example keys is
empty). README states the example as the authoritative reference.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 17:16:00 +02:00
3c62b7b773 #197: security response headers and an explicitly restrictive CORS policy
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m15s
CI / Build container images (pull_request) Successful in 1m9s
CI / Auth e2e pack (pull_request) Successful in 7m43s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m29s
CD / Promote to Int (push) Successful in 14s
CI / Lint, typecheck, test (push) Successful in 5m24s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m35s
CI / Import/export fidelity gate (push) Successful in 55s
Hand-rolled middleware instead of helmet: the header set is small enough
to own, every value is a deliberate decision, and the api gains no
transitive dependency. HSTS (no includeSubDomains — the api cannot speak
for sibling subdomains), nosniff, Referrer-Policy no-referrer,
X-Frame-Options SAMEORIGIN (not DENY: the plugin sandbox frame embeds
same-origin and its CSP has no frame-ancestors, so this header governs),
and a minimal deny-all Permissions-Policy.

CORS grants no foreign origin anything; only the APP_BASE_URL origin is
ever echoed (where browsers do not consult CORS anyway), with
Vary: Origin on every response. No preflight handling — same-origin
requests never preflight, and cross-origin API access is cookie-less by
design (PAT/Bearer).

Wired via the AppModule MiddlewareConsumer so createTestApp boots the
identical middleware. Fences: security-headers.e2e.test.ts (header set,
foreign origin gets no ACAO) and a frame assertion in
plugins.e2e.db.test.ts (framing stays possible). Rationale table in
security.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 17:13:24 +02:00
ed2225bb77 #196: audit-trail retention job
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m5s
CI / Build container images (pull_request) Successful in 2m48s
CI / Auth e2e pack (pull_request) Successful in 7m50s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 15s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m20s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m11s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m38s
CI / Import/export fidelity gate (push) Successful in 56s
audit.retentionDays (instance setting, default 365) bounds the audit_log:
the daily audit-retention job deletes entries past the period and records
the deletion itself (audit.pruned with count, cutoff and period) so a gap
in the trail is always explainable. Lives in its own AuditRetentionService
because the settings service audits its writes - folding retention into
AuditService would close a constructor cycle. The read-access trail
(#222-#225) is deliberately not covered; it gets its own period.

security.md gains the Logging section the schema has cited for a while;
the maintenance-job fence moves 6 -> 7 (the deliberate new row).

Refs #196

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 14:59:28 +02:00
960a806ee3 #195: trashed content leaves the search index itself
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m4s
CI / Build container images (pull_request) Successful in 2m47s
CI / Auth e2e pack (pull_request) Successful in 7m44s
CI / Import/export fidelity gate (pull_request) Successful in 55s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Successful in 1m22s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m9s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m53s
CI / Import/export fidelity gate (push) Successful in 53s
Trashing a page (promote and subtree modes) clears the affected search
vectors, restoring rebuilds them; pond trash clears every page vector of
the pond, pond restore reindexes only the live pages (pages trashed
inside stay out); the GDPR pseudonymization's personal-pond trash does
the same. reindexAll now converges to the invariant (clears trashed,
rebuilds live), and a one-off migration backfills vectors of
already-trashed content.

The query-side deleted_at guards stay untouched as the independent
second layer - the test proves both layers separately, including writing
a vector back onto a trashed page (simulating a future path that forgot
the clear) and asserting the query still hides it. New provider methods
removePond/reindexPond behind the SearchProvider seam.

Refs #195

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 14:07:34 +02:00
02c1f18fe1 adjust the maintenance-job count fence: 6 jobs with the orphan sweep
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 4m58s
CI / Auth e2e pack (pull_request) Successful in 7m50s
CI / Import/export fidelity gate (pull_request) Successful in 53s
CI / Build container images (pull_request) Successful in 1m12s
CD / Build and push images (push) Successful in 16s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m21s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 5m5s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m36s
CI / Import/export fidelity gate (push) Successful in 55s
The system panel spec pins the registered-job count on purpose; the
orphan-file-sweep registration (#194) is the deliberate sixth row (CI
run 493 caught exactly this, 14x resolved to 6).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 13:31:03 +02:00
0bc36aa58c #194: orphan-file sweep, drop the unused Attachment.deletedAt
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m1s
CI / Build container images (pull_request) Successful in 2m48s
CI / Auth e2e pack (pull_request) Failing after 3m12s
CI / Import/export fidelity gate (pull_request) Has been skipped
Nightly sweep with two directions: attachments still unclaimed (pageId
null) after a 24 h grace period - claimed by no collab persist, page
upload, or import - are reclaimed (row, file, quota released); files on
the uploads volume without a database row (drift after a crashed
upload) are removed once older than the grace period. The grace period
protects the paste-then-insert window.

Deliberate deviation from the issue's content-reference idea, documented
in schema comment and operations.md: claimed attachments whose page
content no longer embeds them are NOT auto-deleted. The page attachments
panel lists claimed files as user-managed objects (inserting into the
document is optional there), so 'not embedded' is not 'unused' - an
auto-delete would destroy panel assets. Humans clean those up in the
panel or the pond file manager, which flags orphans already.

Attachment.deletedAt is removed by migration - deletion is hard
everywhere (sweep, purge, manual), there is no soft-delete state; the
never-true deletedAt:null filters in files/export queries went with it.

Refs #194

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 13:19:58 +02:00
370 changed files with 22755 additions and 1091 deletions

View File

@ -86,7 +86,7 @@ jobs:
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
node-version-file: .node-version
cache: pnpm
- name: Install dependencies

View File

@ -38,18 +38,112 @@ jobs:
- name: Check out repository
uses: actions/checkout@v4
# Fails if a real .env (anything but .env.example) is ever tracked, or
# if a tracked file matches an obvious secret pattern (issue #198).
# .env.example is the authoritative reference; real values never enter
# the repository (docs/self-hosting/README.md).
- name: No tracked .env files or secret material
run: |
set -euo pipefail
bad_env=$(git ls-files | grep -E '(^|/)\.env(\.[^/]*)?$' | grep -v '\.env\.example$' || true)
if [ -n "$bad_env" ]; then
echo "tracked .env file(s) — only .env.example may be tracked:"
echo "$bad_env"
exit 1
fi
secrets=$(git grep -nIE -e '-----BEGIN [A-Z ]*PRIVATE KEY-----|AKIA[0-9A-Z]{16}|ghp_[A-Za-z0-9]{36}|glpat-[A-Za-z0-9_-]{20}|xox[baprs]-[0-9A-Za-z-]{10}' -- . || true)
if [ -n "$secrets" ]; then
echo "tracked file matches a secret pattern:"
echo "$secrets"
exit 1
fi
# One authoritative Node version (issue #236): `.node-version` is the
# pin; every Dockerfile image tag and the engines floor must match it
# exactly, and workflows select Node only through node-version-file.
# Raising Node = update .node-version, every `FROM node:` tag and the
# engines floor in ONE commit (procedure: docs/architecture/operations.md
# §Update strategy). The bracketed grep pattern keeps this step from
# matching its own source (same trick as the secret fence above).
- name: Node version pin is consistent
run: |
set -euo pipefail
ver="$(cat .node-version)"
echo "pinned Node version: $ver"
bad=0
for f in apps/*/Dockerfile; do
if grep '^FROM node:' "$f" | grep -v "node:${ver}-alpine"; then
echo "$f pins a different Node image than node:${ver}-alpine"
bad=1
fi
done
if grep -rn "node-version[:] " .gitea/workflows; then
echo "workflows must use node-version-file, not a literal version"
bad=1
fi
if grep -rnE 'node:[0-9][^ ]*-alpine' .gitea/workflows | grep -v "node:${ver}-alpine"; then
echo "a workflow references a different node image than node:${ver}-alpine"
bad=1
fi
if ! grep -q "\"node\": \">=${ver}\"" package.json; then
echo "package.json engines floor does not match ${ver}"
bad=1
fi
exit "$bad"
# Third-party deploy images are pinned by digest (issue #203): every
# image in the deploy compose that is not one of our own
# (${IMAGE_PREFIX}…) must carry @sha256 — the tag stays for
# readability, the digest decides what runs. Update procedure:
# deploy/stages.md §Third-party image digests. compose.dev.yml is a
# local convenience, deliberately not held to this.
- name: Third-party compose images are digest-pinned
run: |
set -euo pipefail
bad=$(grep -hE '^ *image: ' deploy/compose/docker-compose.yml | grep -v 'IMAGE_PREFIX' | grep -v '@sha256:' || true)
if [ -n "$bad" ]; then
echo "third-party image reference(s) without a digest:"
echo "$bad"
exit 1
fi
# A fresh named volume inherits the ownership of the image directory it
# is mounted over. Every /data/… path the api image defaults to must
# therefore be pre-created AND chowned to `node`, or the non-root user
# cannot write to it — found on a real deploy in #303, where the env
# entry was added but the mkdir/chown line was not.
- name: api image pre-creates its data directories node-owned
run: |
set -euo pipefail
dirs=$(grep -oE '[A-Z_]+_DIR=/data/[a-z]+' apps/api/Dockerfile | cut -d= -f2 | sort -u)
bad=0
for d in $dirs; do
grep -q "mkdir -p .*$d" apps/api/Dockerfile || {
echo "$d is not pre-created in apps/api/Dockerfile"; bad=1; }
grep -q "chown -R node:node .*$d" apps/api/Dockerfile || {
echo "$d is not chowned to node in apps/api/Dockerfile"; bad=1; }
done
exit "$bad"
- name: Set up pnpm
uses: pnpm/action-setup@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
node-version-file: .node-version
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
# License allowlist gate (issue #202): fails when any dependency's
# license falls outside the documented policy in
# scripts/check-licenses.mjs (which is also where the reasoning and
# per-package exceptions live).
- name: License allowlist
run: pnpm licenses list --json | node scripts/check-licenses.mjs
# Build first: package type checks resolve @dorfteich/shared through
# its built dist, and i18n:check imports the built helpers.
- name: Build all packages
@ -99,7 +193,7 @@ jobs:
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
node-version-file: .node-version
cache: pnpm
- name: Install dependencies
@ -116,7 +210,16 @@ jobs:
- name: Start api, collab, and static web server
run: |
(cd apps/api && PORT=3001 node dist/main.js > /tmp/api.log 2>&1 &)
# VS_NFD_MODE=marked: the marking pack and the a11y admin scan
# cover the marked state (issue #244); mode off is covered by
# local full runs and the marking pack's off-assertions there.
(cd apps/api && PORT=3001 VS_NFD_MODE=marked node dist/main.js > /tmp/api.log 2>&1 &)
# Second api on the SAME database with VS_NFD_MODE=hidden: the
# marking pack's hidden half runs against it via its own static
# server (issue #245); the mode is env-only, so sharing the db is
# exactly the deploy semantics.
(cd apps/api && PORT=3006 VS_NFD_MODE=hidden MIGRATE_ON_START=false node dist/main.js > /tmp/api-hidden.log 2>&1 &)
(PORT=5176 API_TARGET=http://127.0.0.1:3006 node scripts/e2e-static-server.mjs > /tmp/web-hidden.log 2>&1 &)
(cd apps/collab && PORT=3002 node dist/index.js > /tmp/collab.log 2>&1 &)
(PORT=5173 COLLAB_TARGET=http://127.0.0.1:3002 node scripts/e2e-static-server.mjs > /tmp/web.log 2>&1 &)
for i in $(seq 1 30); do
@ -242,6 +345,16 @@ jobs:
E2E_BASE_URL=http://localhost:5173 \
pnpm --filter @dorfteich/web exec playwright test e2e/social.spec.ts
- name: Reset login rate limit before admin-settings pack
run: |
echo "DELETE FROM rate_limits WHERE key LIKE 'login%';" | \
pnpm --filter @dorfteich/api exec prisma db execute --stdin --url "$DATABASE_URL"
- name: Run admin-settings pack
run: |
E2E_BASE_URL=http://localhost:5173 \
pnpm --filter @dorfteich/web exec playwright test e2e/admin-settings.spec.ts
- name: Reset login rate limit before admin-quotas pack
run: |
echo "DELETE FROM rate_limits WHERE key LIKE 'login%';" | \
@ -262,6 +375,18 @@ jobs:
E2E_BASE_URL=http://localhost:5173 \
pnpm --filter @dorfteich/web exec playwright test e2e/admin-users.spec.ts
- name: Reset login rate limit before invitations pack
run: |
echo "DELETE FROM rate_limits WHERE key LIKE 'login%';" | \
pnpm --filter @dorfteich/api exec prisma db execute --stdin --url "$DATABASE_URL"
# Invitations (issue #332) need the mail catcher like the auth pack:
# the invite link and the follow-up verification both travel by mail.
- name: Run invitations pack
run: |
E2E_BASE_URL=http://localhost:5173 E2E_MAILPIT_URL=http://mailpit:8025 \
pnpm --filter @dorfteich/web exec playwright test e2e/invitations.spec.ts
- name: Reset login rate limit before permission-matrix pack
run: |
echo "DELETE FROM rate_limits WHERE key LIKE 'login%';" | \
@ -535,6 +660,37 @@ jobs:
E2E_BASE_URL=http://localhost:5173 \
pnpm --filter @dorfteich/web exec playwright test e2e/a11y.spec.ts
# Das a11y-Pack kostet seit #301 einen Login mehr (der Reflow-Zaun);
# damit reicht das Budget nicht mehr bis in die VS-NfD-Packs → hier
# zusätzlich zurücksetzen (siehe Hinweis oben).
- name: Reset login rate limit before the VS-NfD packs
run: |
echo "DELETE FROM rate_limits WHERE key LIKE 'login%';" | \
pnpm --filter @dorfteich/api exec prisma db execute --stdin --url "$DATABASE_URL"
# VS-NfD-Markierungen im Modus `marked` (issue #244).
- name: Run VS-NfD marking pack
run: |
E2E_BASE_URL=http://localhost:5173 E2E_VS_NFD_MODE=marked \
pnpm --filter @dorfteich/web exec playwright test e2e/vs-nfd-marking.spec.ts
# Ausblendung + Policy-Hinweis im Modus `hidden` (issue #245).
- name: Run VS-NfD hidden pack
run: |
for i in $(seq 1 30); do
curl -sf http://localhost:3006/api/v1/readyz >/dev/null && break
sleep 2
done
E2E_BASE_URL=http://localhost:5176 E2E_VS_NFD_MODE=hidden \
pnpm --filter @dorfteich/web exec playwright test e2e/vs-nfd-marking.spec.ts
# The marking pack's extra login on top of the six a11y logins pushes
# the theme pack over the 10/min login limit — reset again (#244).
- name: Reset login rate limit before theme pack
run: |
echo "DELETE FROM rate_limits WHERE key LIKE 'login%';" | \
pnpm --filter @dorfteich/api exec prisma db execute --stdin --url "$DATABASE_URL"
# Hell/Dunkel/System-Umschalter (issue #180).
- name: Run theme pack
run: |
@ -574,7 +730,7 @@ jobs:
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: 22
node-version-file: .node-version
cache: pnpm
- name: Install dependencies
@ -598,13 +754,16 @@ jobs:
# image has no iproute2). Sharing the netns means no published ports.
- name: Start pinned pandoc + Gotenberg sidecars
run: |
# Clear any leftovers from an earlier interrupted run so the named
# containers never collide, and nothing leaks on the shared host.
docker rm -f fidelity-pandoc fidelity-gotenberg 2>/dev/null || true
# Sidecar names carry THIS job container's id: parallel runs on the
# shared host must not collide on a fixed name (a fixed-name rm -f
# here even killed a sibling run's live sidecars — run 547).
JOB_ID=$(cat /etc/hostname)
docker run -d --name fidelity-pandoc \
echo "PANDOC_NAME=fidelity-pandoc-${JOB_ID}" >> "$GITHUB_ENV"
echo "GOTENBERG_NAME=fidelity-gotenberg-${JOB_ID}" >> "$GITHUB_ENV"
docker rm -f "fidelity-pandoc-${JOB_ID}" "fidelity-gotenberg-${JOB_ID}" 2>/dev/null || true
docker run -d --name "fidelity-pandoc-${JOB_ID}" \
--network "container:${JOB_ID}" pandoc/core:3.6 server
docker run -d --name fidelity-gotenberg \
docker run -d --name "fidelity-gotenberg-${JOB_ID}" \
--network "container:${JOB_ID}" gotenberg/gotenberg:8
for i in $(seq 1 30); do
curl -sf http://localhost:3030/version >/dev/null && break
@ -628,15 +787,15 @@ jobs:
- name: Dump sidecar logs on failure
if: failure()
run: |
echo '--- pandoc ---'; docker logs fidelity-pandoc 2>&1 | tail -30 || true
echo '--- gotenberg ---'; docker logs fidelity-gotenberg 2>&1 | tail -30 || true
echo '--- pandoc ---'; docker logs "$PANDOC_NAME" 2>&1 | tail -30 || true
echo '--- gotenberg ---'; docker logs "$GOTENBERG_NAME" 2>&1 | tail -30 || true
# Always tear the sidecars down — they run on the shared runner host, so a
# leaked (especially Chromium-backed Gotenberg) container would waste its
# memory until the next run and break re-runs on the container name.
# memory until the next run.
- name: Stop sidecars
if: always()
run: docker rm -f fidelity-pandoc fidelity-gotenberg 2>/dev/null || true
run: docker rm -f "$PANDOC_NAME" "$GOTENBERG_NAME" 2>/dev/null || true
images:
name: Build container images

View File

@ -55,7 +55,7 @@ jobs:
} > comment.md
# JSON-encode via a node container — the runner image guarantees
# only git/curl/docker, not python or node.
docker run --rm -i node:22.15-alpine node -e \
docker run --rm -i node:22.15.1-alpine node -e \
'const fs=require("fs");process.stdout.write(JSON.stringify({body:fs.readFileSync(0,"utf8")}))' \
< comment.md > comment.json
curl -sf -X POST \

View File

@ -38,6 +38,62 @@ jobs:
docker push $IMAGE_BASE-$app:$TAG
done
# Supply-chain artefacts (issue #202): one CycloneDX SBOM per release
# image, one for the pnpm workspace, plus the full license report —
# attached as build artefacts of this run BEFORE the release is
# published, so a red gate stops the release. Mechanics dictated by
# the runner (the job talks to the HOST daemon, so bind mounts of
# workspace paths resolve on the host and go nowhere): files travel
# into the pinned syft container via `docker cp` (an API stream), and
# images via `docker save` to a tar copied the same way — syft cannot
# read a tar from stdin (not seekable).
- name: Generate SBOMs
run: |
set -euo pipefail
TAG=${GITHUB_REF_NAME}
SYFT=anchore/syft:v1.33.0
mkdir -p supply-chain sbom-src
cp pnpm-lock.yaml package.json sbom-src/
c=$(docker create $SYFT scan dir:/src --source-name dorfteich-workspace --source-version "$TAG" -o cyclonedx-json=/out.json)
docker cp sbom-src "$c:/src"
docker start -a "$c"
docker cp "$c:/out.json" supply-chain/sbom-workspace-$TAG.cdx.json
docker rm "$c" > /dev/null
for app in web api collab backup; do
docker save $IMAGE_BASE-$app:$TAG -o image.tar
c=$(docker create $SYFT scan docker-archive:/image.tar --source-name dorfteich-$app --source-version "$TAG" -o cyclonedx-json=/out.json)
docker cp image.tar "$c:/image.tar"
docker start -a "$c"
docker cp "$c:/out.json" supply-chain/sbom-image-$app-$TAG.cdx.json
docker rm "$c" > /dev/null
rm image.tar
done
ls -l supply-chain/
- name: Set up pnpm
uses: pnpm/action-setup@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version-file: .node-version
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: License report and allowlist gate
run: |
set -euo pipefail
pnpm licenses list --json > supply-chain/licenses-${GITHUB_REF_NAME}.json
node scripts/check-licenses.mjs < supply-chain/licenses-${GITHUB_REF_NAME}.json
- name: Attach supply-chain artefacts
uses: actions/upload-artifact@v3
with:
name: supply-chain-${{ github.ref_name }}
path: supply-chain/
- name: Generate release notes and publish the release
run: |
TAG=${GITHUB_REF_NAME}
@ -54,7 +110,7 @@ jobs:
echo '_No database migrations in this release._'
fi
} > notes.md
TAG=$TAG docker run --rm -i -e TAG node:22.15-alpine node -e \
TAG=$TAG docker run --rm -i -e TAG node:22.15.1-alpine node -e \
'const fs=require("fs");const body=fs.readFileSync(0,"utf8");process.stdout.write(JSON.stringify({tag_name:process.env.TAG,name:process.env.TAG,body}))' \
< notes.md > release.json
curl -sf -X POST \

1
.node-version Normal file
View File

@ -0,0 +1 @@
22.15.1

View File

@ -27,13 +27,3 @@ AA) — nicht nachträglich. Kurzfassung; Details und Begründung in
machen — betroffene Specs mit anpassen (scopen), nicht das Label opfern.
Verstöße gelten in Review und Abnahme als Funktionsfehler.
## graphify
This project has a knowledge graph at graphify-out/ with god nodes, community structure, and cross-file relationships.
Rules:
- For codebase questions, first run `graphify query "<question>"` when graphify-out/graph.json exists. Use `graphify path "<A>" "<B>"` for relationships and `graphify explain "<concept>"` for focused concepts. These return a scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output.
- If graphify-out/wiki/index.md exists, use it for broad navigation instead of raw source browsing.
- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain do not surface enough context.
- After modifying code, run `graphify update .` to keep the graph current (AST-only, no API cost).

View File

@ -1,7 +1,7 @@
# Build context is the repository root (workspace build):
# docker build -f apps/api/Dockerfile .
FROM node:22.15-alpine AS build
FROM node:22.15.1-alpine AS build
WORKDIR /repo
RUN npm install -g pnpm@11
COPY pnpm-workspace.yaml pnpm-lock.yaml package.json tsconfig.base.json ./
@ -21,25 +21,27 @@ RUN pnpm install --frozen-lockfile --filter @dorfteich/api... \
# needed for migrate-on-start) at /out.
&& pnpm --filter @dorfteich/api deploy --prod --legacy /out \
&& cp -r apps/api/dist /out/dist \
&& cp -r apps/api/assets /out/assets \
&& cp -r /repo/fonts /out/fonts
FROM node:22.15-alpine
FROM node:22.15.1-alpine
ARG APP_VERSION=0.0.0-dev
# Default the data dirs to the writable, node-owned locations created below, so
# the image works out of the box even where compose does not set them; compose
# still mounts named volumes here for persistence (UPLOADS_DIR/PLUGINS_DIR).
ENV NODE_ENV=production APP_VERSION=${APP_VERSION} UPLOADS_DIR=/data/uploads PLUGINS_DIR=/data/plugins SECRETS_FILE=/data/secrets/secrets.env BACKUPS_DIR=/data/backups
ENV NODE_ENV=production APP_VERSION=${APP_VERSION} UPLOADS_DIR=/data/uploads PLUGINS_DIR=/data/plugins CUSTOM_FONTS_DIR=/data/fonts BRANDING_DIR=/data/branding SECRETS_FILE=/data/secrets/secrets.env BACKUPS_DIR=/data/backups
WORKDIR /app
COPY --from=build --chown=node:node /out /app
# Generate the Prisma client for this image's platform.
RUN node node_modules/prisma/build/index.js generate
# A fresh named volume mounted at /data/uploads or /data/plugins is created
# A fresh named volume mounted at /data/uploads, /data/plugins, /data/fonts
# or /data/branding is created
# root-owned; pre-creating them here (Docker copies an image directory's
# ownership into a new volume on first mount) lets the non-root `node` user
# write to them. /data/backups is mounted read-only here, but pre-creating it
# node-owned keeps the shared `backups` volume writable for the backup
# sidecar even when the api container is the one that initializes it.
RUN mkdir -p /data/uploads /data/plugins /data/secrets /data/backups && chown -R node:node /data/uploads /data/plugins /data/secrets /data/backups
RUN mkdir -p /data/uploads /data/plugins /data/fonts /data/branding /data/secrets /data/backups && chown -R node:node /data/uploads /data/plugins /data/fonts /data/branding /data/secrets /data/backups
USER node
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --retries=3 \

45
apps/api/assets/README.md Normal file
View File

@ -0,0 +1,45 @@
# Runtime assets
## `reference-vs-nfd.docx` / `reference-vs-nfd.odt` (issue #209, ADR 0022)
Pandoc reference documents for the DOCX/ODT export of a **classified**
page: their page setup defines a header and footer carrying the VS-NfD
marking, which pandoc copies into its output — so the marking repeats on
every page in Word and LibreOffice and is not deletable body text.
Unclassified exports pass no reference document and are unchanged.
These are **derived binaries — never edit them by hand.** Source of truth
is `../scripts/gen-classified-reference-docs.mjs`: it takes the default
reference documents of the pinned sidecar (`pandoc/core:3.6`, the exact
image the stages run) and injects the header/footer, with the wording from
`classificationMarking()` in `@dorfteich/shared` (single source, ADR
0022). Regenerate — after a pandoc pin bump, a wording change, or a layout
tweak in the script — with Docker running:
```sh
pnpm --filter @dorfteich/shared build # the script imports the wording
node apps/api/scripts/gen-classified-reference-docs.mjs
```
Commit script and binaries together. The fidelity suite
(`export.fidelity.test.ts`) asserts against the real pinned pandoc that a
marked export carries the header/footer parts and an unmarked one does
not.
### Per-page verification in the office suites
After regenerating, confirm the marking repeats on **every** page of a
multi-page export (not just structurally in the XML):
1. Produce a marked multi-page export (any classified page with a few
screens of text, exported to `.docx` and `.odt`).
2. **LibreOffice** (scriptable):
`soffice --headless --convert-to pdf <file>` and check every PDF page
shows the marking twice (header + footer) — e.g. with `pypdf`.
3. **Word**: open the `.docx`, check header and footer on every page
(print preview). Word's AppleScript/sandbox makes this hard to script —
this step is a quick manual look.
Last verified 2026-07-31 (pandoc 3.6 output): LibreOffice 25.8, both
formats, 5/5 pages with 2 markings each. Word: manual check pending —
sample files in the workspace under `doku/209-marked-sample.docx/.odt`.

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 683 B

Binary file not shown.

Binary file not shown.

View File

@ -30,6 +30,7 @@
"fflate": "^0.8.3",
"fractional-indexing": "^4.0.0",
"i18next": "^26.3.4",
"jose": "^6.2.4",
"jsdom": "^26.1.0",
"multer": "^2.1.1",
"nestjs-pino": "^4.3.0",

View File

@ -0,0 +1,4 @@
-- Issue #194: attachments have exactly one deletion semantics (hard delete
-- by sweep, purge, or manual removal) — the never-written soft-delete
-- marker goes away.
ALTER TABLE "attachments" DROP COLUMN "deleted_at";

View File

@ -0,0 +1,10 @@
-- Issue #195, one-off backfill: the full-text index must hold no trashed
-- content. Clears the search vector of every page that is trashed itself
-- or lives in a trashed pond; the application keeps this invariant from
-- now on (trash hooks + reindex paths).
UPDATE page_content_cache c
SET search_vector = NULL
FROM pages p
LEFT JOIN ponds po ON po.id = p.pond_id
WHERE p.id = c.page_id
AND (p.deleted_at IS NOT NULL OR po.deleted_at IS NOT NULL);

View File

@ -0,0 +1,16 @@
-- #233: conversion job payloads become prunable. The raw input/result bytes
-- are transient; a daily job nulls them once a finished job passes
-- `conversion.payloadRetentionDays` (default 30). The row survives for
-- status/audit purposes.
ALTER TABLE "conversion_jobs" ALTER COLUMN "input" DROP NOT NULL;
-- Backfill: clear the payloads of jobs that already finished longer ago than
-- the default period. Recently finished jobs keep their bytes so a pending
-- download still works; the scheduled job picks them up when they age out.
-- PENDING/RUNNING rows are untouched (the worker's stale-lock recovery may
-- still re-run them).
UPDATE "conversion_jobs"
SET "input" = NULL, "result" = NULL, "result_mime_type" = NULL
WHERE "status" IN ('SUCCEEDED', 'FAILED')
AND "updated_at" < now() - interval '30 days'
AND ("input" IS NOT NULL OR "result" IS NOT NULL);

View File

@ -0,0 +1,5 @@
-- #199: integrity hash for uploaded files. New uploads store the SHA-256 of
-- their bytes at write time; existing rows are hashed by the nightly
-- backfill (part of the orphan-file-sweep job), which reads the uploads
-- volume — something this SQL migration cannot do.
ALTER TABLE "attachments" ADD COLUMN "sha256" TEXT;

View File

@ -0,0 +1,8 @@
-- #204 (ADR 0022): classification becomes first-class page metadata. The
-- column is a marking, not a protection mechanism — permissions are
-- untouched. NOT NULL with a default backfills every existing page to
-- UNCLASSIFIED in the same statement.
CREATE TYPE "PageClassification" AS ENUM ('UNCLASSIFIED', 'VS_NFD');
ALTER TABLE "pages"
ADD COLUMN "classification" "PageClassification" NOT NULL DEFAULT 'UNCLASSIFIED';

View File

@ -0,0 +1,23 @@
-- #222 (ADR 0023): read-access trail for classified pages. Its own table —
-- volume, purpose and legal basis differ from audit_log. No foreign keys:
-- evidence must survive page purges and hard user deletions unchanged.
-- Partitioning and retention follow in #224.
CREATE TABLE "read_events" (
"id" TEXT NOT NULL,
"occurred_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"actor_id" TEXT,
"session_key" TEXT NOT NULL,
"page_id" TEXT,
"pond_id" TEXT NOT NULL,
"channel" TEXT NOT NULL,
"classification" TEXT NOT NULL,
"details" JSONB,
CONSTRAINT "read_events_pkey" PRIMARY KEY ("id")
);
CREATE INDEX "read_events_page_id_occurred_at_idx" ON "read_events"("page_id", "occurred_at");
CREATE INDEX "read_events_actor_id_occurred_at_idx" ON "read_events"("actor_id", "occurred_at");
CREATE INDEX "read_events_occurred_at_idx" ON "read_events"("occurred_at");

View File

@ -0,0 +1,32 @@
-- #223 (ADR 0023): dedup window for the read trail. Aligned buckets
-- (floor(epoch / window)) with a unique (dedup_key, window_bucket) pair make
-- concurrent duplicates collapse race-free at insert time.
ALTER TABLE "read_events"
ADD COLUMN "dedup_key" TEXT,
ADD COLUMN "window_bucket" BIGINT,
ADD COLUMN "window_seconds" INTEGER;
-- Backfill rows written between the #222 and #223 deploys under the default
-- 5-minute window, then apply the window's own semantics retroactively:
-- within one (key, bucket) pair only the FIRST event is the evidence row —
-- exactly what the window would have recorded had it existed.
UPDATE "read_events"
SET "dedup_key" = "session_key" || ':' || COALESCE("page_id", '-') || ':' || "channel",
"window_bucket" = FLOOR(EXTRACT(EPOCH FROM "occurred_at") / 300)::BIGINT,
"window_seconds" = 300
WHERE "dedup_key" IS NULL;
DELETE FROM "read_events" keep
USING "read_events" first
WHERE keep."dedup_key" = first."dedup_key"
AND keep."window_bucket" = first."window_bucket"
AND (first."occurred_at" < keep."occurred_at"
OR (first."occurred_at" = keep."occurred_at" AND first."id" < keep."id"));
ALTER TABLE "read_events"
ALTER COLUMN "dedup_key" SET NOT NULL,
ALTER COLUMN "window_bucket" SET NOT NULL,
ALTER COLUMN "window_seconds" SET NOT NULL;
CREATE UNIQUE INDEX "read_events_dedup_key_window_bucket_key"
ON "read_events"("dedup_key", "window_bucket");

View File

@ -0,0 +1,76 @@
-- #224 (ADR 0023): convert read_events to monthly RANGE partitions on
-- occurred_at. Volume grows unbounded with use; retention then DROPs whole
-- expired partitions instead of scanning deletes. The primary key gains the
-- partition column (PostgreSQL requirement); the dedup unique pair
-- (dedup_key, window_bucket) moves to PER-PARTITION unique indexes — a
-- partitioned parent cannot carry it without the partition key. A bucket
-- spanning a month boundary can therefore record one duplicate; documented
-- in ADR 0023, over-recording is acceptable, gaps are not.
--
-- A DEFAULT partition catches rows outside every maintained range, so a
-- lagging maintenance job can never make classified reads fail (the trail's
-- hard-failure semantics would otherwise turn an ops miss into an outage).
ALTER TABLE "read_events" RENAME TO "read_events_old";
ALTER INDEX "read_events_pkey" RENAME TO "read_events_old_pkey";
ALTER INDEX "read_events_dedup_key_window_bucket_key" RENAME TO "read_events_old_dedup_key";
ALTER INDEX "read_events_page_id_occurred_at_idx" RENAME TO "read_events_old_page_idx";
ALTER INDEX "read_events_actor_id_occurred_at_idx" RENAME TO "read_events_old_actor_idx";
ALTER INDEX "read_events_occurred_at_idx" RENAME TO "read_events_old_at_idx";
CREATE TABLE "read_events" (
"id" TEXT NOT NULL,
"occurred_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"actor_id" TEXT,
"session_key" TEXT NOT NULL,
"page_id" TEXT,
"pond_id" TEXT NOT NULL,
"channel" TEXT NOT NULL,
"classification" TEXT NOT NULL,
"details" JSONB,
"dedup_key" TEXT NOT NULL,
"window_bucket" BIGINT NOT NULL,
"window_seconds" INTEGER NOT NULL,
CONSTRAINT "read_events_pkey" PRIMARY KEY ("id", "occurred_at")
) PARTITION BY RANGE ("occurred_at");
-- Non-unique parent indexes propagate to every partition automatically.
CREATE INDEX "read_events_page_id_occurred_at_idx" ON "read_events"("page_id", "occurred_at");
CREATE INDEX "read_events_actor_id_occurred_at_idx" ON "read_events"("actor_id", "occurred_at");
CREATE INDEX "read_events_occurred_at_idx" ON "read_events"("occurred_at");
-- The safety-net partition, plus the current and the next month — the daily
-- maintenance job (read-trail-maintenance) keeps creating months ahead and
-- adds the same per-partition dedup index to each new one.
CREATE TABLE "read_events_default" PARTITION OF "read_events" DEFAULT;
CREATE UNIQUE INDEX "read_events_default_dedup_key"
ON "read_events_default"("dedup_key", "window_bucket");
DO $$
DECLARE
m DATE;
part TEXT;
BEGIN
FOR i IN 0..1 LOOP
m := date_trunc('month', now())::date + (i || ' month')::interval;
part := 'read_events_y' || to_char(m, 'YYYY') || 'm' || to_char(m, 'MM');
EXECUTE format(
'CREATE TABLE %I PARTITION OF "read_events" FOR VALUES FROM (%L) TO (%L)',
part, m, m + interval '1 month');
EXECUTE format(
'CREATE UNIQUE INDEX %I ON %I ("dedup_key", "window_bucket")',
part || '_dedup_key', part);
END LOOP;
END $$;
INSERT INTO "read_events"
("id", "occurred_at", "actor_id", "session_key", "page_id", "pond_id",
"channel", "classification", "details", "dedup_key", "window_bucket",
"window_seconds")
SELECT "id", "occurred_at", "actor_id", "session_key", "page_id", "pond_id",
"channel", "classification", "details", "dedup_key", "window_bucket",
"window_seconds"
FROM "read_events_old";
DROP TABLE "read_events_old";

View File

@ -0,0 +1,9 @@
-- #217 (ADR 0021): IdP claim mapping. Grants gain an origin so mapped rows
-- are distinguishable from manual ones (the mapping only ever touches its
-- own); the site-admin flag gains a "managed" marker so only a
-- mapping-granted flag can be mapping-revoked.
ALTER TABLE "role_grants"
ADD COLUMN "origin" TEXT NOT NULL DEFAULT 'manual';
ALTER TABLE "users"
ADD COLUMN "is_site_admin_managed" BOOLEAN NOT NULL DEFAULT false;

View File

@ -0,0 +1,4 @@
-- #232: SHA-256 of the installed bundle ZIP, observed at install time.
-- NULL for plugins installed before this migration — the admin UI says so
-- and a reinstall records it.
ALTER TABLE "plugins" ADD COLUMN "bundle_hash" TEXT;

View File

@ -0,0 +1,45 @@
-- #303: operator-uploaded font families (ADR 0016 §#303).
-- The bytes live on disk under CUSTOM_FONTS_DIR; these rows record only what
-- the upload form stated, because the api never parses the font file.
CREATE TABLE "custom_fonts" (
"id" TEXT NOT NULL,
"family" TEXT NOT NULL,
"slug" TEXT NOT NULL,
"category" TEXT NOT NULL,
"licence" TEXT NOT NULL,
"licence_url" TEXT,
"uploaded_by" TEXT NOT NULL,
"created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
"updated_at" TIMESTAMP(3) NOT NULL,
CONSTRAINT "custom_fonts_pkey" PRIMARY KEY ("id")
);
-- Both unique: `family` keeps `fonts.<slot>.family` in pond settings
-- unambiguous, `slug` owns a directory under CUSTOM_FONTS_DIR.
CREATE UNIQUE INDEX "custom_fonts_family_key" ON "custom_fonts"("family");
CREATE UNIQUE INDEX "custom_fonts_slug_key" ON "custom_fonts"("slug");
ALTER TABLE "custom_fonts" ADD CONSTRAINT "custom_fonts_uploaded_by_fkey"
FOREIGN KEY ("uploaded_by") REFERENCES "users"("id")
ON DELETE RESTRICT ON UPDATE CASCADE;
CREATE TABLE "custom_font_weights" (
"id" TEXT NOT NULL,
"font_id" TEXT NOT NULL,
"weight" INTEGER NOT NULL,
"has_woff" BOOLEAN NOT NULL DEFAULT false,
"byte_size" INTEGER NOT NULL,
CONSTRAINT "custom_font_weights_pkey" PRIMARY KEY ("id")
);
CREATE UNIQUE INDEX "custom_font_weights_font_id_weight_key"
ON "custom_font_weights"("font_id", "weight");
-- Deleting a family takes its weights with it; the files on disk are removed
-- by the service in the same operation.
ALTER TABLE "custom_font_weights" ADD CONSTRAINT "custom_font_weights_font_id_fkey"
FOREIGN KEY ("font_id") REFERENCES "custom_fonts"("id")
ON DELETE CASCADE ON UPDATE CASCADE;

View File

@ -0,0 +1,26 @@
-- Peer invitations (issue #332): a user invites an e-mail address; the token
-- allows exactly one registration even while registration is closed.
-- CreateTable
CREATE TABLE "invitations" (
"id" TEXT NOT NULL,
"inviter_id" TEXT NOT NULL,
"email" TEXT NOT NULL,
"token_hash" TEXT NOT NULL,
"expires_at" TIMESTAMP(3) NOT NULL,
"revoked_at" TIMESTAMP(3),
"accepted_at" TIMESTAMP(3),
"accepted_user_id" TEXT,
"created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "invitations_pkey" PRIMARY KEY ("id")
);
-- CreateIndex
CREATE UNIQUE INDEX "invitations_token_hash_key" ON "invitations"("token_hash");
-- CreateIndex
CREATE INDEX "invitations_inviter_id_idx" ON "invitations"("inviter_id");
-- AddForeignKey
ALTER TABLE "invitations" ADD CONSTRAINT "invitations_inviter_id_fkey" FOREIGN KEY ("inviter_id") REFERENCES "users"("id") ON DELETE CASCADE ON UPDATE CASCADE;

View File

@ -31,41 +31,71 @@ enum UserStatus {
/// Account profile. Login methods live in UserIdentity (OIDC-ready,
/// ADR 0007); Site Admin is a user flag, all other roles are grants.
model User {
id String @id @default(uuid())
username String @unique
email String @unique
displayName String @map("display_name")
locale String @default("en")
isSiteAdmin Boolean @default(false) @map("is_site_admin")
id String @id @default(uuid())
username String @unique
email String @unique
displayName String @map("display_name")
locale String @default("en")
isSiteAdmin Boolean @default(false) @map("is_site_admin")
/// True when the flag was last SET by the IdP claim mapping (issue #217):
/// only then may the mapping revoke it again on a later login. A manual
/// admin toggle clears the marker, so hand-granted admins are never
/// demoted by a missing claim.
isSiteAdminManaged Boolean @default(false) @map("is_site_admin_managed")
/// Auto-watch preferences (issue #93): watch pages I create / comment on.
autoWatchOwnPages Boolean @default(true) @map("auto_watch_own_pages")
autoWatchOnComment Boolean @default(true) @map("auto_watch_on_comment")
autoWatchOwnPages Boolean @default(true) @map("auto_watch_own_pages")
autoWatchOnComment Boolean @default(true) @map("auto_watch_on_comment")
/// E-mail digest cadence (issue #95): hourly | daily | off.
digestFrequency String @default("hourly") @map("digest_frequency")
status UserStatus @default(PENDING_VERIFICATION)
emailVerifiedAt DateTime? @map("email_verified_at")
createdAt DateTime @default(now()) @map("created_at")
lastLoginAt DateTime? @map("last_login_at")
digestFrequency String @default("hourly") @map("digest_frequency")
status UserStatus @default(PENDING_VERIFICATION)
emailVerifiedAt DateTime? @map("email_verified_at")
createdAt DateTime @default(now()) @map("created_at")
lastLoginAt DateTime? @map("last_login_at")
identities UserIdentity[]
sessions Session[]
authTokens AuthToken[]
apiTokens ApiToken[]
feedTokens FeedToken[]
mentionRows PageMention[]
ponds Pond[]
pages Page[]
attachments Attachment[]
conversionJobs ConversionJob[]
auditEntries AuditEntry[]
comments Comment[]
watches Watch[]
notifications Notification[]
favorites PageFavorite[]
identities UserIdentity[]
sessions Session[]
authTokens AuthToken[]
apiTokens ApiToken[]
feedTokens FeedToken[]
mentionRows PageMention[]
ponds Pond[]
pages Page[]
attachments Attachment[]
conversionJobs ConversionJob[]
auditEntries AuditEntry[]
comments Comment[]
watches Watch[]
notifications Notification[]
favorites PageFavorite[]
customFonts CustomFont[]
invitations Invitation[] @relation("InvitationsSent")
@@map("users")
}
/// Peer invitations (issue #332): a user invites an e-mail address; the
/// token allows exactly one registration even while registration is
/// closed. Only the SHA-256 hash of the token is stored (auth-tokens
/// pattern); revoked/accepted rows are kept so the settings UI can show
/// history. "Open" (pending, unexpired) rows count against the per-user
/// quota `invitations.maxOpenPerUser`.
model Invitation {
id String @id @default(uuid())
inviterId String @map("inviter_id")
email String
tokenHash String @unique @map("token_hash")
expiresAt DateTime @map("expires_at")
revokedAt DateTime? @map("revoked_at")
acceptedAt DateTime? @map("accepted_at")
acceptedUserId String? @map("accepted_user_id")
createdAt DateTime @default(now()) @map("created_at")
inviter User @relation("InvitationsSent", fields: [inviterId], references: [id], onDelete: Cascade)
@@index([inviterId])
@@map("invitations")
}
/// Persistent audit trail (issue #86, security.md §Logging): auth events and
/// admin actions — grants, member roles, plugin installs, quota and settings
/// changes, setup steps, manual job triggers. Written by AuditService, which
@ -91,6 +121,52 @@ model AuditEntry {
@@map("audit_log")
}
/// Read-access trail for classified pages (issue #222, ADR 0023): one row per
/// read of a `VS_NFD` page, per channel. Separate from `audit_log` because
/// volume, purpose and legal basis all differ. Deliberately WITHOUT foreign
/// keys: evidence must survive a page purge and a hard user deletion — the
/// ids stay as recorded (pseudonymous uuids), history is never rewritten.
///
/// In migrated databases the table is RANGE-partitioned by `occurred_at`
/// (monthly, issue #224) — hence the composite id. The dedup unique pair
/// lives per partition there (a partitioned parent cannot carry it without
/// the partition key); `db push` test databases get it on the plain table.
model ReadEvent {
id String @default(uuid())
occurredAt DateTime @default(now()) @map("occurred_at")
/// Null = anonymous reader (public grant); `sessionKey` still names the
/// browsing session, so the anonymous marker is explicit, not an accident.
actorId String? @map("actor_id")
/// `session:<id>` for cookie sessions, `token:<id>` for PATs, `job:<id>`
/// for background builds (account data export), `anon` for anonymous
/// visitors — the dedup-window key basis (#223).
sessionKey String @map("session_key")
pageId String? @map("page_id")
pondId String @map("pond_id")
/// Which read surface fired: `page_view` | `no_js_shell` | `public_api` |
/// `attachment` | `export` | `collab_join` (READ_CHANNELS union in code).
channel String
/// Classification at read time — a later reclassification must not
/// rewrite history (ADR 0023).
classification String
details Json?
/// Dedup window (issue #223): `<sessionKey>:<pageId|->:<channel>` plus the
/// aligned bucket `floor(epoch / windowSeconds)`. The unique pair makes
/// concurrent duplicate reads collapse race-free (insert or P2002-skip).
dedupKey String @map("dedup_key")
windowBucket BigInt @map("window_bucket")
/// Window length the event was recorded under — the row itself states it
/// represents up to this many seconds, so the evidence is not overread.
windowSeconds Int @map("window_seconds")
@@id([id, occurredAt])
@@unique([dedupKey, windowBucket])
@@index([pageId, occurredAt])
@@index([actorId, occurredAt])
@@index([occurredAt])
@@map("read_events")
}
/// Threaded page comments (issue #91, data-model.md §Comments). Threads are
/// one level deep: roots carry the optional document anchor and the resolve
/// state, replies reference the root via `parentId`. Purging a page cascades
@ -186,14 +262,14 @@ model Pond {
deletedAt DateTime? @map("deleted_at")
deletedBy String? @map("deleted_by")
owner User @relation(fields: [ownerId], references: [id])
usage PondUsage?
pages Page[]
attachments Attachment[]
labels Label[]
grants RoleGrant[]
owner User @relation(fields: [ownerId], references: [id])
usage PondUsage?
pages Page[]
attachments Attachment[]
labels Label[]
grants RoleGrant[]
conversionJobs ConversionJob[]
pondPlugins PondPlugin[]
pondPlugins PondPlugin[]
@@index([ownerId])
@@map("ponds")
@ -239,6 +315,10 @@ model RoleGrant {
scopeType GrantScopeType @map("scope_type")
scopeId String? @map("scope_id")
effect GrantEffect
/// `manual` (admin-created) or `idp` (written by the claim mapping,
/// issue #217). The mapping only ever creates and revokes ITS OWN rows —
/// manual grants are never touched, which is the documented precedence.
origin String @default("manual")
createdBy String @map("created_by")
createdAt DateTime @default(now()) @map("created_at")
@ -260,19 +340,30 @@ model RoleGrant {
/// page never changes its URL or breaks wikilinks. Trashed pages keep their
/// `parentId` (restore re-attaches to the nearest live ancestor, issue #107);
/// `SetNull` is only the FK backstop — purge promotes children explicitly.
/// VS-NfD marking level of a page (ADR 0022). Deliberately an enum on Page,
/// not a label: instance-wide meaning, not user-deletable in routine content
/// work, inherits down the tree (#205), reaches every output channel
/// (#206#212). It is a MARKING, not a protection mechanism — separation of
/// levels happens outside the application (one instance per level).
enum PageClassification {
UNCLASSIFIED
VS_NFD
}
model Page {
id String @id @default(uuid())
pondId String @map("pond_id")
parentId String? @map("parent_id")
title String
slug String
ydocState Bytes @map("ydoc_state")
sortKey String @map("sort_key")
createdBy String @map("created_by")
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
deletedAt DateTime? @map("deleted_at")
deletedBy String? @map("deleted_by")
id String @id @default(uuid())
pondId String @map("pond_id")
parentId String? @map("parent_id")
title String
slug String
ydocState Bytes @map("ydoc_state")
sortKey String @map("sort_key")
classification PageClassification @default(UNCLASSIFIED)
createdBy String @map("created_by")
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
deletedAt DateTime? @map("deleted_at")
deletedBy String? @map("deleted_by")
pond Pond @relation(fields: [pondId], references: [id])
parent Page? @relation("PageHierarchy", fields: [parentId], references: [id], onDelete: SetNull)
@ -394,12 +485,12 @@ model CollabOpenSession {
/// built from the Yjs state via the shared editor schema. `outline` is the
/// heading tree (`OutlineEntry[]` from @dorfteich/shared) as jsonb.
model PageContentCache {
pageId String @id @map("page_id")
plainText String @map("plain_text")
markdown String
html String
outline Json
updatedAt DateTime @updatedAt @map("updated_at")
pageId String @id @map("page_id")
plainText String @map("plain_text")
markdown String
html String
outline Json
updatedAt DateTime @updatedAt @map("updated_at")
/// Weighted full-text search vector (title A, labels B, body C; issue #49,
/// ADR 0010). Maintained by the SearchProvider and the collab persistence
/// hook (both write it with the same weighting). The GIN index is added in
@ -529,26 +620,33 @@ model PondUsage {
/// `<uploadsDir>/<pondId>/<id>` (FileStorageService); this row carries the
/// metadata needed to serve and account for it. `pageId` starts unset —
/// images are uploaded before the page referencing them is known
/// (paste-then-insert, issue #28) — and is set on every page state save to
/// whichever page's document currently embeds the file (issue #31,
/// `PagesService.saveState`); the trash-purge job uses that link to delete
/// a purged page's files. Not touched when an image is later removed from
/// its page's content — an orphan-file sweep to reclaim those is a
/// separate future maintenance job (operations.md), not this one.
/// `deletedAt` stays unused for now — purge hard-deletes attachments
/// directly rather than soft-deleting them first — reserved for that same
/// future orphan-sweep job.
/// (paste-then-insert, issue #28) — and is claimed on every collab persist
/// by whichever page's document embeds the file (issue #31), or at upload
/// for the page attachments panel (#61); the trash-purge job uses that
/// link to delete a purged page's files. A row whose `pageId` is STILL
/// null after a grace period was claimed by nothing and is reclaimed by
/// the nightly orphan-file sweep (issue #194, OrphanSweepService).
/// Claimed files are deliberately NOT auto-reclaimed when the content
/// stops referencing them: the page attachments panel lists them as
/// user-managed objects (insert is optional there), so "not embedded" is
/// not "unused" — the pond file manager is the human cleanup path.
/// Deletion is hard everywhere (sweep, purge, manual) — there is no
/// soft-delete state on attachments (issue #194 removed `deletedAt`).
model Attachment {
id String @id @default(uuid())
pondId String @map("pond_id")
pageId String? @map("page_id")
fileName String @map("file_name")
mimeType String @map("mime_type")
sizeBytes Int @map("size_bytes")
storagePath String @map("storage_path")
uploadedBy String @map("uploaded_by")
createdAt DateTime @default(now()) @map("created_at")
deletedAt DateTime? @map("deleted_at")
id String @id @default(uuid())
pondId String @map("pond_id")
pageId String? @map("page_id")
fileName String @map("file_name")
mimeType String @map("mime_type")
sizeBytes Int @map("size_bytes")
storagePath String @map("storage_path")
uploadedBy String @map("uploaded_by")
/// SHA-256 (hex) of the stored bytes (issue #199), computed from the
/// in-memory upload buffer as it is written — never by re-reading disk.
/// Downloads verify against it and fail closed on mismatch. Null only
/// for rows that predate #199 until the nightly backfill hashes them.
sha256 String?
createdAt DateTime @default(now()) @map("created_at")
pond Pond @relation(fields: [pondId], references: [id])
page Page? @relation(fields: [pageId], references: [id])
@ -740,9 +838,11 @@ enum ConversionJobStatus {
/// enqueued PENDING, a worker claims it (`FOR UPDATE SKIP LOCKED`, `lockedAt`
/// recovers a crashed run), calls the pandoc sidecar with a timeout, and
/// stores the output bytes or an `errorCode`. `input`/`result` are the raw
/// document bytes — kept small by the request size limit and pruned by a
/// later maintenance job (they are transient, not the durable copy an
/// Attachment is). The polling endpoint `GET /jobs/:id` is owner-scoped.
/// document bytes — kept small by the request size limit and transient, not
/// the durable copy an Attachment is: the daily `conversion-payload-prune`
/// job (#233) nulls both once a finished job passes
/// `conversion.payloadRetentionDays`; the row survives for status/audit.
/// The polling endpoint `GET /jobs/:id` is owner-scoped.
model ConversionJob {
id String @id @default(uuid())
ownerId String @map("owner_id")
@ -752,7 +852,9 @@ model ConversionJob {
sourceFormat String @map("source_format")
targetFormat String @map("target_format")
standalone Boolean @default(true)
input Bytes
/// Null once the retention job (#233) pruned a finished job's payload —
/// never while the job is PENDING/RUNNING (incl. stale-lock recovery).
input Bytes?
status ConversionJobStatus @default(PENDING)
attempts Int @default(0)
result Bytes?
@ -761,10 +863,11 @@ model ConversionJob {
lockedAt DateTime? @map("locked_at")
/// For a data-export job (#68): when its stored result stops being
/// downloadable and is purged (GDPR data minimization). Null for every
/// other job kind, whose result never expires.
/// other job kind, whose payload the general retention (#233) prunes.
expiresAt DateTime? @map("expires_at")
/// Kind-specific job options (issue #117): a vault import carries
/// `{parentPageId, labelIds, frontmatterMode}`. Null for other kinds.
/// `{parentPageId, labelIds, frontmatterMode}`; a PDF/DOCX/ODT export of a
/// classified page carries `{marking}` (issues #208/#209). Null otherwise.
options Json?
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
@ -776,9 +879,9 @@ model ConversionJob {
sourceName String? @map("source_name")
resultPageId String? @map("result_page_id")
owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade)
pond Pond? @relation(fields: [pondId], references: [id], onDelete: Cascade)
page Page? @relation(fields: [resultPageId], references: [id], onDelete: SetNull)
owner User @relation(fields: [ownerId], references: [id], onDelete: Cascade)
pond Pond? @relation(fields: [pondId], references: [id], onDelete: Cascade)
page Page? @relation(fields: [resultPageId], references: [id], onDelete: SetNull)
@@index([status, createdAt])
@@map("conversion_jobs")
@ -807,6 +910,8 @@ model Plugin {
mode PluginInstanceMode @default(DISABLED)
/// The full manifest as validated at install time (@dorfteich/plugin-sdk).
manifest Json
/// SHA-256 (hex) of the installed bundle ZIP (#232); null = pre-#232 install.
bundleHash String? @map("bundle_hash")
installedAt DateTime @default(now()) @map("installed_at")
updatedAt DateTime @updatedAt @map("updated_at")
/// Set when uninstalled; active queries filter `removedAt: null`.
@ -833,3 +938,48 @@ model PondPlugin {
@@id([pondId, pluginId])
@@map("pond_plugins")
}
/// An operator-uploaded font family (issue #303, ADR 0016 §#303). The bytes
/// live on disk under CUSTOM_FONTS_DIR — this row only records what the
/// upload form stated, because the api never parses the font file itself.
/// Additive to the compile-time catalog: a family whose name or slug
/// collides with a catalog entry is rejected, so `fonts.<slot>.family` in a
/// pond's settings stays unambiguous.
model CustomFont {
id String @id @default(uuid())
/// CSS `font-family` name, as typed by the uploader.
family String @unique
/// URL/file-safe form; names the directory under CUSTOM_FONTS_DIR.
slug String @unique
/// Drives the system fallback stack, like FontCatalogEntry.category.
category String
/// Free-text licence label, e.g. "Commercial — Foundry XY". Required so
/// an attribution obligation can be met on the font catalogue page.
licence String
licenceUrl String? @map("licence_url")
uploadedBy String @map("uploaded_by")
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")
uploader User @relation(fields: [uploadedBy], references: [id])
weights CustomFontWeight[]
@@map("custom_fonts")
}
/// One weight of a custom family. Style is always `normal`: the PDF
/// `@font-face` builder emits only that, and browsers synthesise oblique —
/// italic uploads are a follow-up, not a silent half-feature.
model CustomFontWeight {
id String @id @default(uuid())
fontId String @map("font_id")
weight Int
/// Whether a legacy WOFF was supplied next to the required WOFF2.
hasWoff Boolean @default(false) @map("has_woff")
byteSize Int @map("byte_size")
font CustomFont @relation(fields: [fontId], references: [id], onDelete: Cascade)
@@unique([fontId, weight])
@@map("custom_font_weights")
}

View File

@ -311,6 +311,36 @@ async function seedContentFixtures(ownerId: string): Promise<void> {
deriveContentOf(everyElementDoc),
);
// "Classified Note" (issue #206, ADR 0022): a VS-NfD-marked page so e2e
// (a11y pack) can assert the marking banner in both themes. Kept simple —
// the marking, not the content, is what the fixture exists for.
const classifiedDoc = editorSchema.node('doc', null, [
editorSchema.node('heading', { level: 1 }, [editorSchema.text('Classified Note')]),
editorSchema.node('paragraph', null, [
editorSchema.text('This fixture page carries the VS-NfD marking.'),
]),
]);
const classifiedYdoc = new Y.Doc();
prosemirrorJSONToYXmlFragment(
editorSchema,
classifiedDoc.toJSON(),
classifiedYdoc.getXmlFragment('default'),
);
const classifiedState = new Uint8Array(Y.encodeStateAsUpdate(classifiedYdoc));
classifiedYdoc.destroy();
const classifiedPageId = await upsertFixturePage(
pond.id,
'classified-note',
'Classified Note',
ownerId,
classifiedState,
deriveContentOf(classifiedDoc),
);
await prisma.page.update({
where: { id: classifiedPageId },
data: { classification: 'VS_NFD' },
});
// "Fixture Image": one real, servable uploaded image (the Markdown
// fixture above only carries a placeholder fileId for round-trip
// testing — this is the one that actually resolves via /media/:fileId).

View File

@ -0,0 +1,118 @@
/**
* Regenerate the classified reference documents (issue #209, ADR 0022):
* `apps/api/assets/reference-vs-nfd.docx` / `.odt`.
*
* The DOCX/ODT export of a classified page passes these to pandoc via
* `--reference-doc`; pandoc copies the reference's page setup including
* headers and footers into its output, which is how the VS-NfD marking
* repeats on every page in Word and LibreOffice without being deletable
* body text.
*
* The binaries are DERIVED files: base = the default reference documents of
* the PINNED pandoc (`pandoc/core:3.6`, the exact sidecar the stages run),
* plus a header and footer carrying the marking. Never edit the binaries by
* hand edit this script and re-run it (Docker required):
*
* node apps/api/scripts/gen-classified-reference-docs.mjs
*
* The marking wording comes from @dorfteich/shared (single source, ADR
* 0022); the shared package must be built (`pnpm --filter @dorfteich/shared
* build`).
*/
import { execFileSync } from 'node:child_process';
import { mkdirSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { classificationMarking } from '@dorfteich/shared';
import { strToU8, strFromU8, unzipSync, zipSync } from 'fflate';
const PANDOC_IMAGE = 'pandoc/core:3.6';
const MARKING = classificationMarking('vs_nfd');
const outDir = join(dirname(fileURLToPath(import.meta.url)), '../assets');
function defaultReference(name) {
return execFileSync('docker', ['run', '--rm', PANDOC_IMAGE, '--print-default-data-file', name], {
maxBuffer: 64 * 1024 * 1024,
});
}
function escapeXml(value) {
return value.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
}
/** DOCX: add word/header1.xml + word/footer1.xml, register them in the
* content types and document relationships, and reference them from the
* document's sectPr Word repeats them on every page. */
function patchDocx(bytes) {
const zip = unzipSync(new Uint8Array(bytes));
const marking = escapeXml(MARKING);
const partXml = (root) =>
`<?xml version="1.0" encoding="UTF-8" standalone="yes"?>\n` +
`<w:${root} xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main">` +
`<w:p><w:pPr><w:jc w:val="center"/></w:pPr>` +
`<w:r><w:rPr><w:b/></w:rPr><w:t xml:space="preserve">${marking}</w:t></w:r>` +
`</w:p></w:${root}>`;
zip['word/header1.xml'] = strToU8(partXml('hdr'));
zip['word/footer1.xml'] = strToU8(partXml('ftr'));
const types = strFromU8(zip['[Content_Types].xml']);
zip['[Content_Types].xml'] = strToU8(
types.replace(
'</Types>',
'<Override PartName="/word/header1.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.header+xml" />' +
'<Override PartName="/word/footer1.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.footer+xml" />' +
'</Types>',
),
);
const rels = strFromU8(zip['word/_rels/document.xml.rels']);
zip['word/_rels/document.xml.rels'] = strToU8(
rels.replace(
'</Relationships>',
'<Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/header" Id="rIdVsNfdHeader" Target="header1.xml" />' +
'<Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/footer" Id="rIdVsNfdFooter" Target="footer1.xml" />' +
'</Relationships>',
),
);
const doc = strFromU8(zip['word/document.xml']);
if (!doc.includes('<w:sectPr>')) throw new Error('reference.docx has no sectPr');
zip['word/document.xml'] = strToU8(
doc.replace(
'<w:sectPr>',
'<w:sectPr>' +
'<w:headerReference xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" w:type="default" r:id="rIdVsNfdHeader" />' +
'<w:footerReference xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" w:type="default" r:id="rIdVsNfdFooter" />',
),
);
return zipSync(zip);
}
/** ODT: give the Standard master page a header with the marking and put the
* marking next to the existing page number in its footer LibreOffice
* repeats master-page headers/footers on every page. */
function patchOdt(bytes) {
const zip = unzipSync(new Uint8Array(bytes));
const marking = escapeXml(MARKING);
const styles = strFromU8(zip['styles.xml']);
if (!styles.includes('<style:footer>')) throw new Error('reference.odt has no footer');
const patched = styles
.replace(
'<style:footer>',
`<style:header><text:p text:style-name="MP1">${marking}</text:p></style:header><style:footer>`,
)
.replace(
'<style:footer>\n <text:p text:style-name="MP1">',
`<style:footer>\n <text:p text:style-name="MP1">${marking} · `,
);
zip['styles.xml'] = strToU8(patched);
return zipSync(zip);
}
mkdirSync(outDir, { recursive: true });
writeFileSync(join(outDir, 'reference-vs-nfd.docx'), patchDocx(defaultReference('reference.docx')));
writeFileSync(join(outDir, 'reference-vs-nfd.odt'), patchOdt(defaultReference('reference.odt')));
console.log(`generated reference-vs-nfd.docx/.odt in ${outDir} (marking: ${MARKING})`);

View File

@ -0,0 +1,127 @@
#!/usr/bin/env node
/**
* Generates the shipped default favicons (issue #306):
* `apps/api/assets/default-favicon-32.png` and `-180.png`.
*
* The api serves these whenever an operator has not uploaded one, so an
* instance always has a tab icon the `<link rel="icon">` in index.html is
* static and its resource must never 404.
*
* Drawn here rather than pulled in as a binary: the whole toolchain must
* survive the `--network none` offline build (96-offline-build-protokoll.md),
* and adding an image library for one 32×32 icon would be the tail wagging
* the dog. Node's own zlib is enough to write a PNG.
*
* Motif: a pond seen from above the accent-green disc with two ripples.
*
* Regenerate with `node apps/api/scripts/gen-default-favicon.mjs`, commit
* script and binaries together.
*/
import { deflateSync } from 'node:zlib';
import { writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
/** Brand green — the same value as index.html's light `theme-color`. */
const GREEN = [0x2f, 0x6f, 0x4f];
const LIGHT = [0xe8, 0xf2, 0xec];
const crcTable = Array.from({ length: 256 }, (_, n) => {
let c = n;
for (let k = 0; k < 8; k += 1) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
return c >>> 0;
});
function crc32(buf) {
let c = 0xffffffff;
for (const byte of buf) c = crcTable[(c ^ byte) & 0xff] ^ (c >>> 8);
return (c ^ 0xffffffff) >>> 0;
}
function chunk(type, data) {
const length = Buffer.alloc(4);
length.writeUInt32BE(data.length);
const body = Buffer.concat([Buffer.from(type, 'ascii'), data]);
const crc = Buffer.alloc(4);
crc.writeUInt32BE(crc32(body));
return Buffer.concat([length, body, crc]);
}
/** Minimal RGBA PNG writer — no filtering, one IDAT. */
function encodePng(size, rgba) {
const ihdr = Buffer.alloc(13);
ihdr.writeUInt32BE(size, 0);
ihdr.writeUInt32BE(size, 4);
ihdr[8] = 8; // bit depth
ihdr[9] = 6; // colour type RGBA
const raw = Buffer.alloc(size * (size * 4 + 1));
for (let y = 0; y < size; y += 1) {
raw[y * (size * 4 + 1)] = 0; // filter: none
rgba.copy(raw, y * (size * 4 + 1) + 1, y * size * 4, (y + 1) * size * 4);
}
return Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
chunk('IHDR', ihdr),
chunk('IDAT', deflateSync(raw, { level: 9 })),
chunk('IEND', Buffer.alloc(0)),
]);
}
/**
* Colour at one point of the unit square, in continuous coordinates the
* caller supersamples it, which is where the anti-aliasing comes from.
*/
function sample(x, y) {
const dx = x - 0.5;
const dy = y - 0.5;
const r = Math.hypot(dx, dy);
if (r > 0.48) return null; // outside the disc: transparent
// Two ripples spreading from a point struck slightly above centre — rings
// rather than a bullseye, which is why the centre stays green and the
// spacing widens outward the way real ripples do.
const rr = Math.hypot(dx, dy + 0.06);
const onRing = (radius, width) => Math.abs(rr - radius) < width;
if (onRing(0.33, 0.028) || onRing(0.19, 0.026)) return LIGHT;
return GREEN;
}
function render(size) {
const SS = 4; // supersampling factor
const out = Buffer.alloc(size * size * 4);
for (let y = 0; y < size; y += 1) {
for (let x = 0; x < size; x += 1) {
let r = 0;
let g = 0;
let b = 0;
let a = 0;
for (let sy = 0; sy < SS; sy += 1) {
for (let sx = 0; sx < SS; sx += 1) {
const c = sample((x + (sx + 0.5) / SS) / size, (y + (sy + 0.5) / SS) / size);
if (c) {
r += c[0];
g += c[1];
b += c[2];
a += 255;
}
}
}
const n = SS * SS;
const covered = a / 255;
const i = (y * size + x) * 4;
// Premultiplied average of the covered samples only, so the edge fades
// in alpha rather than towards black.
out[i] = covered ? Math.round(r / covered) : 0;
out[i + 1] = covered ? Math.round(g / covered) : 0;
out[i + 2] = covered ? Math.round(b / covered) : 0;
out[i + 3] = Math.round(a / n);
}
}
return out;
}
const assets = join(dirname(fileURLToPath(import.meta.url)), '../assets');
for (const size of [32, 180]) {
const file = join(assets, `default-favicon-${size}.png`);
writeFileSync(file, encodePng(size, render(size)));
console.log(`wrote ${file}`);
}

View File

@ -11,9 +11,17 @@ import {
} from '../settings/instance-settings.service';
import { SiteAdminGuard } from './site-admin.guard';
// Lifecycle markers, not configuration: never editable through this
// endpoint (the setup lock must be irreversible, issue #80).
const INTERNAL_KEYS: ReadonlySet<InstanceSettingKey> = new Set(['setup.completedAt']);
// Lifecycle markers and file-backed metadata, not configuration: never
// editable through this endpoint. The setup lock must be irreversible
// (issue #80), and the branding entries only describe bytes on disk
// (issue #306) — writing one by hand would claim an asset that is not
// there. Both have their own write paths.
const INTERNAL_KEYS: ReadonlySet<InstanceSettingKey> = new Set([
'setup.completedAt',
'instance.logo',
'instance.logoDark',
'instance.favicon',
]);
// Partial update: any subset of the known settings, each validated by
// its own schema inside the service (double validation is fine — this

View File

@ -2,8 +2,10 @@ import { Module } from '@nestjs/common';
import { AuthModule } from '../auth/auth.module';
import { BackupModule } from '../backup/backup.module';
import { PondsModule } from '../ponds/ponds.module';
import { QuotasModule } from '../quotas/quotas.module';
import { SchedulerModule } from '../scheduler/scheduler.module';
import { SearchModule } from '../search/search.module';
import { UsersModule } from '../users/users.module';
import { AdminSettingsController } from './admin.controller';
@ -18,7 +20,15 @@ import { UserAdminController } from './user-admin.controller';
import { UserAdminService } from './user-admin.service';
@Module({
imports: [QuotasModule, UsersModule, AuthModule, SchedulerModule, BackupModule],
imports: [
QuotasModule,
UsersModule,
AuthModule,
SchedulerModule,
BackupModule,
SearchModule,
PondsModule,
],
controllers: [
AdminSettingsController,
BackupAdminController,

View File

@ -4,6 +4,7 @@ import { PinoLogger } from 'nestjs-pino';
import { AuditService } from '../audit/audit.service';
import { PrismaService } from '../prisma/prisma.service';
import { SearchProvider } from '../search/search.provider';
/**
* GDPR account deletion (issue #59, security.md §Privacy). Rather than
@ -17,6 +18,7 @@ export class PseudonymizationService {
constructor(
private readonly prisma: PrismaService,
private readonly audit: AuditService,
private readonly search: SearchProvider,
private readonly logger: PinoLogger,
) {
this.logger.setContext(PseudonymizationService.name);
@ -45,6 +47,14 @@ export class PseudonymizationService {
data: { deletedAt: new Date(), deletedBy: userId },
});
});
// Trash path includes leaving the search index (issue #195).
const personalPonds = await this.prisma.pond.findMany({
where: { ownerId: userId, type: 'PERSONAL' },
select: { id: true },
});
for (const pond of personalPonds) {
await this.search.removePond(pond.id);
}
await this.audit.record({
action: 'user.pseudonymized',
targetType: 'user',

View File

@ -1,16 +1,21 @@
import { Controller, Get, Param, Post, Query, Req, UseGuards } from '@nestjs/common';
import {
auditListQuerySchema,
readEventListQuerySchema,
type AuditListQuery,
type AuditListView,
type JobTriggerResult,
type ReadEventListQuery,
type ReadEventListView,
type StorageOverviewView,
type SystemBackupView,
type SystemJobView,
type VsNfdProfileView,
} from '@dorfteich/shared';
import { AuthedRequest } from '../auth/auth.guard';
import { ZodValidationPipe } from '../common/zod-validation.pipe';
import { VsNfdProfileService } from '../settings/vs-nfd-profile.service';
import { SiteAdminGuard } from './site-admin.guard';
import { SystemAdminService } from './system-admin.service';
@ -18,7 +23,17 @@ import { SystemAdminService } from './system-admin.service';
@Controller('admin/system')
@UseGuards(SiteAdminGuard)
export class SystemAdminController {
constructor(private readonly system: SystemAdminService) {}
constructor(
private readonly system: SystemAdminService,
private readonly vsNfdProfile: VsNfdProfileService,
) {}
/** Active VS-NfD mode + catalog verdict for the running configuration
* (issue #243, ADR 0027). Exposure only the treatments are #244#246. */
@Get('vs-nfd-profile')
vsNfd(): Promise<VsNfdProfileView> {
return this.vsNfdProfile.evaluate();
}
@Get('jobs')
async jobs(): Promise<SystemJobView[]> {
@ -45,6 +60,15 @@ export class SystemAdminController {
return this.system.auditLog(query);
}
/** Read-access trail queries (issue #224): "who read page X", "what did
* user Y read" Site-Admin only, like the audit viewer above. */
@Get('read-events')
async readEvents(
@Query(new ZodValidationPipe(readEventListQuerySchema)) query: ReadEventListQuery,
): Promise<ReadEventListView> {
return this.system.readEvents(query);
}
@Get('storage')
async storage(): Promise<StorageOverviewView> {
return this.system.storage();

View File

@ -7,10 +7,13 @@ import {
AUDIT_PAGE_SIZE,
BACKUP_FRESH_MAX_AGE_HOURS,
BACKUP_STATUS_FILE,
READ_EVENT_PAGE_SIZE,
type AuditListQuery,
type AuditListView,
type BackupStatus,
type JobTriggerResult,
type ReadEventListQuery,
type ReadEventListView,
type StorageOverviewView,
type SystemBackupView,
type SystemJobView,
@ -160,6 +163,66 @@ export class SystemAdminService {
};
}
/**
* The Site-Admin query path over the read-access trail (issue #224,
* ADR 0023) evidence nobody can read is not evidence. Answers "who read
* page X" and "what did user Y read" within a period. API-only by design
* (no panel yet): the trail is an examiner's tool, not a daily screen
* documented in data-model.md §read_events.
*/
async readEvents(query: ReadEventListQuery): Promise<ReadEventListView> {
const where: Prisma.ReadEventWhereInput = {};
if (query.pageId) where.pageId = query.pageId;
if (query.actor) {
const actor = await this.prisma.user.findUnique({ where: { username: query.actor } });
// An unknown username matches nothing rather than everything.
where.actorId = actor?.id ?? '00000000-0000-0000-0000-000000000000';
}
if (query.channel) where.channel = query.channel;
if (query.from || query.to) {
where.occurredAt = {
...(query.from ? { gte: query.from } : {}),
...(query.to ? { lte: query.to } : {}),
};
}
const total = await this.prisma.readEvent.count({ where });
const pageCount = Math.max(1, Math.ceil(total / READ_EVENT_PAGE_SIZE));
const page = Math.min(query.page, pageCount);
const events = await this.prisma.readEvent.findMany({
where,
orderBy: { occurredAt: 'desc' },
skip: (page - 1) * READ_EVENT_PAGE_SIZE,
take: READ_EVENT_PAGE_SIZE,
});
// No FK on actor_id (evidence outlives accounts) — resolve what still
// exists in one query, show the bare id otherwise.
const actorIds = [...new Set(events.map((e) => e.actorId).filter((id): id is string => !!id))];
const actors = actorIds.length
? await this.prisma.user.findMany({
where: { id: { in: actorIds } },
select: { id: true, username: true, displayName: true },
})
: [];
const actorById = new Map(actors.map((a) => [a.id, a]));
return {
entries: events.map((event) => ({
id: event.id,
occurredAt: event.occurredAt.toISOString(),
actor: event.actorId ? (actorById.get(event.actorId) ?? null) : null,
pageId: event.pageId,
pondId: event.pondId,
channel: event.channel,
classification: event.classification,
windowSeconds: event.windowSeconds,
details: (event.details as Record<string, unknown> | null) ?? null,
})),
page,
pageCount,
total,
};
}
async storage(): Promise<StorageOverviewView> {
const usages = await this.prisma.pondUsage.findMany({
where: { pond: { deletedAt: null } },

View File

@ -12,9 +12,11 @@ import {
UseGuards,
} from '@nestjs/common';
import {
AdminCreateUserInput,
AdminUserListQuery,
AdminUserListView,
AdminUserView,
adminCreateUserSchema,
adminUserListQuerySchema,
setSiteAdminSchema,
setUserDisabledSchema,
@ -31,6 +33,14 @@ import { UserAdminService } from './user-admin.service';
export class UserAdminController {
constructor(private readonly users: UserAdminService) {}
@Post()
async create(
@Body(new ZodValidationPipe(adminCreateUserSchema)) input: AdminCreateUserInput,
@Req() request: AuthedRequest,
): Promise<AdminUserView> {
return this.users.createUser(request.user!, input);
}
@Get()
async list(
@Query(new ZodValidationPipe(adminUserListQuerySchema)) query: AdminUserListQuery,

View File

@ -6,7 +6,7 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { PondsService } from '../ponds/ponds.service';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { createTestPrisma, deletePondsWhere, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
/**
@ -62,13 +62,71 @@ describe.skipIf(!hasTestDb)('user admin (e2e, issue #59)', () => {
afterAll(async () => {
const all = Object.values(ids);
await prisma.session.deleteMany({ where: { userId: { in: all } } });
await prisma.pond.deleteMany({ where: { ownerId: { in: all } } });
await deletePondsWhere(prisma, { ownerId: { in: all } });
await prisma.userIdentity.deleteMany({ where: { userId: { in: all } } });
await prisma.user.deleteMany({ where: { id: { in: all } } });
await prisma.$disconnect();
await app.close();
});
it('creates an account that can log in right away, with a personal pond (issue #331)', async () => {
const username = `ua-created-${suffix}`;
const res = await api()
.post('/api/v1/admin/users')
.set('Cookie', cookies.admin1!)
.send({
username,
email: `${username}@example.org`,
displayName: 'UA Created',
password,
locale: 'de',
})
.expect(201);
const created = res.body as { id: string; status: string };
ids.created = created.id;
// No verification hop: the admin vouched for the address.
expect(created.status).toBe('ACTIVE');
await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: username, password })
.expect(200);
// The personal pond exists exactly like after self-registration.
expect(await prisma.pond.count({ where: { ownerId: created.id, type: 'PERSONAL' } })).toBe(1);
});
it('rejects duplicate usernames with a field-level conflict', async () => {
await api()
.post('/api/v1/admin/users')
.set('Cookie', cookies.admin1!)
.send({
username: `ua-created-${suffix}`,
email: `ua-created-other-${suffix}@example.org`,
displayName: 'UA Dup',
password,
locale: 'en',
})
.expect(409)
.expect((r) =>
expect((r.body as { details: Record<string, string[]> }).details.username).toEqual([
'validation.taken',
]),
);
});
it('refuses creation for non-admins', async () => {
await api()
.post('/api/v1/admin/users')
.set('Cookie', cookies.bob!)
.send({
username: `ua-sneak-${suffix}`,
email: `ua-sneak-${suffix}@example.org`,
displayName: 'UA Sneak',
password,
locale: 'en',
})
.expect(403);
});
it('lists and searches users (Site-Admin only)', async () => {
const res = await api()
.get(`/api/v1/admin/users?q=ua-bob-${suffix}`)

View File

@ -1,5 +1,6 @@
import { BadRequestException, Injectable, NotFoundException } from '@nestjs/common';
import {
AdminCreateUserInput,
AdminUserListQuery,
AdminUserListView,
AdminUserStatus,
@ -10,7 +11,9 @@ import { PinoLogger } from 'nestjs-pino';
import { AuthService } from '../auth/auth.service';
import { AuditService } from '../audit/audit.service';
import { PondsService } from '../ponds/ponds.service';
import { PrismaService } from '../prisma/prisma.service';
import { UsersService } from '../users/users.service';
import { PseudonymizationService } from './pseudonymization.service';
/**
@ -27,12 +30,33 @@ export class UserAdminService {
private readonly prisma: PrismaService,
private readonly pseudonymizer: PseudonymizationService,
private readonly auth: AuthService,
private readonly users: UsersService,
private readonly ponds: PondsService,
private readonly audit: AuditService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(UserAdminService.name);
}
/**
* Creates an account on behalf of a user (issue #331). The e-mail is
* marked verified immediately the admin vouches for the address and
* the personal pond is provisioned exactly like the verify-email path
* does, so the account is indistinguishable from a self-registered one.
*/
async createUser(actor: User, input: AdminCreateUserInput): Promise<AdminUserView> {
const user = await this.users.createUser(input);
const verified = await this.users.markEmailVerified(user.id);
await this.ponds.ensurePersonalPond(verified);
await this.audit.record({
action: 'user.created_by_admin',
actorId: actor.id,
targetType: 'user',
targetId: user.id,
});
return this.viewOf(verified, await this.pondCountOf(user.id));
}
async list(query: AdminUserListQuery): Promise<AdminUserListView> {
const q = query.q?.trim();
const where: Prisma.UserWhereInput = q
@ -115,7 +139,9 @@ export class UserAdminService {
if (!value && user.isSiteAdmin) await this.assertNotLastSiteAdmin();
const updated = await this.prisma.user.update({
where: { id },
data: { isSiteAdmin: value },
// A manual toggle takes ownership of the flag: the IdP mapping
// (#217) may only revoke what it itself set.
data: { isSiteAdmin: value, isSiteAdminManaged: false },
});
await this.audit.record({
action: 'user.site_admin_set',

View File

@ -1,4 +1,4 @@
import { Module } from '@nestjs/common';
import { MiddlewareConsumer, Module, NestModule } from '@nestjs/common';
import { APP_FILTER } from '@nestjs/core';
import { LoggerModule } from 'nestjs-pino';
@ -6,8 +6,10 @@ import { AdminModule } from './admin/admin.module';
import { AuditModule } from './audit/audit.module';
import { AuthModule } from './auth/auth.module';
import { BackupModule } from './backup/backup.module';
import { BrandingModule } from './branding/branding.module';
import { ApiExceptionFilter } from './common/api-exception.filter';
import { maskTokenParam } from './common/mask-token-param';
import { SecurityHeadersMiddleware } from './common/security-headers.middleware';
import { CommentsModule } from './comments/comments.module';
import { CompactionModule } from './compaction/compaction.module';
import { AppConfig } from './config/app-config.service';
@ -16,6 +18,7 @@ import { FilesModule } from './files/files.module';
import { GrantsModule } from './grants/grants.module';
import { HealthModule } from './health/health.module';
import { HomeModule } from './home/home.module';
import { FontsModule } from './fonts/fonts.module';
import { ImportExportModule } from './import-export/import-export.module';
import { LabelsModule } from './labels/labels.module';
import { LegalModule } from './legal/legal.module';
@ -31,6 +34,7 @@ import { PrismaModule } from './prisma/prisma.module';
import { PublicApiModule } from './public-api/public-api.module';
import { PublicModule } from './public/public.module';
import { RateLimitModule } from './rate-limit/rate-limit.module';
import { ReadTrailModule } from './read-trail/read-trail.module';
import { SearchModule } from './search/search.module';
import { SettingsModule } from './settings/settings.module';
import { SetupModule } from './setup/setup.module';
@ -46,6 +50,7 @@ import { VersionsModule } from './versions/versions.module';
ConfigModule,
PrismaModule,
AuditModule,
ReadTrailModule,
RateLimitModule,
MailModule,
SettingsModule,
@ -78,6 +83,8 @@ import { VersionsModule } from './versions/versions.module';
PublicModule,
PublicApiModule,
McpModule,
BrandingModule,
FontsModule,
ImportExportModule,
PluginsModule,
AuthModule,
@ -104,4 +111,10 @@ import { VersionsModule } from './versions/versions.module';
],
providers: [{ provide: APP_FILTER, useClass: ApiExceptionFilter }],
})
export class AppModule {}
export class AppModule implements NestModule {
configure(consumer: MiddlewareConsumer): void {
// Module-level (not main.ts) so createTestApp boots the identical
// security-header/CORS middleware — see security-headers.middleware.ts.
consumer.apply(SecurityHeadersMiddleware).forRoutes('{*path}');
}
}

View File

@ -0,0 +1,75 @@
/**
* The audit event catalogue (issue #201): every action id the trail may
* carry, with the severity the stdout line is stamped with. This const is
* the CODE half of the published catalogue in
* `docs/architecture/audit-events.md` `audit-catalogue.test.ts` fails
* whenever the two drift, so an id cannot be added, renamed, or removed
* without its documentation moving in the same commit.
*
* Compatibility promise (the reason this exists): ids are never repurposed.
* New events may be added (minor catalogue version); an id that stops being
* emitted is retired in the catalogue document, its meaning frozen forever
* so an operator's SIEM rules survive our releases.
*/
export const AUDIT_EVENTS = {
'api.token_created': { severity: 'info' },
'api.token_revoked': { severity: 'info' },
'api.write': { severity: 'info' },
'audit.pruned': { severity: 'info' },
'auth.email_verified': { severity: 'info' },
'auth.identity_linked': { severity: 'notice' },
'auth.login_failed': { severity: 'warning' },
'auth.login_succeeded': { severity: 'info' },
'auth.password_reset': { severity: 'notice' },
'auth.proxy_rejected': { severity: 'warning' },
'auth.signup': { severity: 'info' },
'backup.restore_requested': { severity: 'warning' },
'backup.run_triggered': { severity: 'info' },
'backup.settings_changed': { severity: 'notice' },
'file.integrity_failed': { severity: 'critical' },
'grant.created': { severity: 'notice' },
'grant.deleted': { severity: 'notice' },
'invitation.accepted': { severity: 'notice' },
'invitation.created': { severity: 'info' },
'invitation.revoked': { severity: 'info' },
'job.triggered': { severity: 'info' },
'member.added': { severity: 'notice' },
'member.removed': { severity: 'notice' },
'member.role_changed': { severity: 'notice' },
'page.classification_lowered': { severity: 'warning' },
'page.classification_raised': { severity: 'notice' },
'plugin.installed': { severity: 'notice' },
'plugin.rejected': { severity: 'warning' },
'plugin.mode_set': { severity: 'notice' },
'plugin.pond_toggled': { severity: 'info' },
'plugin.uninstalled': { severity: 'notice' },
'pond.archived': { severity: 'notice' },
'pond.purged': { severity: 'notice' },
'quota.override_cleared': { severity: 'notice' },
'quota.override_set': { severity: 'notice' },
'read_trail.pruned': { severity: 'info' },
'settings.changed': { severity: 'notice' },
'branding.changed': { severity: 'notice' },
'font.uploaded': { severity: 'notice' },
'font.deleted': { severity: 'notice' },
'setup.admin_created': { severity: 'notice' },
'setup.completed': { severity: 'info' },
'setup.preseeded': { severity: 'info' },
'setup.smtp_stored': { severity: 'info' },
'user.created_by_admin': { severity: 'notice' },
'user.deleted': { severity: 'notice' },
'user.disabled_set': { severity: 'notice' },
'user.pseudonymized': { severity: 'notice' },
'user.site_admin_set': { severity: 'notice' },
'user.verification_resent': { severity: 'info' },
} as const satisfies Record<string, { severity: AuditSeverity }>;
/** Severity vocabulary of the catalogue syslog-inspired, four levels are
* enough for rule routing (critical pages someone, warning feeds detection,
* notice is configuration drift, info is lifecycle noise). */
export type AuditSeverity = 'info' | 'notice' | 'warning' | 'critical';
/** A catalogued action id the ONLY thing {@link AuditService.record}
* accepts, so an uncatalogued event cannot be emitted (compile-time), and
* the doc fence keeps the catalogue document in step (test-time). */
export type AuditAction = keyof typeof AUDIT_EVENTS;

View File

@ -0,0 +1,48 @@
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import { describe, expect, it } from 'vitest';
import { AUDIT_EVENTS } from './audit-actions';
/**
* The fence that keeps the published audit catalogue and the code together
* (issue #201): every id in `AUDIT_EVENTS` must appear as an event row in
* `docs/architecture/audit-events.md` with the same severity, and the
* document may not describe ids the code does not know. Emission of an
* uncatalogued id is already a TYPE error (AuditAction union) this test
* covers the half the compiler cannot see: the document.
*/
// __dirname, not import.meta: the api package compiles CJS (tsconfig has no
// nodenext module), and vitest resolves both — the compiler only the former.
const doc = readFileSync(join(__dirname, '../../../../docs/architecture/audit-events.md'), 'utf8');
/** Event rows are `| \`ns.event\` | trigger | severity | ` the dot in the
* id keeps field-set rows (`msg`, `severity`, ) out of the match. The
* namespace may carry an underscore since `read_trail.*` (issue #224). */
function documentedEvents(): Map<string, string> {
const events = new Map<string, string>();
for (const line of doc.split('\n')) {
const id = /^\| `([a-z_]+\.[a-z_]+)` +\|/.exec(line)?.[1];
if (!id) continue;
const cells = line.split('|').map((cell) => cell.trim());
// cells[0] is the empty string before the leading pipe.
events.set(id, cells[3] ?? '');
}
return events;
}
describe('audit catalogue fence (issue #201)', () => {
it('documents exactly the ids the code can emit', () => {
const documented = documentedEvents();
const inCode = Object.keys(AUDIT_EVENTS).sort();
expect([...documented.keys()].sort()).toEqual(inCode);
});
it('documents each id with the severity the code stamps', () => {
const documented = documentedEvents();
for (const [action, { severity }] of Object.entries(AUDIT_EVENTS)) {
expect(`${action}: ${documented.get(action)}`).toBe(`${action}: ${severity}`);
}
});
});

View File

@ -0,0 +1,90 @@
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { AuditRetentionService } from './audit-retention.service';
const DAY = 24 * 60 * 60 * 1000;
/**
* Audit-trail retention (issue #196): entries past `audit.retentionDays`
* are pruned, newer ones stay, and the pruning itself lands in the trail
* (`audit.pruned` with count and cutoff) so the gap is explainable.
*/
describe.skipIf(!hasTestDb)('audit retention (e2e, issue #196)', () => {
let app: INestApplication;
let prisma: PrismaClient;
const suffix = uniqueSuffix();
const marker = `retention-${suffix}`;
beforeAll(async () => {
prisma = createTestPrisma();
// A short period so ages are unambiguous; written straight to the row
// BEFORE the app boots (the settings cache is in-process and fills on
// first read). The key is cleaned afterAll.
await prisma.instanceSetting.upsert({
where: { key: 'audit.retentionDays' },
create: { key: 'audit.retentionDays', value: 30 },
update: { value: 30 },
});
app = await createTestApp();
});
afterAll(async () => {
await prisma.instanceSetting.deleteMany({ where: { key: 'audit.retentionDays' } });
await prisma.auditEntry.deleteMany({
where: { OR: [{ targetId: { contains: suffix } }, { action: 'audit.pruned' }] },
});
await prisma.$disconnect();
await app.close();
});
it('prunes entries past the period, keeps newer ones, and records the pruning', async () => {
await prisma.auditEntry.createMany({
data: [
{
action: 'test.old',
targetType: 'test',
targetId: marker,
at: new Date(Date.now() - 40 * DAY),
},
{
action: 'test.older',
targetType: 'test',
targetId: marker,
at: new Date(Date.now() - 400 * DAY),
},
{
action: 'test.fresh',
targetType: 'test',
targetId: marker,
at: new Date(Date.now() - 5 * DAY),
},
],
});
const pruned = await app.get(AuditRetentionService).pruneExpired();
expect(pruned).toBeGreaterThanOrEqual(2);
const remaining = await prisma.auditEntry.findMany({ where: { targetId: marker } });
expect(remaining.map((entry) => entry.action)).toEqual(['test.fresh']);
// The gap is explainable: the pruning run is itself on the trail.
const prunedEvent = await prisma.auditEntry.findFirst({
where: { action: 'audit.pruned' },
orderBy: { at: 'desc' },
});
expect(prunedEvent).not.toBeNull();
expect(prunedEvent!.details).toMatchObject({ retentionDays: 30 });
expect((prunedEvent!.details as { count: number }).count).toBeGreaterThanOrEqual(2);
});
it('is a no-op when nothing is due', async () => {
const pruned = await app.get(AuditRetentionService).pruneExpired();
expect(pruned).toBe(0);
// The fresh marker entry from the first test is untouched.
expect(await prisma.auditEntry.count({ where: { targetId: marker } })).toBe(1);
});
});

View File

@ -0,0 +1,47 @@
import { Injectable } from '@nestjs/common';
import { PinoLogger } from 'nestjs-pino';
import { ClockService } from '../common/clock.service';
import { PrismaService } from '../prisma/prisma.service';
import { InstanceSettingsService } from '../settings/instance-settings.service';
import { AuditService } from './audit.service';
const MS_PER_DAY = 24 * 60 * 60 * 1000;
/**
* Audit-trail retention (issue #196): the daily job deletes `audit_log`
* entries older than the configurable `audit.retentionDays` (default one
* year) and records the deletion itself (`audit.pruned` with count and
* cutoff) so a gap in the trail is always explainable. Separate from
* {@link AuditService} because the settings service audits its own writes
* folding retention into AuditService would close a constructor cycle.
* The read-access trail (#224) is deliberately not covered here.
*/
@Injectable()
export class AuditRetentionService {
constructor(
private readonly prisma: PrismaService,
private readonly settings: InstanceSettingsService,
private readonly audit: AuditService,
private readonly clock: ClockService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(AuditRetentionService.name);
}
async pruneExpired(): Promise<number> {
const retentionDays = await this.settings.get('audit.retentionDays');
const cutoff = new Date(this.clock.now().getTime() - retentionDays * MS_PER_DAY);
const { count } = await this.prisma.auditEntry.deleteMany({
where: { at: { lt: cutoff } },
});
if (count > 0) {
await this.audit.record({
action: 'audit.pruned',
details: { count, cutoff: cutoff.toISOString(), retentionDays },
});
}
return count;
}
}

View File

@ -1,7 +1,16 @@
import { Global, Module } from '@nestjs/common';
import { Global, Module, OnModuleInit } from '@nestjs/common';
import { CommonModule } from '../common/common.module';
import { SchedulerModule } from '../scheduler/scheduler.module';
import { SchedulerService } from '../scheduler/scheduler.service';
import { SettingsModule } from '../settings/settings.module';
import { AuditRetentionService } from './audit-retention.service';
import { AuditService } from './audit.service';
/** Daily, per operations.md's maintenance-jobs table (issue #196). */
const AUDIT_RETENTION_CADENCE_SECONDS = 24 * 60 * 60;
/**
* Global because the audit trail cuts across nearly every feature module
* (auth, grants, members, admin, plugins, setup) like PrismaModule, one
@ -9,7 +18,23 @@ import { AuditService } from './audit.service';
*/
@Global()
@Module({
providers: [AuditService],
imports: [CommonModule, SchedulerModule, SettingsModule],
providers: [AuditService, AuditRetentionService],
exports: [AuditService],
})
export class AuditModule {}
export class AuditModule implements OnModuleInit {
constructor(
private readonly scheduler: SchedulerService,
private readonly retention: AuditRetentionService,
) {}
onModuleInit(): void {
this.scheduler.register({
name: 'audit-retention',
cadenceSeconds: AUDIT_RETENTION_CADENCE_SECONDS,
run: async () => {
await this.retention.pruneExpired();
},
});
}
}

View File

@ -4,9 +4,13 @@ import { PinoLogger } from 'nestjs-pino';
import { PrismaService } from '../prisma/prisma.service';
import { AUDIT_EVENTS, AuditAction } from './audit-actions';
export interface AuditEvent {
/** Stable dot-namespaced id, e.g. `grant.created` — the UI translates it. */
action: string;
/** Stable dot-namespaced id from the catalogue (issue #201,
* docs/architecture/audit-events.md) the UI translates it, SIEM rules
* key on it. The union makes an uncatalogued emission a type error. */
action: AuditAction;
/** The acting user; null/undefined for anonymous events. */
actorId?: string | null;
targetType?: string;
@ -37,7 +41,15 @@ export class AuditService {
async record(event: AuditEvent): Promise<void> {
const { action, actorId, targetType, targetId, details } = event;
this.logger.info(
{ actor: actorId ?? null, targetType, targetId, ...details },
// `severity` is the catalogue's routing hint for SIEM rules (#201) —
// pino's own `level` stays 30/info so log transport is unaffected.
{
severity: AUDIT_EVENTS[action].severity,
actor: actorId ?? null,
targetType,
targetId,
...details,
},
`audit: ${action}`,
);
try {

View File

@ -1,5 +1,6 @@
import { Body, Controller, Get, HttpCode, Post, Req, Res } from '@nestjs/common';
import {
AuthMethodsView,
CurrentUser as CurrentUserShape,
LoginInput,
SignupInput,
@ -20,12 +21,14 @@ import { InstanceSettingsService } from '../settings/instance-settings.service';
import { SetupExempt } from '../setup/setup.guard';
import {
AuthedRequest,
LocalCredentialFlow,
Public,
SESSION_COOKIE,
setSessionCookie,
toCurrentUser,
} from './auth.guard';
import { AuthService } from './auth.service';
import { OidcService } from './oidc.service';
import { SessionsService, sessionAbsoluteMs } from './sessions.service';
@AuthenticatedOnly() // routes reachable without a session opt out via @Public
@ -36,6 +39,7 @@ export class AuthController {
private readonly sessions: SessionsService,
private readonly config: AppConfig,
private readonly settings: InstanceSettingsService,
private readonly oidc: OidcService,
) {}
/** Public: the SPA hides the signup route while registration is closed. */
@ -45,8 +49,21 @@ export class AuthController {
return { mode: await this.settings.get('auth.registrationMode') };
}
/** Public: what the login screen offers (issue #214) the local form
* and/or the deploy-configured OIDC provider. */
@SetupExempt()
@Public()
@Get('methods')
methods(): AuthMethodsView {
return {
local: this.config.env.AUTH_LOCAL_ENABLED,
oidc: this.oidc.enabled ? { label: this.oidc.providerLabel } : null,
};
}
@Public()
@Post('signup')
@LocalCredentialFlow()
@HttpCode(201)
@RateLimit({ scope: 'signup', limit: 5, windowSeconds: 60 * 60 })
async signup(@Body(new ZodValidationPipe(signupInputSchema)) input: SignupInput): Promise<void> {
@ -55,6 +72,7 @@ export class AuthController {
@Public()
@Post('verify-email')
@LocalCredentialFlow()
@HttpCode(204)
@RateLimit({ scope: 'verify-email', limit: 20, windowSeconds: 60 * 60 })
async verifyEmail(
@ -65,6 +83,7 @@ export class AuthController {
@Public()
@Post('resend-verification')
@LocalCredentialFlow()
@HttpCode(204)
@RateLimit({ scope: 'resend-verification', limit: 5, windowSeconds: 60 * 60 })
async resendVerification(
@ -78,6 +97,7 @@ export class AuthController {
@SetupExempt()
@Public()
@Post('login')
@LocalCredentialFlow()
@HttpCode(200)
@RateLimit({ scope: 'login', limit: 10, windowSeconds: 60 })
async login(
@ -121,6 +141,7 @@ export class AuthController {
@Public()
@Post('forgot-password')
@LocalCredentialFlow()
@HttpCode(204)
@RateLimit({ scope: 'forgot-password', limit: 5, windowSeconds: 60 * 60 })
async forgotPassword(
@ -131,6 +152,7 @@ export class AuthController {
@Public()
@Post('reset-password')
@LocalCredentialFlow()
@HttpCode(204)
@RateLimit({ scope: 'reset-password', limit: 10, windowSeconds: 60 * 60 })
async resetPassword(

View File

@ -4,7 +4,7 @@ import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { createTestPrisma, deletePondsWhere, hasTestDb, uniqueSuffix } from '../testing/test-db';
describe.skipIf(!hasTestDb)('auth flows (e2e)', () => {
let app: INestApplication;
@ -46,7 +46,7 @@ describe.skipIf(!hasTestDb)('auth flows (e2e)', () => {
afterAll(async () => {
// Verified users own a personal pond (#21) — remove it before them.
await prisma.pond.deleteMany({ where: { owner: { username: { contains: suffix } } } });
await deletePondsWhere(prisma, { owner: { username: { contains: suffix } } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.mailOutbox.deleteMany({ where: { toAddress: { contains: suffix } } });
await prisma.$disconnect();

View File

@ -3,6 +3,7 @@ import {
ExecutionContext,
ForbiddenException,
Injectable,
NotFoundException,
SetMetadata,
UnauthorizedException,
createParamDecorator,
@ -13,6 +14,7 @@ import type { User } from '@prisma/client';
import type { Request, Response } from 'express';
import { AppConfig } from '../config/app-config.service';
import { ProxyIdentityService } from './proxy-identity.service';
import { SessionsService } from './sessions.service';
export const SESSION_COOKIE = 'dt_session';
@ -21,6 +23,18 @@ const IS_PUBLIC_KEY = 'isPublic';
/** Marks a route as reachable without a session (login, signup, healthz…). */
export const Public = (): MethodDecorator & ClassDecorator => SetMetadata(IS_PUBLIC_KEY, true);
export const LOCAL_CREDENTIAL_KEY = 'isLocalCredentialFlow';
/**
* Marks a route as part of the LOCAL credential machinery (issue #216,
* ADR 0021): password login, signup, e-mail verification, password
* forgot/reset/change. With `AUTH_LOCAL_ENABLED=false` every marked route
* answers 404 (existence hidden, the switch precedent) and the
* enumeration fence in `local-auth-switch.e2e.db.test.ts` fails when an
* auth route is neither marked nor on its reviewed allowlist, so a new
* credential flow cannot ship unswitched by accident.
*/
export const LocalCredentialFlow = (): MethodDecorator => SetMetadata(LOCAL_CREDENTIAL_KEY, true);
export interface AuthedRequest extends Request {
user?: User;
sessionId?: string;
@ -84,19 +98,38 @@ export class AuthGuard implements CanActivate {
private readonly reflector: Reflector,
private readonly sessions: SessionsService,
private readonly config: AppConfig,
private readonly proxyIdentity: ProxyIdentityService,
) {}
async canActivate(context: ExecutionContext): Promise<boolean> {
const request = context.switchToHttp().getRequest<AuthedRequest>();
// The hard local-auth switch (issue #216): marked credential routes
// disappear entirely — before any session or CSRF logic runs.
if (!this.config.env.AUTH_LOCAL_ENABLED) {
const isLocalFlow = this.reflector.getAllAndOverride<boolean>(LOCAL_CREDENTIAL_KEY, [
context.getHandler(),
context.getClass(),
]);
if (isLocalFlow) throw new NotFoundException();
}
const rawToken = (request.cookies as Record<string, string> | undefined)?.[SESSION_COOKIE];
if (rawToken && MUTATING_METHODS.has(request.method)) {
this.assertSameOrigin(request);
}
// Attach the user whenever the cookie is valid — public routes may
// still want to know who is asking.
if (rawToken) {
// Trusted-proxy identity first (issue #215): when the perimeter
// authenticates, its header IS the identity for this request — a
// session cookie riding along never escalates beyond it, and an
// untrusted peer carrying the header is rejected inside resolve().
const proxyUser = await this.proxyIdentity.resolve(request);
if (proxyUser) {
request.user = proxyUser;
} else if (rawToken) {
// Attach the user whenever the cookie is valid — public routes may
// still want to know who is asking.
const validated = await this.sessions.validate(rawToken);
if (validated) {
request.user = validated.user;

View File

@ -1,6 +1,10 @@
import { Module } from '@nestjs/common';
import { Logger, Module, OnModuleInit } from '@nestjs/common';
import { APP_GUARD } from '@nestjs/core';
import { AppConfig } from '../config/app-config.service';
import { GrantsModule } from '../grants/grants.module';
import { InvitationsModule } from '../invitations/invitations.module';
import { MailModule } from '../mail/mail.module';
import { PondsModule } from '../ponds/ponds.module';
import { UsersModule } from '../users/users.module';
@ -8,18 +12,42 @@ import { AuthController } from './auth.controller';
import { AuthGuard } from './auth.guard';
import { AuthService } from './auth.service';
import { AuthTokensService } from './auth-tokens.service';
import { ClaimMappingService } from './claim-mapping.service';
import { OidcController } from './oidc.controller';
import { OidcService } from './oidc.service';
import { ProxyIdentityService } from './proxy-identity.service';
import { SessionsModule } from './sessions.module';
@Module({
imports: [UsersModule, MailModule, SessionsModule, PondsModule],
controllers: [AuthController],
imports: [UsersModule, MailModule, SessionsModule, PondsModule, GrantsModule, InvitationsModule],
controllers: [AuthController, OidcController],
providers: [
AuthService,
AuthTokensService,
ClaimMappingService,
OidcService,
ProxyIdentityService,
// Global default-protected: every route needs a session unless it
// opts out with @Public().
{ provide: APP_GUARD, useClass: AuthGuard },
],
exports: [AuthTokensService, AuthService],
exports: [AuthTokensService, AuthService, OidcService],
})
export class AuthModule {}
export class AuthModule implements OnModuleInit {
constructor(
private readonly config: AppConfig,
private readonly oidc: OidcService,
private readonly proxyIdentity: ProxyIdentityService,
) {}
onModuleInit(): void {
// #216: local auth off without ANY external path means nobody can ever
// sign in — loudly stated at boot, because the operator will otherwise
// discover it at the login screen.
if (!this.config.env.AUTH_LOCAL_ENABLED && !this.oidc.enabled && !this.proxyIdentity.enabled) {
new Logger(AuthModule.name).warn(
'AUTH_LOCAL_ENABLED=false with neither OIDC nor proxy authentication configured — no sign-in path exists',
);
}
}
}

View File

@ -9,6 +9,7 @@ import { User } from '@prisma/client';
import { PinoLogger } from 'nestjs-pino';
import { AppConfig } from '../config/app-config.service';
import { InvitationsService } from '../invitations/invitations.service';
import { MailService } from '../mail/mail.service';
import { PondsService } from '../ponds/ponds.service';
import { AuditService } from '../audit/audit.service';
@ -33,6 +34,7 @@ export class AuthService {
private readonly sessions: SessionsService,
private readonly mail: MailService,
private readonly ponds: PondsService,
private readonly invitations: InvitationsService,
private readonly rateLimits: RateLimitService,
private readonly audit: AuditService,
private readonly config: AppConfig,
@ -43,10 +45,37 @@ export class AuthService {
}
async signup(input: SignupInput): Promise<void> {
if ((await this.settings.get('auth.registrationMode')) === 'closed') {
// An invitation token (issue #332) lets exactly one signup through a
// closed registration. Claimed atomically BEFORE the account exists;
// rolled back if the signup fails (duplicate username), so the invitee
// can retry with the same link.
const invitation = input.invitationToken
? await this.invitations.redeem(input.invitationToken)
: null;
if (input.invitationToken && !invitation) {
throw new BadRequestException({ code: 'token_invalid' });
}
if (!invitation && (await this.settings.get('auth.registrationMode')) === 'closed') {
throw new ForbiddenException({ code: 'registration_closed' });
}
const user = await this.users.createUser(input);
let user: User;
try {
user = await this.users.createUser(input);
} catch (error) {
if (invitation) await this.invitations.unredeem(invitation.id);
throw error;
}
if (invitation) {
await this.invitations.markAccepted(invitation.id, user.id);
await this.audit.record({
action: 'invitation.accepted',
actorId: user.id,
targetType: 'invitation',
targetId: invitation.id,
});
}
// The invite link proves nothing about the mailbox (it can be
// forwarded), so the usual verification mail still applies.
await this.sendVerificationMail(user);
await this.audit.record({ action: 'auth.signup', actorId: user.id });
}

View File

@ -0,0 +1,272 @@
import { createServer, type Server } from 'node:http';
import type { AddressInfo } from 'node:net';
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import { SignJWT, exportJWK, generateKeyPair, type JWTPayload } from 'jose';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import { PondAccessNotifier } from '../ponds/pond-access-notifier.service';
import { InstanceSettingsService } from '../settings/instance-settings.service';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
/**
* IdP claim mapping (issue #217, ADR 0021): declarative `idpMapping.rules`
* turn ID-token claims into pond roles and the site-admin flag on every
* OIDC login through the same grant-service path as manual grants (the
* collab revocation notify is asserted), with removal on the next login,
* "manual wins" precedence, and audited changes.
*/
describe.skipIf(!hasTestDb)('idp claim mapping (e2e, issue #217)', () => {
let app: INestApplication;
let prisma: PrismaClient;
let idp: Server;
let issuer: string;
const suffix = uniqueSuffix();
let signingKey: CryptoKey;
let publicJwk: Record<string, unknown>;
let nextClaims: (nonce: string) => JWTPayload;
let currentNonce = '';
let adminId: string;
let pondId: string;
const pondSlug = `mapped-${suffix}`;
const api = () => request(app.getHttpServer());
async function loginViaIdp(): Promise<string> {
const begin = await api().get('/api/v1/auth/oidc/login').expect(302);
const url = new URL(begin.headers.location!);
currentNonce = url.searchParams.get('nonce')!;
const stateCookie = (begin.headers['set-cookie'] as unknown as string[])
.find((c) => c.startsWith('dt_oidc='))!
.split(';')[0]!;
const res = await api()
.get(
`/api/v1/auth/oidc/callback?code=fake&state=${encodeURIComponent(
url.searchParams.get('state')!,
)}`,
)
.set('Cookie', stateCookie)
.expect(302);
expect(res.headers.location!).toMatch(/\/$/);
return sessionCookieOf(res);
}
function subjectClaims(groups: string[]): (nonce: string) => JWTPayload {
return (nonce) => ({
iss: issuer,
aud: 'dorfteich-map',
sub: `mapped-${suffix}`,
nonce,
email: `mapped-${suffix}@idp.example`,
email_verified: true,
preferred_username: `mapped-${suffix}`,
groups,
});
}
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
let signingPublic: CryptoKey;
({ privateKey: signingKey, publicKey: signingPublic } = await generateKeyPair('RS256', {
extractable: true,
}));
publicJwk = { ...(await exportJWK(signingPublic)), kid: 'map-key', alg: 'RS256' };
idp = createServer((req, res) => {
void (async () => {
res.setHeader('content-type', 'application/json');
if (req.url === '/.well-known/openid-configuration') {
res.end(
JSON.stringify({
issuer,
authorization_endpoint: `${issuer}/authorize`,
token_endpoint: `${issuer}/token`,
jwks_uri: `${issuer}/jwks`,
}),
);
} else if (req.url === '/jwks') {
res.end(JSON.stringify({ keys: [publicJwk] }));
} else if (req.url === '/token') {
req.resume();
req.on('end', () => {
void (async () => {
const now = Math.floor(Date.now() / 1000);
const idToken = await new SignJWT({ ...nextClaims(currentNonce) })
.setProtectedHeader({ alg: 'RS256', kid: 'map-key' })
.setIssuedAt(now)
.setExpirationTime(now + 300)
.sign(signingKey);
res.end(JSON.stringify({ id_token: idToken }));
})();
});
} else {
res.statusCode = 404;
res.end();
}
})();
});
await new Promise<void>((resolve) => idp.listen(0, '127.0.0.1', resolve));
issuer = `http://127.0.0.1:${(idp.address() as AddressInfo).port}`;
process.env.OIDC_ISSUER = issuer;
process.env.OIDC_CLIENT_ID = 'dorfteich-map';
app = await createTestApp();
// A pond to map into, owned by an admin user (created via the service,
// grants via prisma BEFORE the first permission query — test-db rule).
const users = app.get(UsersService);
const admin = await users.createUser({
username: `map-admin-${suffix}`,
email: `map-admin-${suffix}@example.test`,
displayName: 'Map Admin',
password: 'mapping admin 123',
locale: 'en',
});
await users.markEmailVerified(admin.id);
adminId = admin.id;
const pond = await prisma.pond.create({
data: { slug: pondSlug, name: 'Mapped Pond', type: 'SHARED', ownerId: adminId },
});
pondId = pond.id;
await prisma.roleGrant.create({
data: {
pondId,
subjectType: 'USER',
subjectId: adminId,
role: 'POND_ADMIN',
scopeType: 'POND',
scopeId: null,
effect: 'ALLOW',
createdBy: adminId,
},
});
await app.get(InstanceSettingsService).set(
'idpMapping.rules',
[
{ claim: 'groups', value: 'wiki-editors', role: 'editor', pondSlug },
{ claim: 'groups', value: 'wiki-admins', role: 'site_admin' },
],
adminId,
);
});
afterAll(async () => {
delete process.env.OIDC_ISSUER;
delete process.env.OIDC_CLIENT_ID;
await new Promise<void>((resolve) => idp.close(() => resolve()));
await prisma.instanceSetting.deleteMany({ where: { key: 'idpMapping.rules' } });
await prisma.userIdentity.deleteMany({ where: { provider: `oidc:${issuer}` } });
await prisma.roleGrant.deleteMany({ where: { pondId } });
await prisma.page.deleteMany({
where: { pond: { owner: { username: { contains: suffix } } } },
});
await prisma.roleGrant.deleteMany({
where: { pond: { owner: { username: { contains: suffix } } } },
});
await prisma.pond.deleteMany({ where: { owner: { username: { contains: suffix } } } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();
});
it('grants the mapped pond role on login and access actually works', async () => {
nextClaims = subjectClaims(['wiki-editors']);
const session = await loginViaIdp();
const grant = await prisma.roleGrant.findFirst({
where: { pondId, subjectType: 'USER', origin: 'idp' },
});
expect(grant).toMatchObject({ role: 'EDITOR', effect: 'ALLOW' });
// The permission model actually honours it (no raw-row bypass).
const pages = await api()
.get(`/api/v1/ponds/${pondId}/pages`)
.set('Cookie', session)
.expect(200);
expect(Array.isArray(pages.body)).toBe(true);
const audit = await prisma.auditEntry.findFirst({
where: { action: 'grant.created', targetId: pondId },
orderBy: { at: 'desc' },
});
expect(audit?.details).toMatchObject({ origin: 'idp_mapping' });
});
it('revokes the mapped grant on the next login without the claim — via the revocation path', async () => {
const notifier = app.get(PondAccessNotifier);
const notifySpy = vi.spyOn(notifier, 'notifyAccessChanged');
nextClaims = subjectClaims([]);
const session = await loginViaIdp();
try {
expect(await prisma.roleGrant.findFirst({ where: { pondId, origin: 'idp' } })).toBeNull();
// The removal travelled through the grant service: the collab
// revocation notify fired for this pond (the pg_notify access
// listener terminates live sessions — that path's own tests cover
// the socket close).
expect(notifySpy.mock.calls.some(([id]) => id === pondId)).toBe(true);
// …and the pond is out of reach again (404: existence hidden).
await api().get(`/api/v1/ponds/${pondId}/pages`).set('Cookie', session).expect(404);
} finally {
notifySpy.mockRestore();
}
});
it('never touches a manual grant, and re-creating over one is skipped (manual wins)', async () => {
const user = await prisma.user.findUnique({
where: { email: `mapped-${suffix}@idp.example` },
});
// A manual reader grant made by the pond admin.
await prisma.roleGrant.create({
data: {
pondId,
subjectType: 'USER',
subjectId: user!.id,
role: 'READER',
scopeType: 'POND',
scopeId: null,
effect: 'ALLOW',
createdBy: adminId,
origin: 'manual',
},
});
// Login without any mapped claim: the manual grant survives.
nextClaims = subjectClaims([]);
await loginViaIdp();
const manual = await prisma.roleGrant.findFirst({
where: { pondId, subjectId: user!.id, origin: 'manual' },
});
expect(manual).not.toBeNull();
expect(manual!.role).toBe('READER');
});
it('maps and revokes the site-admin flag — but never demotes a hand-promoted admin', async () => {
nextClaims = subjectClaims(['wiki-admins']);
await loginViaIdp();
let user = await prisma.user.findUnique({ where: { email: `mapped-${suffix}@idp.example` } });
expect(user).toMatchObject({ isSiteAdmin: true, isSiteAdminManaged: true });
nextClaims = subjectClaims([]);
await loginViaIdp();
user = await prisma.user.findUnique({ where: { email: `mapped-${suffix}@idp.example` } });
expect(user).toMatchObject({ isSiteAdmin: false, isSiteAdminManaged: false });
// Hand-promoted (managed=false): a claimless login must not demote.
await prisma.user.update({
where: { id: user!.id },
data: { isSiteAdmin: true, isSiteAdminManaged: false },
});
nextClaims = subjectClaims([]);
await loginViaIdp();
user = await prisma.user.findUnique({ where: { email: `mapped-${suffix}@idp.example` } });
expect(user!.isSiteAdmin).toBe(true);
});
});

View File

@ -0,0 +1,161 @@
import { Injectable } from '@nestjs/common';
import { User } from '@prisma/client';
import type { JWTPayload } from 'jose';
import { PinoLogger } from 'nestjs-pino';
import { AuditService } from '../audit/audit.service';
import { GrantsService } from '../grants/grants.service';
import { PrismaService } from '../prisma/prisma.service';
import { InstanceSettingsService } from '../settings/instance-settings.service';
/**
* IdP claim mapping (issue #217, ADR 0021): on every OIDC login the
* declarative rules in `idpMapping.rules` are evaluated against the ID
* token's claims and reconciled against the user's MAPPING-OWNED state:
*
* - Pond grants are created and revoked through {@link GrantsService}
* the same path as manual grants, so the permission cache is
* invalidated and live collab sessions are revalidated
* (`notifyAccessChanged` the collab access listener) exactly as on a
* manual change. No raw row writes.
* - The mapping only ever touches rows with `origin = 'idp'` and only
* demotes a site admin whose flag it itself set
* (`isSiteAdminManaged`) **manual wins**: hand-made grants and
* hand-promoted admins are never revoked by a missing claim.
* - Every change is audited (grant.created/grant.deleted with
* `origin: idp_mapping`; user.site_admin_set with the same marker).
*
* Reconciliation happens at login because that is when fresh claims
* exist; between logins the leaver case is the IdP's (disable there =
* no new login) plus the operator's account-disable flag.
*/
@Injectable()
export class ClaimMappingService {
constructor(
private readonly prisma: PrismaService,
private readonly grants: GrantsService,
private readonly settings: InstanceSettingsService,
private readonly audit: AuditService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(ClaimMappingService.name);
}
async apply(user: User, payload: JWTPayload): Promise<void> {
const rules = await this.settings.get('idpMapping.rules');
if (rules.length === 0) return;
const matched = rules.filter((rule) => claimMatches(payload[rule.claim], rule.value));
await this.reconcileSiteAdmin(
user,
matched.some((rule) => rule.role === 'site_admin'),
);
// Desired pond grants, resolved slug → id (unknown slugs are a
// configuration error: logged, never fatal for the login).
const desired = new Map<string, 'pond_admin' | 'editor' | 'reader'>();
for (const rule of matched) {
if (rule.role === 'site_admin') continue;
const pond = await this.prisma.pond.findFirst({
where: { slug: rule.pondSlug!, deletedAt: null },
select: { id: true },
});
if (!pond) {
this.logger.warn({ pondSlug: rule.pondSlug }, 'idp mapping: unknown pond slug');
continue;
}
// Multiple rules for one pond: the strongest role wins.
const current = desired.get(pond.id);
if (!current || rank(rule.role) > rank(current)) desired.set(pond.id, rule.role);
}
const existing = await this.prisma.roleGrant.findMany({
where: { subjectType: 'USER', subjectId: user.id, origin: 'idp' },
});
for (const grant of existing) {
const wanted = desired.get(grant.pondId);
if (wanted && toDbRole(wanted) === grant.role) {
desired.delete(grant.pondId); // already in place
continue;
}
try {
await this.grants.deleteGrant(user, grant.pondId, grant.id, { origin: 'idp' });
} catch (error) {
// E.g. the last-Pond-Admin protection: the grant stays, the login
// proceeds — an operator decision is needed, not a lockout.
this.logger.warn(
{ grantId: grant.id, pondId: grant.pondId, err: error },
'idp mapping: grant revocation refused',
);
}
}
for (const [pondId, role] of desired) {
try {
await this.grants.createGrant(
user,
pondId,
{
subjectType: 'user',
subjectId: user.id,
role,
scopeType: 'pond',
scopeId: null,
effect: 'allow',
},
{ origin: 'idp' },
);
} catch (error) {
// A colliding MANUAL grant (grant_exists) is fine — manual wins,
// the mapping never replaces it with an owned copy.
this.logger.warn({ pondId, role, err: error }, 'idp mapping: grant creation skipped');
}
}
}
private async reconcileSiteAdmin(user: User, shouldBeAdmin: boolean): Promise<void> {
if (shouldBeAdmin && !user.isSiteAdmin) {
await this.prisma.user.update({
where: { id: user.id },
data: { isSiteAdmin: true, isSiteAdminManaged: true },
});
await this.audit.record({
action: 'user.site_admin_set',
actorId: user.id,
targetType: 'user',
targetId: user.id,
details: { isSiteAdmin: true, origin: 'idp_mapping' },
});
} else if (!shouldBeAdmin && user.isSiteAdmin && user.isSiteAdminManaged) {
// Only the mapping's own promotion is revocable by a missing claim.
await this.prisma.user.update({
where: { id: user.id },
data: { isSiteAdmin: false, isSiteAdminManaged: false },
});
await this.audit.record({
action: 'user.site_admin_set',
actorId: user.id,
targetType: 'user',
targetId: user.id,
details: { isSiteAdmin: false, origin: 'idp_mapping' },
});
}
}
}
/** A claim matches when it equals the value or, as an array, contains it. */
function claimMatches(claim: unknown, value: string): boolean {
if (Array.isArray(claim)) return claim.some((entry) => String(entry) === value);
if (claim === undefined || claim === null) return false;
return String(claim) === value;
}
function rank(role: 'pond_admin' | 'editor' | 'reader'): number {
return role === 'pond_admin' ? 3 : role === 'editor' ? 2 : 1;
}
function toDbRole(role: 'pond_admin' | 'editor' | 'reader'): 'POND_ADMIN' | 'EDITOR' | 'READER' {
return role === 'pond_admin' ? 'POND_ADMIN' : role === 'editor' ? 'EDITOR' : 'READER';
}

View File

@ -0,0 +1,157 @@
import 'reflect-metadata';
import { INestApplication } from '@nestjs/common';
import { PATH_METADATA } from '@nestjs/common/constants';
import { PrismaClient } from '@prisma/client';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
import { LOCAL_CREDENTIAL_KEY } from './auth.guard';
import { AuthController } from './auth.controller';
import { OidcController } from './oidc.controller';
import { SessionsService } from './sessions.service';
/**
* The hard local-auth switch (issue #216, ADR 0021): AUTH_LOCAL_ENABLED=false
* closes EVERY local credential flow with 404 enumerated, not assumed
* while sessions themselves, logout, and token issuance for
* externally-authenticated users keep working (the stated decision: PATs
* and feed tokens authorize API access under their own switches, they are
* not interactive sign-in). A fence asserts every auth route is either
* marked as a local flow or on the reviewed allowlist.
*/
describe.skipIf(!hasTestDb)('local-auth switch (e2e, issue #216)', () => {
let app: INestApplication;
let prisma: PrismaClient;
const suffix = uniqueSuffix();
/** Every local credential surface — the enumeration the issue demands. */
const LOCAL_ROUTES: { method: 'post'; path: string; body: Record<string, unknown> }[] = [
{ method: 'post', path: '/api/v1/auth/login', body: { usernameOrEmail: 'x', password: 'y' } },
{
method: 'post',
path: '/api/v1/auth/signup',
body: {
username: `switch-${suffix}`,
email: `switch-${suffix}@example.test`,
displayName: 'x',
password: 'ein langes passwort 123',
locale: 'en',
},
},
{ method: 'post', path: '/api/v1/auth/verify-email', body: { token: 'x' } },
{
method: 'post',
path: '/api/v1/auth/resend-verification',
body: { email: 'x@example.test' },
},
{ method: 'post', path: '/api/v1/auth/forgot-password', body: { email: 'x@example.test' } },
{
method: 'post',
path: '/api/v1/auth/reset-password',
body: { token: 'x', password: 'ein langes passwort 123' },
},
{
method: 'post',
path: '/api/v1/users/me/change-password',
body: { currentPassword: 'x', newPassword: 'ein langes passwort 123' },
},
];
const api = () => request(app.getHttpServer());
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
process.env.AUTH_LOCAL_ENABLED = 'false';
app = await createTestApp();
});
afterAll(async () => {
delete process.env.AUTH_LOCAL_ENABLED;
await prisma.apiToken.deleteMany({ where: { user: { username: { contains: suffix } } } });
await prisma.feedToken.deleteMany({ where: { user: { username: { contains: suffix } } } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();
});
it('answers 404 on every enumerated local credential route', async () => {
for (const route of LOCAL_ROUTES) {
const res = await api()[route.method](route.path).send(route.body);
expect(`${route.path}: ${res.status}`).toBe(`${route.path}: 404`);
}
});
it('reports local:false so the login screen hides the form', async () => {
const res = await api().get('/api/v1/auth/methods').expect(200);
expect(res.body.local).toBe(false);
});
it('keeps sessions, logout, and PAT/feed-token issuance working for externally-authenticated users', async () => {
// An externally-authenticated user is simulated by creating the session
// through the session service — exactly what the OIDC/proxy paths do.
const users = app.get(UsersService);
const user = await users.createUser({
username: `ext-${suffix}`,
email: `ext-${suffix}@example.test`,
displayName: 'External',
password: 'nie benutzt weil lokal aus',
locale: 'en',
});
await users.markEmailVerified(user.id);
const token = await app.get(SessionsService).create(user.id, undefined);
const cookie = `dt_session=${token}`;
const me = await api().get('/api/v1/auth/me').set('Cookie', cookie).expect(200);
expect(me.body.id).toBe(user.id);
// Stated decision (#216): token issuance is API authorization, not
// interactive sign-in — it stays available under its own switches.
await api()
.post('/api/v1/users/me/api-tokens')
.set('Cookie', cookie)
.send({ name: `switch-${suffix}`, scope: 'read' })
.expect(201);
await api()
.post('/api/v1/users/me/feed-tokens')
.set('Cookie', cookie)
.send({ name: `switch-${suffix}` })
.expect(201);
await api().post('/api/v1/auth/logout').set('Cookie', cookie).expect(204);
await api().get('/api/v1/auth/me').set('Cookie', cookie).expect(401);
});
it('fence: every auth route is either a marked local flow or on the reviewed allowlist', () => {
// Routes that must stay reachable with local auth off — reviewed here.
const allowlist = new Set([
'registration', // signup-mode discovery; harmless metadata
'methods', // the login screen's discovery endpoint
'logout', // ending a session is not a credential flow
'me', // session introspection
'login', // OidcController: IdP redirect
'link', // OidcController: explicit identity linking
'callback', // OidcController: IdP return leg
]);
for (const controller of [AuthController, OidcController]) {
for (const name of Object.getOwnPropertyNames(controller.prototype)) {
if (name === 'constructor') continue;
const handler = controller.prototype[name as keyof typeof controller.prototype] as (
...args: unknown[]
) => unknown;
const path = Reflect.getMetadata(PATH_METADATA, handler) as string | undefined;
if (path === undefined) continue; // not a route
const marked = Reflect.getMetadata(LOCAL_CREDENTIAL_KEY, handler) === true;
expect(
marked || allowlist.has(path),
`${controller.name}.${name} (path "${path}") is neither @LocalCredentialFlow nor allowlisted`,
).toBe(true);
}
}
});
});

View File

@ -0,0 +1,106 @@
import { Controller, Get, Query, Req, Res } from '@nestjs/common';
import type { Response } from 'express';
import { AppConfig } from '../config/app-config.service';
import { AuthenticatedOnly } from '../permissions/permission.decorators';
import { RateLimit } from '../rate-limit/rate-limit.guard';
import { AuthedRequest, Public, setSessionCookie } from './auth.guard';
import { OidcService } from './oidc.service';
import { sessionAbsoluteMs } from './sessions.service';
/** Carries state+nonce+PKCE verifier across the IdP round-trip signed
* (purpose-derived key), HttpOnly, Lax so the top-level callback
* navigation still sends it, and 10 minutes short-lived. */
const OIDC_STATE_COOKIE = 'dt_oidc';
/**
* OIDC endpoints (issue #214, ADR 0021). Browser-navigation shaped: `login`
* and `link` answer 302 to the IdP, the callback lands back here and
* redirects into the SPA errors become `/login?error=<code>` so the SPA
* can translate them.
*/
@AuthenticatedOnly()
@Controller('auth/oidc')
export class OidcController {
constructor(
private readonly oidc: OidcService,
private readonly config: AppConfig,
) {}
private stateCookie(response: Response, value: string): void {
response.cookie(OIDC_STATE_COOKIE, value, {
httpOnly: true,
sameSite: 'lax',
secure: this.config.env.NODE_ENV === 'production',
maxAge: 10 * 60 * 1000,
path: '/',
});
}
@Public()
@Get('login')
@RateLimit({ scope: 'oidc-login', limit: 30, windowSeconds: 60 })
async login(@Res() response: Response): Promise<void> {
this.oidc.assertEnabled();
const { url, stateToken } = await this.oidc.beginLogin();
this.stateCookie(response, stateToken);
response.redirect(url);
}
/** The deliberate account-linking flow (ADR 0021 §2): only a logged-in
* user attaches an IdP identity to their own account. */
@Get('link')
@RateLimit({ scope: 'oidc-login', limit: 30, windowSeconds: 60 })
async link(@Req() request: AuthedRequest, @Res() response: Response): Promise<void> {
this.oidc.assertEnabled();
const { url, stateToken } = await this.oidc.beginLogin(request.user!.id);
this.stateCookie(response, stateToken);
response.redirect(url);
}
@Public()
@Get('callback')
@RateLimit({ scope: 'oidc-callback', limit: 30, windowSeconds: 60 })
async callback(
@Query('code') code: string | undefined,
@Query('state') state: string | undefined,
@Query('error') idpError: string | undefined,
@Req() request: AuthedRequest,
@Res() response: Response,
): Promise<void> {
this.oidc.assertEnabled();
const base = this.config.env.APP_BASE_URL;
response.clearCookie(OIDC_STATE_COOKIE, { path: '/' });
const stateToken = (request.cookies as Record<string, string> | undefined)?.[OIDC_STATE_COOKIE];
if (idpError || !code || !state || !stateToken) {
response.redirect(`${base}/login?error=oidc_cancelled`);
return;
}
try {
const result = await this.oidc.completeLogin(
code,
state,
stateToken,
request.headers['user-agent'],
);
if (result.linked) {
response.redirect(`${base}/settings?oidc=linked`);
return;
}
setSessionCookie(
response,
result.sessionToken!,
this.config.env.NODE_ENV === 'production',
sessionAbsoluteMs(this.config.env),
);
response.redirect(`${base}/`);
} catch (error) {
const code_ =
typeof (error as { response?: { code?: string } })?.response?.code === 'string'
? (error as { response: { code: string } }).response.code
: 'oidc_failed';
response.redirect(`${base}/login?error=${encodeURIComponent(code_)}`);
}
}
}

View File

@ -0,0 +1,351 @@
import { createServer, type Server } from 'node:http';
import type { AddressInfo } from 'node:net';
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import { SignJWT, exportJWK, generateKeyPair, type JWTPayload } from 'jose';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
/**
* OIDC Authorization Code + PKCE against a local fake IdP (issue #214,
* ADR 0021): discovery, JWKS-validated ID tokens, state/nonce binding, PKCE
* verifier at the token endpoint, JIT account creation, the documented
* refusal to link silently by e-mail, and the explicit link flow. The fake
* IdP is protocol-shaped exactly like Keycloak's endpoints the Keycloak
* verification itself is a manual procedure (security.md §External
* authentication).
*/
describe.skipIf(!hasTestDb)('oidc login (e2e, issue #214)', () => {
let app: INestApplication;
let prisma: PrismaClient;
let idp: Server;
let issuer: string;
const suffix = uniqueSuffix();
let signingKey: CryptoKey;
let publicJwk: Record<string, unknown>;
let wrongKey: CryptoKey;
/** What the fake token endpoint returns next (set per test). */
let nextIdToken: (() => Promise<string>) | null = null;
/** The last body the token endpoint received (PKCE assertions). */
let lastTokenRequest: URLSearchParams | null = null;
const api = () => request(app.getHttpServer());
async function mintIdToken(
claims: JWTPayload,
options: { key?: CryptoKey; expired?: boolean } = {},
): Promise<string> {
const now = Math.floor(Date.now() / 1000);
return new SignJWT({ ...claims })
.setProtectedHeader({ alg: 'RS256', kid: 'test-key' })
.setIssuedAt(options.expired ? now - 7200 : now)
.setExpirationTime(options.expired ? now - 3600 : now + 300)
.sign(options.key ?? signingKey);
}
/** Runs /auth/oidc/login and returns the pieces the callback needs. */
async function beginLogin(cookie?: string) {
const req = api().get('/api/v1/auth/oidc/login');
const res = await (cookie ? req.set('Cookie', cookie) : req).expect(302);
const url = new URL(res.headers.location!);
const stateCookie = (res.headers['set-cookie'] as unknown as string[])
.find((c) => c.startsWith('dt_oidc='))!
.split(';')[0]!;
return {
state: url.searchParams.get('state')!,
nonce: url.searchParams.get('nonce')!,
challenge: url.searchParams.get('code_challenge')!,
stateCookie,
authorizeUrl: url,
};
}
async function callback(state: string, stateCookie: string) {
return api()
.get(`/api/v1/auth/oidc/callback?code=fake-code&state=${encodeURIComponent(state)}`)
.set('Cookie', stateCookie);
}
function redirectTarget(res: request.Response): string {
return res.headers.location!;
}
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
let signingPublic: CryptoKey;
({ privateKey: signingKey, publicKey: signingPublic } = await generateKeyPair('RS256', {
extractable: true,
}));
({ privateKey: wrongKey } = await generateKeyPair('RS256', { extractable: true }));
publicJwk = { ...(await exportJWK(signingPublic)), kid: 'test-key', alg: 'RS256' };
idp = createServer((req, res) => {
void (async () => {
if (req.url === '/.well-known/openid-configuration') {
res.setHeader('content-type', 'application/json');
res.end(
JSON.stringify({
issuer,
authorization_endpoint: `${issuer}/authorize`,
token_endpoint: `${issuer}/token`,
jwks_uri: `${issuer}/jwks`,
}),
);
return;
}
if (req.url === '/jwks') {
res.setHeader('content-type', 'application/json');
res.end(JSON.stringify({ keys: [publicJwk] }));
return;
}
if (req.url === '/token') {
let body = '';
req.on('data', (chunk) => (body += chunk));
req.on('end', () => {
void (async () => {
lastTokenRequest = new URLSearchParams(body);
res.setHeader('content-type', 'application/json');
if (!nextIdToken) {
res.statusCode = 400;
res.end(JSON.stringify({ error: 'invalid_grant' }));
return;
}
res.end(JSON.stringify({ id_token: await nextIdToken(), token_type: 'Bearer' }));
})();
});
return;
}
res.statusCode = 404;
res.end();
})();
});
await new Promise<void>((resolve) => idp.listen(0, '127.0.0.1', resolve));
issuer = `http://127.0.0.1:${(idp.address() as AddressInfo).port}`;
process.env.OIDC_ISSUER = issuer;
process.env.OIDC_CLIENT_ID = 'dorfteich-test';
process.env.OIDC_PROVIDER_LABEL = 'Fake IdP';
app = await createTestApp();
});
afterAll(async () => {
delete process.env.OIDC_ISSUER;
delete process.env.OIDC_CLIENT_ID;
delete process.env.OIDC_PROVIDER_LABEL;
await new Promise<void>((resolve) => idp.close(() => resolve()));
await prisma.userIdentity.deleteMany({ where: { provider: `oidc:${issuer}` } });
await prisma.page.deleteMany({
where: { pond: { owner: { email: { contains: `${suffix}@idp.example` } } } },
});
await prisma.roleGrant.deleteMany({
where: { pond: { owner: { email: { contains: `${suffix}@idp.example` } } } },
});
await prisma.pond.deleteMany({
where: { owner: { email: { contains: `${suffix}@idp.example` } } },
});
await prisma.user.deleteMany({ where: { email: { contains: `${suffix}@idp.example` } } });
await prisma.user.deleteMany({ where: { username: { contains: `local-${suffix}` } } });
await prisma.$disconnect();
await app.close();
});
it('advertises the provider on /auth/methods', async () => {
const res = await api().get('/api/v1/auth/methods').expect(200);
expect(res.body).toEqual({ local: true, oidc: { label: 'Fake IdP' } });
});
it('logs in end to end: PKCE at the token endpoint, JIT user, identity, personal pond, session', async () => {
const { state, nonce, challenge, stateCookie, authorizeUrl } = await beginLogin();
expect(authorizeUrl.searchParams.get('code_challenge_method')).toBe('S256');
expect(authorizeUrl.searchParams.get('client_id')).toBe('dorfteich-test');
nextIdToken = () =>
mintIdToken({
iss: issuer,
aud: 'dorfteich-test',
sub: `subject-${suffix}`,
nonce,
email: `nadia-${suffix}@idp.example`,
email_verified: true,
preferred_username: `nadia-${suffix}`,
name: 'Nadia IdP',
});
const res = await callback(state, stateCookie);
expect(res.status).toBe(302);
expect(redirectTarget(res)).toMatch(/\/$/);
const session = sessionCookieOf(res);
expect(session).toContain('dt_session=');
// PKCE: the verifier travelled to the token endpoint and matches the
// challenge from the authorize redirect.
expect(lastTokenRequest?.get('grant_type')).toBe('authorization_code');
const verifier = lastTokenRequest?.get('code_verifier');
expect(verifier).toBeTruthy();
const { createHash } = await import('node:crypto');
expect(createHash('sha256').update(verifier!).digest('base64url')).toBe(challenge);
const user = await prisma.user.findUnique({
where: { email: `nadia-${suffix}@idp.example` },
});
expect(user).toMatchObject({ status: 'ACTIVE', displayName: 'Nadia IdP' });
const identity = await prisma.userIdentity.findUnique({
where: {
provider_subject: { provider: `oidc:${issuer}`, subject: `subject-${suffix}` },
},
});
expect(identity?.userId).toBe(user!.id);
const personal = await prisma.pond.findFirst({
where: { ownerId: user!.id, type: 'PERSONAL' },
});
expect(personal).not.toBeNull();
const me = await api().get('/api/v1/auth/me').set('Cookie', session).expect(200);
expect(me.body.email).toBe(`nadia-${suffix}@idp.example`);
});
it('reuses the existing account on the next login of the same subject', async () => {
const before = await prisma.user.count({ where: { email: { contains: `${suffix}@idp` } } });
const { state, nonce, stateCookie } = await beginLogin();
nextIdToken = () =>
mintIdToken({
iss: issuer,
aud: 'dorfteich-test',
sub: `subject-${suffix}`,
nonce,
email: `nadia-${suffix}@idp.example`,
email_verified: true,
});
const res = await callback(state, stateCookie);
expect(res.status).toBe(302);
expect(redirectTarget(res)).toMatch(/\/$/);
const after = await prisma.user.count({ where: { email: { contains: `${suffix}@idp` } } });
expect(after).toBe(before);
});
it('rejects a wrong state, a foreign nonce, a bad signature, wrong issuer/audience and an expired token', async () => {
// Wrong state: cookie from one round, state from nowhere.
const first = await beginLogin();
const bad = await callback('not-the-state', first.stateCookie);
expect(redirectTarget(bad)).toContain('error=oidc_state_invalid');
const cases: {
claims: (nonce: string) => JWTPayload;
options?: { key?: CryptoKey; expired?: boolean };
}[] = [
// Foreign nonce.
{ claims: () => baseClaims('other-nonce') },
// Signature from the wrong key.
{ claims: (n) => baseClaims(n), options: { key: wrongKey } },
// Wrong issuer.
{ claims: (n) => ({ ...baseClaims(n), iss: 'https://evil.example' }) },
// Wrong audience.
{ claims: (n) => ({ ...baseClaims(n), aud: 'someone-else' }) },
// Expired.
{ claims: (n) => baseClaims(n), options: { expired: true } },
];
function baseClaims(nonce: string): JWTPayload {
return {
iss: issuer,
aud: 'dorfteich-test',
sub: `reject-${suffix}`,
nonce,
email: `reject-${suffix}@idp.example`,
email_verified: true,
};
}
for (const testCase of cases) {
const { state, nonce, stateCookie } = await beginLogin();
nextIdToken = () => mintIdToken(testCase.claims(nonce), testCase.options);
const res = await callback(state, stateCookie);
expect(redirectTarget(res)).toContain('error=oidc_token_invalid');
}
// None of the rejected attempts created anything.
expect(
await prisma.user.findUnique({ where: { email: `reject-${suffix}@idp.example` } }),
).toBeNull();
});
it('refuses to adopt an existing local account by e-mail — and links it via the explicit flow', async () => {
const users = app.get(UsersService);
const password = 'lokales konto 123';
const local = await users.createUser({
username: `local-${suffix}`,
email: `local-${suffix}@idp.example`,
displayName: 'Local User',
password,
locale: 'en',
});
await users.markEmailVerified(local.id);
// Silent adoption refused (ADR 0021 §2 — account-takeover path).
const attempt = await beginLogin();
nextIdToken = () =>
mintIdToken({
iss: issuer,
aud: 'dorfteich-test',
sub: `local-subject-${suffix}`,
nonce: attempt.nonce,
email: `local-${suffix}@idp.example`,
email_verified: true,
});
const refused = await callback(attempt.state, attempt.stateCookie);
expect(redirectTarget(refused)).toContain('error=oidc_link_required');
// The explicit link flow, from a logged-in session.
const login = await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: `local-${suffix}`, password })
.expect(200);
const sessionCookie = sessionCookieOf(login);
const linkRes = await api()
.get('/api/v1/auth/oidc/link')
.set('Cookie', sessionCookie)
.expect(302);
const linkUrl = new URL(linkRes.headers.location!);
const linkState = linkUrl.searchParams.get('state')!;
const linkNonce = linkUrl.searchParams.get('nonce')!;
const linkCookie = (linkRes.headers['set-cookie'] as unknown as string[])
.find((c) => c.startsWith('dt_oidc='))!
.split(';')[0]!;
nextIdToken = () =>
mintIdToken({
iss: issuer,
aud: 'dorfteich-test',
sub: `local-subject-${suffix}`,
nonce: linkNonce,
email: `local-${suffix}@idp.example`,
email_verified: true,
});
const linked = await callback(linkState, linkCookie);
expect(redirectTarget(linked)).toContain('oidc=linked');
const identity = await prisma.userIdentity.findUnique({
where: {
provider_subject: { provider: `oidc:${issuer}`, subject: `local-subject-${suffix}` },
},
});
expect(identity?.userId).toBe(local.id);
// From now on the IdP login lands in the linked account.
const again = await beginLogin();
nextIdToken = () =>
mintIdToken({
iss: issuer,
aud: 'dorfteich-test',
sub: `local-subject-${suffix}`,
nonce: again.nonce,
email: `local-${suffix}@idp.example`,
email_verified: true,
});
const res = await callback(again.state, again.stateCookie);
const session = sessionCookieOf(res);
const me = await api().get('/api/v1/auth/me').set('Cookie', session).expect(200);
expect(me.body.id).toBe(local.id);
});
});

View File

@ -0,0 +1,359 @@
import { createHash, randomBytes } from 'node:crypto';
import {
BadRequestException,
ConflictException,
Injectable,
NotFoundException,
ServiceUnavailableException,
} from '@nestjs/common';
import { slugify } from '@dorfteich/shared';
import { deriveTokenKey } from '@dorfteich/shared/token-crypto';
import { User } from '@prisma/client';
import { SignJWT, createRemoteJWKSet, jwtVerify, type JWTPayload } from 'jose';
import { PinoLogger } from 'nestjs-pino';
import { AuditService } from '../audit/audit.service';
import { AppConfig } from '../config/app-config.service';
import { PondsService } from '../ponds/ponds.service';
import { PrismaService } from '../prisma/prisma.service';
import { UsersService } from '../users/users.service';
import { ClaimMappingService } from './claim-mapping.service';
import { SessionsService } from './sessions.service';
/** The state cookie's signed payload lives this long ample for one
* round-trip to the IdP's login form. */
const STATE_TTL_SECONDS = 10 * 60;
/** Explicit asymmetric allowlist for ID-token signatures (no HS*, no
* `none`): Keycloak's default RS256 plus the common EC profile. */
const ID_TOKEN_ALGORITHMS = ['RS256', 'ES256'];
/** What we mint into the signed, HttpOnly state cookie before redirecting
* to the IdP: CSRF binding (`state`), replay binding (`nonce`), the PKCE
* verifier, and for the deliberate account-linking flow the session
* user the new identity must attach to. */
interface OidcStateClaims extends JWTPayload {
state: string;
nonce: string;
codeVerifier: string;
linkUserId?: string;
}
interface DiscoveryDocument {
issuer: string;
authorization_endpoint: string;
token_endpoint: string;
jwks_uri: string;
end_session_endpoint?: string;
}
/**
* OIDC Authorization Code with PKCE (issue #214, ADR 0021). Deliberately
* built on `jose` (the vetted library from #188) plus `fetch` no new
* dependency enters the supply chain for a security base function.
* Discovery-based: nothing here is Keycloak-specific; Keycloak is the
* reference IdP the flow is verified against (procedure in
* `docs/architecture/security.md` §External authentication).
*
* Identity linking follows ADR 0021 §2: `provider = "oidc:<issuer>"`,
* `subject` from the token. An existing local account is NEVER linked
* silently by e-mail that would be an account-takeover path. Instead the
* login is refused with `oidc_link_required`, and the user (logged in
* locally) links explicitly via `GET /auth/oidc/link`.
*/
@Injectable()
export class OidcService {
private discoveryCache: DiscoveryDocument | null = null;
private jwks: ReturnType<typeof createRemoteJWKSet> | null = null;
constructor(
private readonly prisma: PrismaService,
private readonly users: UsersService,
private readonly sessions: SessionsService,
private readonly ponds: PondsService,
private readonly claimMapping: ClaimMappingService,
private readonly audit: AuditService,
private readonly config: AppConfig,
private readonly logger: PinoLogger,
) {
this.logger.setContext(OidcService.name);
}
/** OIDC is a deploy-level decision (ADR 0021): enabled iff issuer and
* client id are configured. */
get enabled(): boolean {
return Boolean(this.config.env.OIDC_ISSUER && this.config.env.OIDC_CLIENT_ID);
}
get providerLabel(): string {
return this.config.env.OIDC_PROVIDER_LABEL;
}
private get issuer(): string {
return this.config.env.OIDC_ISSUER!;
}
private get clientId(): string {
return this.config.env.OIDC_CLIENT_ID!;
}
private get redirectUri(): string {
return `${this.config.env.APP_BASE_URL}/api/v1/auth/oidc/callback`;
}
/** The identity provider key: one issuer, one provider namespace. */
private get provider(): string {
return `oidc:${this.issuer}`;
}
assertEnabled(): void {
// 404, not 403: consistent with the instance switches (`api.enabled`
// et al.) — an unconfigured surface hides its existence.
if (!this.enabled) throw new NotFoundException();
}
private async discover(): Promise<DiscoveryDocument> {
if (this.discoveryCache) return this.discoveryCache;
const url = `${this.issuer.replace(/\/$/, '')}/.well-known/openid-configuration`;
const response = await fetch(url).catch(() => null);
if (!response?.ok) {
throw new ServiceUnavailableException({ code: 'oidc_discovery_failed' });
}
const doc = (await response.json()) as DiscoveryDocument;
if (doc.issuer !== this.issuer) {
// RFC 8414 §3.3: the advertised issuer must match the configured one.
throw new ServiceUnavailableException({ code: 'oidc_discovery_failed' });
}
this.discoveryCache = doc;
this.jwks = createRemoteJWKSet(new URL(doc.jwks_uri));
return doc;
}
/** Builds the IdP redirect plus the signed state-cookie value. */
async beginLogin(linkUserId?: string): Promise<{ url: string; stateToken: string }> {
const doc = await this.discover();
const state = randomBytes(24).toString('base64url');
const nonce = randomBytes(24).toString('base64url');
const codeVerifier = randomBytes(48).toString('base64url');
const challenge = createHash('sha256').update(codeVerifier).digest('base64url');
const url = new URL(doc.authorization_endpoint);
url.searchParams.set('response_type', 'code');
url.searchParams.set('client_id', this.clientId);
url.searchParams.set('redirect_uri', this.redirectUri);
url.searchParams.set('scope', this.config.env.OIDC_SCOPES);
url.searchParams.set('state', state);
url.searchParams.set('nonce', nonce);
url.searchParams.set('code_challenge', challenge);
url.searchParams.set('code_challenge_method', 'S256');
const now = Math.floor(Date.now() / 1000);
const claims: OidcStateClaims = { state, nonce, codeVerifier };
if (linkUserId) claims.linkUserId = linkUserId;
const stateToken = await new SignJWT({ ...claims })
.setProtectedHeader({ alg: 'HS256', typ: 'JWT' })
.setIssuedAt(now)
.setExpirationTime(now + STATE_TTL_SECONDS)
.sign(deriveTokenKey(this.config.env.COLLAB_TOKEN_SECRET, 'oidc-state'));
return { url: url.toString(), stateToken };
}
private async verifyStateToken(stateToken: string): Promise<OidcStateClaims> {
try {
const { payload } = await jwtVerify(
stateToken,
deriveTokenKey(this.config.env.COLLAB_TOKEN_SECRET, 'oidc-state'),
{ algorithms: ['HS256'] },
);
if (typeof payload.state !== 'string' || typeof payload.nonce !== 'string') throw new Error();
if (typeof payload.codeVerifier !== 'string') throw new Error();
return payload as OidcStateClaims;
} catch {
throw new BadRequestException({ code: 'oidc_state_invalid' });
}
}
/**
* The callback half: state check, code exchange, ID-token validation
* (signature via JWKS, issuer, audience, expiry and the nonce binding),
* then identity resolution. Returns the session token to set plus where
* the SPA should land.
*/
async completeLogin(
code: string,
state: string,
stateToken: string,
userAgent: string | undefined,
): Promise<{ sessionToken: string | null; linked: boolean }> {
const doc = await this.discover();
const stored = await this.verifyStateToken(stateToken);
if (state !== stored.state) {
throw new BadRequestException({ code: 'oidc_state_invalid' });
}
const body = new URLSearchParams({
grant_type: 'authorization_code',
code,
redirect_uri: this.redirectUri,
client_id: this.clientId,
code_verifier: stored.codeVerifier,
});
// Confidential client: secret via client_secret_post (Keycloak default
// accepts it); a public client authenticates with PKCE alone.
if (this.config.env.OIDC_CLIENT_SECRET) {
body.set('client_secret', this.config.env.OIDC_CLIENT_SECRET);
}
const tokenResponse = await fetch(doc.token_endpoint, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body,
}).catch(() => null);
if (!tokenResponse?.ok) {
this.logger.warn({ status: tokenResponse?.status }, 'oidc: code exchange failed');
throw new BadRequestException({ code: 'oidc_exchange_failed' });
}
const tokens = (await tokenResponse.json()) as { id_token?: string };
if (!tokens.id_token) throw new BadRequestException({ code: 'oidc_exchange_failed' });
let payload: JWTPayload;
try {
({ payload } = await jwtVerify(tokens.id_token, this.jwks!, {
issuer: this.issuer,
audience: this.clientId,
algorithms: ID_TOKEN_ALGORITHMS,
}));
} catch (error) {
this.logger.warn({ err: error }, 'oidc: id token rejected');
throw new BadRequestException({ code: 'oidc_token_invalid' });
}
if (typeof payload.nonce !== 'string' || payload.nonce !== stored.nonce) {
throw new BadRequestException({ code: 'oidc_token_invalid' });
}
if (typeof payload.sub !== 'string' || payload.sub.length === 0) {
throw new BadRequestException({ code: 'oidc_token_invalid' });
}
if (stored.linkUserId) {
await this.linkIdentity(stored.linkUserId, payload.sub);
return { sessionToken: null, linked: true };
}
const user = await this.resolveUser(payload);
if (user.status === 'DISABLED') {
throw new BadRequestException({ code: 'account_disabled' });
}
// Claim mapping (issue #217): reconcile mapped grants and the managed
// site-admin flag against this login's fresh claims — before the
// session exists, so the first request already sees the new state.
await this.claimMapping.apply(user, payload);
const sessionToken = await this.sessions.create(user.id, userAgent);
await this.prisma.user.update({ where: { id: user.id }, data: { lastLoginAt: new Date() } });
await this.audit.record({
action: 'auth.login_succeeded',
actorId: user.id,
details: { provider: this.provider },
});
return { sessionToken, linked: false };
}
/** The deliberate linking rule (ADR 0021 §2): only an authenticated user
* links an IdP identity to their own account never automatic by mail. */
private async linkIdentity(userId: string, subject: string): Promise<void> {
const existing = await this.prisma.userIdentity.findUnique({
where: { provider_subject: { provider: this.provider, subject } },
});
if (existing && existing.userId !== userId) {
throw new ConflictException({ code: 'oidc_identity_taken' });
}
if (!existing) {
await this.prisma.userIdentity.create({
data: { userId, provider: this.provider, subject },
});
await this.audit.record({
action: 'auth.identity_linked',
actorId: userId,
details: { provider: this.provider },
});
}
}
private async resolveUser(payload: JWTPayload): Promise<User> {
const identity = await this.prisma.userIdentity.findUnique({
where: { provider_subject: { provider: this.provider, subject: payload.sub! } },
});
if (identity) {
const user = await this.users.findById(identity.userId);
if (!user) throw new BadRequestException({ code: 'oidc_token_invalid' });
return user;
}
// First login of this subject: just-in-time creation. The IdP owns the
// account lifecycle (ADR 0021), so the account arrives ACTIVE and
// mail-verified — provided the IdP says the address is verified.
const email = typeof payload.email === 'string' ? payload.email.toLowerCase() : null;
if (!email) throw new BadRequestException({ code: 'oidc_email_missing' });
if (payload.email_verified === false) {
throw new BadRequestException({ code: 'oidc_email_unverified' });
}
const clash = await this.users.findByEmail(email);
if (clash) {
// The documented refusal: the local owner of this address must link
// explicitly (GET /auth/oidc/link) — silent adoption would be an
// account-takeover path (ADR 0021 §2).
throw new ConflictException({ code: 'oidc_link_required' });
}
const preferred =
typeof payload.preferred_username === 'string' && payload.preferred_username
? payload.preferred_username
: email.split('@')[0]!;
const displayName =
typeof payload.name === 'string' && payload.name.trim() ? payload.name.trim() : preferred;
const username = await this.uniqueUsername(slugify(preferred) || 'user');
const user = await this.prisma.$transaction(async (tx) => {
const created = await tx.user.create({
data: {
username,
email,
displayName,
locale: 'en',
status: 'ACTIVE',
emailVerifiedAt: new Date(),
},
});
await tx.userIdentity.create({
data: { userId: created.id, provider: this.provider, subject: payload.sub! },
});
return created;
});
// Same invariant as e-mail verification: every active account owns a
// personal pond (idempotent).
await this.ponds.ensurePersonalPond(user);
await this.audit.record({
action: 'auth.signup',
actorId: user.id,
details: { provider: this.provider },
});
return user;
}
private async uniqueUsername(base: string): Promise<string> {
const taken = new Set(
(
await this.prisma.user.findMany({
where: { OR: [{ username: base }, { username: { startsWith: `${base}-` } }] },
select: { username: true },
})
).map((row) => row.username),
);
if (!taken.has(base)) return base;
for (let n = 2; ; n += 1) {
const candidate = `${base}-${n}`;
if (!taken.has(candidate)) return candidate;
}
}
}

View File

@ -0,0 +1,157 @@
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
const HEADER = 'x-auth-user';
/**
* Trusted reverse-proxy authentication (issue #215, ADR 0021): off by
* default (header fully ignored), identity only from a trusted TCP peer, a
* spoofing peer rejected AND audited, no privilege escalation past a
* riding-along session cookie, and the mTLS variant mapping a forwarded
* certificate DN attribute.
*/
describe.skipIf(!hasTestDb)('trusted-proxy identity (e2e, issue #215)', () => {
let prisma: PrismaClient;
const suffix = uniqueSuffix();
const password = 'proxy identitaet 123';
const PROXY_ENV = ['AUTH_PROXY_HEADER', 'AUTH_PROXY_TRUSTED_PEERS', 'AUTH_PROXY_MODE'] as const;
async function bootApp(env: Partial<Record<(typeof PROXY_ENV)[number], string>>) {
for (const key of PROXY_ENV) delete process.env[key];
Object.assign(process.env, env);
return createTestApp();
}
async function makeUser(app: INestApplication, handle: string) {
const users = app.get(UsersService);
const user = await users.createUser({
username: `${handle}-${suffix}`,
email: `${handle}-${suffix}@example.test`,
displayName: handle,
password,
locale: 'en',
});
await users.markEmailVerified(user.id);
return user;
}
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
});
afterAll(async () => {
for (const key of PROXY_ENV) delete process.env[key];
await prisma.auditEntry.deleteMany({ where: { action: 'auth.proxy_rejected' } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
});
it('ignores the header entirely while the feature is off', async () => {
const app = await bootApp({});
try {
await makeUser(app, 'off');
await request(app.getHttpServer())
.get('/api/v1/auth/me')
.set(HEADER, `off-${suffix}`)
.expect(401);
} finally {
await app.close();
}
});
it('authenticates a trusted peer, maps by username, and never escalates past a session cookie', async () => {
const app = await bootApp({
AUTH_PROXY_HEADER: HEADER,
AUTH_PROXY_TRUSTED_PEERS: '127.0.0.1',
});
try {
const alice = await makeUser(app, 'alice');
const bob = await makeUser(app, 'bob');
const api = () => request(app.getHttpServer());
const me = await api().get('/api/v1/auth/me').set(HEADER, alice.username).expect(200);
expect(me.body.id).toBe(alice.id);
// Unknown identity: authenticated by nobody.
await api().get('/api/v1/auth/me').set(HEADER, `ghost-${suffix}`).expect(401);
// A session cookie riding along never escalates beyond the header
// identity: bob's cookie plus alice's header acts as alice.
const login = await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: bob.username, password })
.expect(200);
const both = await api()
.get('/api/v1/auth/me')
.set('Cookie', sessionCookieOf(login))
.set(HEADER, alice.username)
.expect(200);
expect(both.body.id).toBe(alice.id);
// Without the header the same cookie still works normally.
const cookieOnly = await api()
.get('/api/v1/auth/me')
.set('Cookie', sessionCookieOf(login))
.expect(200);
expect(cookieOnly.body.id).toBe(bob.id);
} finally {
await app.close();
}
});
it('rejects and audits the header from an untrusted peer — even with a valid session', async () => {
const app = await bootApp({
AUTH_PROXY_HEADER: HEADER,
AUTH_PROXY_TRUSTED_PEERS: '203.0.113.9',
});
try {
const carol = await makeUser(app, 'carol');
const api = () => request(app.getHttpServer());
await api().get('/api/v1/auth/me').set(HEADER, carol.username).expect(403);
const audit = await prisma.auditEntry.findFirst({
where: { action: 'auth.proxy_rejected' },
orderBy: { at: 'desc' },
});
expect(audit?.details).toMatchObject({ header: HEADER });
const login = await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: carol.username, password })
.expect(200);
// The spoofed header poisons the request even alongside a valid
// cookie — rejecting is safer than guessing which identity wins.
await api()
.get('/api/v1/auth/me')
.set('Cookie', sessionCookieOf(login))
.set(HEADER, carol.username)
.expect(403);
} finally {
await app.close();
}
});
it('maps the configured DN attribute in mtls-dn mode', async () => {
const app = await bootApp({
AUTH_PROXY_HEADER: HEADER,
AUTH_PROXY_TRUSTED_PEERS: '127.0.0.1',
AUTH_PROXY_MODE: 'mtls-dn',
});
try {
const dana = await makeUser(app, 'dana');
const me = await request(app.getHttpServer())
.get('/api/v1/auth/me')
.set(HEADER, `CN=${dana.username},OU=unit,O=example`)
.expect(200);
expect(me.body.id).toBe(dana.id);
} finally {
await app.close();
}
});
});

View File

@ -0,0 +1,102 @@
import { ForbiddenException, Injectable, UnauthorizedException } from '@nestjs/common';
import { User } from '@prisma/client';
import { PinoLogger } from 'nestjs-pino';
import { AuditService } from '../audit/audit.service';
import { AppConfig } from '../config/app-config.service';
import { UsersService } from '../users/users.service';
import type { AuthedRequest } from './auth.guard';
/**
* Trusted reverse-proxy authentication (issue #215, ADR 0021): the
* perimeter (proxy or mTLS terminator) authenticates and forwards the
* identity in a configured header; the application trusts that header ONLY
* when the request's TCP peer is on the configured allowlist.
*
* The trust boundary, stated plainly (security.md §External
* authentication): everything upstream of the configured peers is the
* operator's responsibility; the application's contribution is that the
* header is worthless from anywhere else a header from an untrusted peer
* rejects the request outright and lands in the audit trail
* (`auth.proxy_rejected`), because someone is attempting a spoof.
*
* Deliberately NO just-in-time creation here: the header carries no
* verified e-mail, so accounts must already exist (the IdP/OIDC path or an
* admin creates them) and are mapped by username or e-mail explicit
* configuration, never guessed.
*/
@Injectable()
export class ProxyIdentityService {
constructor(
private readonly users: UsersService,
private readonly audit: AuditService,
private readonly config: AppConfig,
private readonly logger: PinoLogger,
) {
this.logger.setContext(ProxyIdentityService.name);
}
/** Enabled only with BOTH the header name and a non-empty allowlist. */
get enabled(): boolean {
return Boolean(
this.config.env.AUTH_PROXY_HEADER && this.config.env.AUTH_PROXY_TRUSTED_PEERS.length > 0,
);
}
/**
* Resolves the request's proxy identity, or null when the feature is off
* or the header is absent. Throws 403 (audited) for an untrusted peer
* carrying the header, 401 for an unknown identity.
*/
async resolve(request: AuthedRequest): Promise<User | null> {
if (!this.enabled) return null;
const headerName = this.config.env.AUTH_PROXY_HEADER!.toLowerCase();
const raw = request.headers[headerName];
const value = Array.isArray(raw) ? raw[0] : raw;
if (!value) return null;
const peer = normalizePeer(request.socket.remoteAddress ?? '');
const trusted = this.config.env.AUTH_PROXY_TRUSTED_PEERS.map(normalizePeer);
if (!trusted.includes(peer)) {
// A spoof attempt, not a misconfiguration: reject and evidence it.
await this.audit.record({
action: 'auth.proxy_rejected',
details: { peer, header: headerName },
});
throw new ForbiddenException({ code: 'proxy_peer_untrusted' });
}
const identity = this.extractIdentity(value);
if (!identity) throw new UnauthorizedException({ code: 'proxy_identity_unknown' });
const user =
this.config.env.AUTH_PROXY_MAP === 'email'
? await this.users.findByEmail(identity)
: await this.users.findByUsernameOrEmail(identity);
if (!user || user.status !== 'ACTIVE') {
throw new UnauthorizedException({ code: 'proxy_identity_unknown' });
}
return user;
}
/** `plain`: the value is the identity. `mtls-dn`: the value is a client
* certificate subject DN as forwarded by the TLS terminator; the identity
* is the configured attribute (default CN). */
private extractIdentity(value: string): string | null {
if (this.config.env.AUTH_PROXY_MODE === 'plain') return value.trim() || null;
const attribute = this.config.env.AUTH_PROXY_DN_ATTRIBUTE.toLowerCase();
for (const part of value.split(/[,/]/)) {
const [key, ...rest] = part.split('=');
if (key?.trim().toLowerCase() === attribute) {
const extracted = rest.join('=').trim();
return extracted || null;
}
}
return null;
}
}
/** `::ffff:127.0.0.1` and `127.0.0.1` are the same peer. */
function normalizePeer(address: string): string {
return address.replace(/^::ffff:/i, '').trim();
}

View File

@ -0,0 +1,50 @@
import { mkdir, readFile, rm, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import { Injectable } from '@nestjs/common';
import { AppConfig } from '../config/app-config.service';
/**
* Filesystem binding for branding assets (issue #306; pond overrides #307).
*
* One flat directory of PNGs named by a caller-supplied key
* (`instance-logo-light`, later `pond-<id>-favicon-32`). Flat because there
* are a handful of files per instance and the backup archives the directory
* as a whole a tree would buy nothing and cost a traversal question.
*
* The key is constrained here rather than trusted from the route: it is the
* only thing between a request parameter and a path.
*/
@Injectable()
export class BrandingStorageService {
constructor(private readonly config: AppConfig) {}
/** Lowercase, digits and dashes only no dot, so no `..`, and no slash,
* so the file cannot leave the directory whatever a caller sends. */
private pathFor(key: string): string {
if (!/^[a-z0-9-]{1,120}$/.test(key)) throw new Error(`invalid branding key: ${key}`);
return join(this.config.env.BRANDING_DIR, `${key}.png`);
}
async save(key: string, bytes: Buffer): Promise<void> {
await mkdir(this.config.env.BRANDING_DIR, { recursive: true });
await writeFile(this.pathFor(key), bytes);
}
/** The bytes, or null when the file is absent a missing asset is a normal
* state here (nothing uploaded, or metadata and disk drifted after a
* partial restore), and every caller has a fallback. */
async read(key: string): Promise<Buffer | null> {
try {
return await readFile(this.pathFor(key));
} catch {
return null;
}
}
/** Idempotent: removing what is not there is success. */
async remove(key: string): Promise<void> {
await rm(this.pathFor(key), { force: true });
}
}

View File

@ -0,0 +1,253 @@
import {
BadRequestException,
Controller,
Delete,
Get,
NotFoundException,
Param,
Post,
Query,
Req,
Res,
UploadedFiles,
UseGuards,
UseInterceptors,
} from '@nestjs/common';
import { AnyFilesInterceptor } from '@nestjs/platform-express';
import {
BrandingView,
FAVICON_SIZES,
FaviconSize,
LOGO_VARIANTS,
LogoVariant,
MAX_BRANDING_BYTES,
PondBranding,
} from '@dorfteich/shared';
import type { Response } from 'express';
import { SiteAdminGuard } from '../admin/site-admin.guard';
import { AuthedRequest, Public } from '../auth/auth.guard';
import { RequiresPondRole } from '../permissions/permission.decorators';
import { PrismaService } from '../prisma/prisma.service';
import { BrandingService } from './branding.service';
function parseVariant(value: unknown): LogoVariant {
if (!LOGO_VARIANTS.includes(value as LogoVariant)) {
throw new BadRequestException({ code: 'bad_request' });
}
return value as LogoVariant;
}
/**
* Public branding surface (issue #306).
*
* Unauthenticated by design and worth stating plainly in the admin UI: the
* login screen carries the branding and the browser fetches the favicon before
* anyone signs in, so an operator's logo IS visible to anonymous visitors.
*/
@Controller('branding')
export class BrandingController {
constructor(private readonly branding: BrandingService) {}
@Public()
@Get()
view(): Promise<BrandingView> {
return this.branding.view();
}
@Public()
@Get('logo')
async logo(
@Query('variant') variant: string | undefined,
@Query('pond') pondId: string | undefined,
@Res() res: Response,
): Promise<void> {
const wanted = parseVariant(variant ?? 'light');
// A pond scope serves the pond's own bytes and nothing else: the caller
// already resolved WHICH level applies (`resolveBranding`), so silently
// falling back here would mix variants across levels — exactly what #307
// forbids.
const bytes = pondId
? await this.branding.pondLogoBytes(pondId, wanted)
: await this.branding.logoBytes(wanted);
// No shipped default: without a logo the app renders the instance NAME as
// text, so an empty answer here is the honest one.
if (!bytes) {
res.status(404).json({ code: 'not_found', message: 'no logo' });
return;
}
res.setHeader('Content-Type', 'image/png');
// The caller puts the content hash in the query string, so a given URL
// never changes what it points at.
res.setHeader('Cache-Control', 'public, max-age=31536000, immutable');
res.send(bytes);
}
@Public()
@Get('favicon')
async favicon(
@Query('size') size: string | undefined,
@Query('pond') pondId: string | undefined,
@Res() res: Response,
): Promise<void> {
const wanted = Number(size ?? 32);
if (!(FAVICON_SIZES as readonly number[]).includes(wanted)) {
throw new BadRequestException({ code: 'bad_request' });
}
const pondBytes = pondId
? await this.branding.pondFaviconBytes(pondId, wanted as FaviconSize)
: null;
const { bytes, uploaded } = pondBytes
? { bytes: pondBytes, uploaded: true }
: await this.branding.faviconBytes(wanted as FaviconSize);
res.setHeader('Content-Type', 'image/png');
// The `<link rel="icon">` href is a constant in index.html, so this URL
// cannot carry a hash — revalidation is the only way a replaced favicon
// ever reaches a browser that already has one.
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('ETag', `"${uploaded ? 'custom' : 'default'}-${bytes.length}"`);
res.send(bytes);
}
}
/** Site-Admin management of the instance branding (issue #306). */
@Controller('admin/branding')
@UseGuards(SiteAdminGuard)
export class BrandingAdminController {
constructor(private readonly branding: BrandingService) {}
@Post('logo')
@UseInterceptors(AnyFilesInterceptor({ limits: { fileSize: MAX_BRANDING_BYTES } }))
async setLogo(
@Query('variant') variant: string | undefined,
@Req() request: AuthedRequest,
@UploadedFiles() files: Express.Multer.File[] | undefined,
): Promise<BrandingView> {
const file = files?.find((entry) => entry.fieldname === 'file');
if (!file) throw new BadRequestException({ code: 'branding_file_missing' });
return this.branding.setLogo(request.user!, parseVariant(variant ?? 'light'), file.buffer);
}
@Delete('logo')
clearLogo(
@Query('variant') variant: string | undefined,
@Req() request: AuthedRequest,
): Promise<BrandingView> {
return this.branding.clearLogo(request.user!, parseVariant(variant ?? 'light'));
}
@Post('favicon')
@UseInterceptors(AnyFilesInterceptor({ limits: { fileSize: MAX_BRANDING_BYTES } }))
async setFavicon(
@Req() request: AuthedRequest,
@UploadedFiles() files: Express.Multer.File[] | undefined,
): Promise<BrandingView> {
// Field names are the pixel sizes the browser rendered: `png-32`, `png-180`.
const byField = new Map((files ?? []).map((file) => [file.fieldname, file.buffer]));
const collected = {} as Record<FaviconSize, Buffer>;
for (const size of FAVICON_SIZES) {
const bytes = byField.get(`png-${size}`);
if (!bytes) throw new BadRequestException({ code: 'branding_file_missing' });
collected[size] = bytes;
}
return this.branding.setFavicon(request.user!, collected);
}
@Delete('favicon')
clearFavicon(@Req() request: AuthedRequest): Promise<BrandingView> {
return this.branding.clearFavicon(request.user!);
}
}
/**
* Pond-level branding (issue #307). The uploader here is an ordinary Pond
* Admin rather than the operator, so the security rules of #306 are not
* relaxed by a single line: SVG refused, magic bytes checked server-side,
* size caps enforced, content type pinned on serving, no image parsing.
*
* 404/403 policy: a user who cannot see the pond gets 404 from the pond-role
* guard, one who can see but not administer it gets 403.
*/
@Controller('ponds/:pondId/branding')
export class PondBrandingController {
constructor(
private readonly branding: BrandingService,
private readonly prisma: PrismaService,
) {}
/** The pond row the quota is charged to. */
private async pondOf(pondId: string): Promise<{ id: string; ownerId: string }> {
const pond = await this.prisma.pond.findUnique({
where: { id: pondId },
select: { id: true, ownerId: true },
});
if (!pond) throw new NotFoundException();
return pond;
}
@Get()
@RequiresPondRole('reader', { idParam: 'pondId' })
view(@Param('pondId') pondId: string): Promise<PondBranding> {
return this.branding.pondBranding(pondId);
}
@Post('logo')
@RequiresPondRole('pond_admin', { idParam: 'pondId' })
@UseInterceptors(AnyFilesInterceptor({ limits: { fileSize: MAX_BRANDING_BYTES } }))
async setLogo(
@Param('pondId') pondId: string,
@Query('variant') variant: string | undefined,
@Req() request: AuthedRequest,
@UploadedFiles() files: Express.Multer.File[] | undefined,
): Promise<PondBranding> {
const file = files?.find((entry) => entry.fieldname === 'file');
if (!file) throw new BadRequestException({ code: 'branding_file_missing' });
return this.branding.setPondLogo(
request.user!,
await this.pondOf(pondId),
parseVariant(variant ?? 'light'),
file.buffer,
);
}
@Delete('logo')
@RequiresPondRole('pond_admin', { idParam: 'pondId' })
async clearLogo(
@Param('pondId') pondId: string,
@Query('variant') variant: string | undefined,
@Req() request: AuthedRequest,
): Promise<PondBranding> {
return this.branding.clearPondLogo(
request.user!,
await this.pondOf(pondId),
parseVariant(variant ?? 'light'),
);
}
@Post('favicon')
@RequiresPondRole('pond_admin', { idParam: 'pondId' })
@UseInterceptors(AnyFilesInterceptor({ limits: { fileSize: MAX_BRANDING_BYTES } }))
async setFavicon(
@Param('pondId') pondId: string,
@Req() request: AuthedRequest,
@UploadedFiles() files: Express.Multer.File[] | undefined,
): Promise<PondBranding> {
const byField = new Map((files ?? []).map((file) => [file.fieldname, file.buffer]));
const collected = {} as Record<FaviconSize, Buffer>;
for (const size of FAVICON_SIZES) {
const bytes = byField.get(`png-${size}`);
if (!bytes) throw new BadRequestException({ code: 'branding_file_missing' });
collected[size] = bytes;
}
return this.branding.setPondFavicon(request.user!, await this.pondOf(pondId), collected);
}
@Delete('favicon')
@RequiresPondRole('pond_admin', { idParam: 'pondId' })
async clearFavicon(
@Param('pondId') pondId: string,
@Req() request: AuthedRequest,
): Promise<PondBranding> {
return this.branding.clearPondFavicon(request.user!, await this.pondOf(pondId));
}
}

View File

@ -0,0 +1,250 @@
import { mkdtemp, readFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
/**
* A real PNG of `size`×`size`, built the same way the shipped default is
* the api reads the IHDR, so the header has to be genuine.
*/
async function png(size: number): Promise<Buffer> {
const { deflateSync } = await import('node:zlib');
const crcTable = Array.from({ length: 256 }, (_, n) => {
let c = n;
for (let k = 0; k < 8; k += 1) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
return c >>> 0;
});
const crc32 = (buf: Buffer): number => {
let c = 0xffffffff;
for (const byte of buf) c = crcTable[(c ^ byte) & 0xff]! ^ (c >>> 8);
return (c ^ 0xffffffff) >>> 0;
};
const chunk = (type: string, data: Buffer): Buffer => {
const length = Buffer.alloc(4);
length.writeUInt32BE(data.length);
const body = Buffer.concat([Buffer.from(type, 'ascii'), data]);
const crc = Buffer.alloc(4);
crc.writeUInt32BE(crc32(body));
return Buffer.concat([length, body, crc]);
};
const ihdr = Buffer.alloc(13);
ihdr.writeUInt32BE(size, 0);
ihdr.writeUInt32BE(size, 4);
ihdr[8] = 8;
ihdr[9] = 6;
const raw = Buffer.alloc(size * (size * 4 + 1));
return Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
chunk('IHDR', ihdr),
chunk('IDAT', deflateSync(raw)),
chunk('IEND', Buffer.alloc(0)),
]);
}
describe.skipIf(!hasTestDb)('instance branding (e2e, issue #306)', () => {
let app: INestApplication;
let prisma: PrismaClient;
let brandingDir: string;
const suffix = uniqueSuffix();
const password = 'markenzeichen mit teich 1';
const admin = { username: `ba-${suffix}` };
const plain = { username: `bp-${suffix}` };
let adminCookie: string;
let plainCookie: string;
const api = () => request(app.getHttpServer());
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
// A real directory: the point is that bytes land somewhere and come back.
brandingDir = await mkdtemp(join(tmpdir(), 'dorfteich-branding-'));
process.env.BRANDING_DIR = brandingDir;
app = await createTestApp();
const users = app.get(UsersService);
const adminUser = await users.createUser({
username: admin.username,
email: `${admin.username}@example.org`,
displayName: `Branding Admin ${suffix}`,
password,
locale: 'en',
});
await users.markEmailVerified(adminUser.id);
await prisma.user.update({ where: { id: adminUser.id }, data: { isSiteAdmin: true } });
const plainUser = await users.createUser({
username: plain.username,
email: `${plain.username}@example.org`,
displayName: `Branding Plain ${suffix}`,
password,
locale: 'en',
});
await users.markEmailVerified(plainUser.id);
const login = async (username: string): Promise<string> =>
sessionCookieOf(
await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: username, password })
.expect(200),
);
adminCookie = await login(admin.username);
plainCookie = await login(plain.username);
});
afterAll(async () => {
await prisma.instanceSetting.deleteMany({
where: { key: { in: ['instance.logo', 'instance.logoDark', 'instance.favicon'] } },
});
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();
await rm(brandingDir, { recursive: true, force: true });
delete process.env.BRANDING_DIR;
});
it('serves the shipped default favicon before anything is uploaded', async () => {
// The `<link rel="icon">` in index.html is a constant — this route must
// never 404, or the browser keeps its generic icon for good.
const res = await api().get('/api/v1/branding/favicon').expect(200);
expect(res.headers['content-type']).toContain('image/png');
expect(res.body.subarray(0, 8).toString('latin1')).toContain('PNG');
});
it('stores a logo, reports it, and serves the bytes without a session', async () => {
const bytes = await png(64);
const view = await api()
.post('/api/v1/admin/branding/logo?variant=light')
.set('Cookie', adminCookie)
.attach('file', bytes, 'logo.png')
.expect(201);
expect(view.body.logo).toMatchObject({ width: 64, height: 64 });
expect(view.body.logoDark).toBeNull();
// On disk, under the key the pond override (#307) will extend.
const onDisk = await readFile(join(brandingDir, 'instance-logo-light.png'));
expect(onDisk.length).toBe(bytes.length);
// Anonymous: the login screen carries the branding.
const served = await api().get('/api/v1/branding/logo?variant=light').expect(200);
expect(served.headers['content-type']).toContain('image/png');
const anon = await api().get('/api/v1/branding').expect(200);
expect(anon.body.logo.hash).toBe(view.body.logo.hash);
expect(anon.body.instanceName).toBeTruthy();
});
it('answers 404 for a logo variant that was never uploaded', async () => {
// No shipped default for the logo: without one the app renders the
// instance NAME, so an empty answer is the honest one.
await api().get('/api/v1/branding/logo?variant=dark').expect(404);
});
it('rejects an SVG with its own message, not a generic one', async () => {
const res = await api()
.post('/api/v1/admin/branding/logo?variant=light')
.set('Cookie', adminCookie)
.attach('file', Buffer.from('<?xml version="1.0"?><svg xmlns="..."><script/></svg>'), 'x.png')
.expect(400);
expect(res.body.code).toBe('branding_svg_rejected');
});
it('rejects bytes that are not a PNG at all', async () => {
const res = await api()
.post('/api/v1/admin/branding/logo?variant=light')
.set('Cookie', adminCookie)
.attach('file', Buffer.from('GIF89a and then some'), 'x.png')
.expect(400);
expect(res.body.code).toBe('branding_not_a_png');
});
it('rejects a logo larger than the maximum edge', async () => {
const res = await api()
.post('/api/v1/admin/branding/logo?variant=light')
.set('Cookie', adminCookie)
.attach('file', await png(600), 'x.png')
.expect(400);
expect(res.body.code).toBe('branding_image_too_large');
});
it('takes both favicon sizes together and serves each back', async () => {
await api()
.post('/api/v1/admin/branding/favicon')
.set('Cookie', adminCookie)
.attach('png-32', await png(32), 'f32.png')
.attach('png-180', await png(180), 'f180.png')
.expect(201);
for (const size of [32, 180]) {
const res = await api().get(`/api/v1/branding/favicon?size=${size}`).expect(200);
expect(res.body.length).toBe((await png(size)).length);
}
});
it('refuses a favicon whose bytes do not match the size they claim', async () => {
const res = await api()
.post('/api/v1/admin/branding/favicon')
.set('Cookie', adminCookie)
.attach('png-32', await png(64), 'f32.png')
.attach('png-180', await png(180), 'f180.png')
.expect(400);
expect(res.body.code).toBe('branding_favicon_not_square');
});
it('clears an asset and falls back again', async () => {
await api().delete('/api/v1/admin/branding/favicon').set('Cookie', adminCookie).expect(200);
const view = await api().get('/api/v1/branding').expect(200);
expect(view.body.favicon).toBeNull();
// Back to the shipped default rather than a 404.
await api().get('/api/v1/branding/favicon').expect(200);
await api()
.delete('/api/v1/admin/branding/logo?variant=light')
.set('Cookie', adminCookie)
.expect(200);
await api().get('/api/v1/branding/logo?variant=light').expect(404);
});
it('keeps management away from a non-admin, but not reading', async () => {
await api()
.post('/api/v1/admin/branding/logo?variant=light')
.set('Cookie', plainCookie)
.attach('file', await png(32), 'x.png')
.expect(403);
await api().delete('/api/v1/admin/branding/favicon').set('Cookie', plainCookie).expect(403);
await api().get('/api/v1/branding').set('Cookie', plainCookie).expect(200);
});
it('audits every branding change with scope, asset and direction', async () => {
await api()
.post('/api/v1/admin/branding/logo?variant=dark')
.set('Cookie', adminCookie)
.attach('file', await png(48), 'logo.png')
.expect(201);
const entry = await prisma.auditEntry.findFirst({
where: { action: 'branding.changed', targetId: 'instance.logoDark' },
orderBy: { at: 'desc' },
});
expect(entry).not.toBeNull();
expect(entry!.details).toMatchObject({ scope: 'instance', asset: 'logoDark', change: 'set' });
});
it('refuses to write branding metadata through the settings endpoint', async () => {
// The metadata describes bytes on disk; hand-writing it would claim an
// asset that is not there, so the settings PATCH does not accept it.
const res = await api()
.patch('/api/v1/admin/settings')
.set('Cookie', adminCookie)
.send({ 'instance.logo': { hash: 'deadbeefdeadbeef', width: 10, height: 10 } })
.expect(400);
expect(res.body.code).toBe('bad_request');
});
});

View File

@ -0,0 +1,23 @@
import { Module } from '@nestjs/common';
import { PermissionsModule } from '../permissions/permissions.module';
import { QuotasModule } from '../quotas/quotas.module';
import {
BrandingAdminController,
BrandingController,
PondBrandingController,
} from './branding.controller';
import { BrandingStorageService } from './branding-storage.service';
import { BrandingService } from './branding.service';
/** Instance branding logo and favicon (issue #306). Exports the services so
* the pond-level override (#307) can build on the same storage and the same
* resolution path instead of a parallel one. */
@Module({
imports: [PermissionsModule, QuotasModule],
controllers: [BrandingController, BrandingAdminController, PondBrandingController],
providers: [BrandingService, BrandingStorageService],
exports: [BrandingService, BrandingStorageService],
})
export class BrandingModule {}

View File

@ -0,0 +1,380 @@
import { createHash } from 'node:crypto';
import { readFile } from 'node:fs/promises';
import { join } from 'node:path';
import { BadRequestException, Injectable, NotFoundException } from '@nestjs/common';
import {
BrandingAsset,
BrandingView,
FAVICON_SIZES,
FaviconSize,
LOGO_VARIANTS,
LogoVariant,
PondBranding,
pondSettingsSchema,
MAX_BRANDING_BYTES,
MAX_LOGO_EDGE,
hasPngMagic,
looksLikeSvg,
pngDimensions,
} from '@dorfteich/shared';
import { User } from '@prisma/client';
import { AuditService } from '../audit/audit.service';
import { PrismaService } from '../prisma/prisma.service';
import { QuotaService } from '../quotas/quota.service';
import { InstanceSettingsService } from '../settings/instance-settings.service';
import { BrandingStorageService } from './branding-storage.service';
/** The settings key each instance asset's metadata lives under. */
const INSTANCE_KEYS = {
logoLight: 'instance.logo',
logoDark: 'instance.logoDark',
favicon: 'instance.favicon',
} as const;
/**
* Instance branding (issue #306): the logo shown at the top of the sidebar and
* the favicon served to the browser.
*
* The api stores and serves bytes; it never decodes them. Validation is the
* PNG signature, the IHDR dimensions and the size cap see
* `packages/shared/src/branding.ts` for why that line is drawn there.
*/
@Injectable()
export class BrandingService {
constructor(
private readonly settings: InstanceSettingsService,
private readonly storage: BrandingStorageService,
private readonly audit: AuditService,
private readonly prisma: PrismaService,
private readonly quotas: QuotaService,
) {}
static logoKey(variant: LogoVariant): string {
return `instance-logo-${variant}`;
}
static faviconKey(size: FaviconSize): string {
return `instance-favicon-${size}`;
}
/** Pond assets share the directory and the naming rules (issue #307); the
* pond id keeps them apart and makes purge a prefix delete. */
static pondLogoKey(pondId: string, variant: LogoVariant): string {
return `pond-${pondId}-logo-${variant}`;
}
static pondFaviconKey(pondId: string, size: FaviconSize): string {
return `pond-${pondId}-favicon-${size}`;
}
/** Every branding file a pond can own the purge deletes exactly this set
* (issue #307). The purge standard is absolute: after it, nothing
* referencing the pond survives, rows or files. */
static pondKeys(pondId: string): string[] {
return [
...LOGO_VARIANTS.map((variant) => BrandingService.pondLogoKey(pondId, variant)),
...FAVICON_SIZES.map((size) => BrandingService.pondFaviconKey(pondId, size)),
];
}
/**
* Rejects anything that is not a PNG within the caps, before a byte is
* written. SVG gets its own message: an operator who tried one deserves to
* learn that it is refused on purpose, not that "the file is broken".
*/
private assertUsablePng(bytes: Buffer, maxEdge: number): { width: number; height: number } {
if (bytes.length === 0) throw new BadRequestException({ code: 'branding_file_empty' });
if (bytes.length > MAX_BRANDING_BYTES) {
throw new BadRequestException({ code: 'branding_file_too_large' });
}
if (looksLikeSvg(bytes)) throw new BadRequestException({ code: 'branding_svg_rejected' });
if (!hasPngMagic(bytes)) throw new BadRequestException({ code: 'branding_not_a_png' });
const size = pngDimensions(bytes);
if (!size) throw new BadRequestException({ code: 'branding_not_a_png' });
if (size.width > maxEdge || size.height > maxEdge) {
throw new BadRequestException({ code: 'branding_image_too_large' });
}
return size;
}
/**
* Reserve the pond's storage for a branding asset, releasing what the asset
* it replaces occupied. Doing it in that order means replacing a logo with
* one of the same size costs nothing otherwise every re-upload would eat
* the quota again, which is how "a pond admin fills the disk with logos"
* happens.
*/
private async chargeQuota(
pond: { id: string; ownerId: string },
bytes: number,
previous: BrandingAsset | null,
): Promise<void> {
if (previous?.byteSize) await this.quotas.release(pond.id, previous.byteSize);
try {
await this.quotas.checkAndConsume(pond.id, pond.ownerId, bytes);
} catch (error) {
// Put the released reservation back: a refused upload must not leave
// the pond with MORE room than before.
if (previous?.byteSize) {
await this.quotas.checkAndConsume(pond.id, pond.ownerId, previous.byteSize);
}
throw error;
}
}
private assetOf(bytes: Buffer, size: { width: number; height: number }): BrandingAsset {
return {
// Short digest: it only has to change when the bytes change, and it
// travels in every logo URL.
hash: createHash('sha256').update(bytes).digest('hex').slice(0, 16),
byteSize: bytes.length,
...size,
};
}
async view(): Promise<BrandingView> {
const [logo, logoDark, favicon, instanceName] = await Promise.all([
this.settings.get(INSTANCE_KEYS.logoLight),
this.settings.get(INSTANCE_KEYS.logoDark),
this.settings.get(INSTANCE_KEYS.favicon),
this.settings.get('instance.name'),
]);
return { logo, logoDark, favicon, instanceName };
}
async setLogo(admin: User, variant: LogoVariant, bytes: Buffer): Promise<BrandingView> {
const size = this.assertUsablePng(bytes, MAX_LOGO_EDGE);
await this.storage.save(BrandingService.logoKey(variant), bytes);
await this.settings.set(
variant === 'dark' ? INSTANCE_KEYS.logoDark : INSTANCE_KEYS.logoLight,
this.assetOf(bytes, size),
admin.id,
);
await this.record(admin, variant === 'dark' ? 'logoDark' : 'logo', 'set');
return this.view();
}
async clearLogo(admin: User, variant: LogoVariant): Promise<BrandingView> {
await this.storage.remove(BrandingService.logoKey(variant));
await this.settings.set(
variant === 'dark' ? INSTANCE_KEYS.logoDark : INSTANCE_KEYS.logoLight,
null,
admin.id,
);
await this.record(admin, variant === 'dark' ? 'logoDark' : 'logo', 'cleared');
return this.view();
}
/**
* Both favicon sizes arrive together: the browser produced them from one
* source on the same canvas, and the api cannot resize. Storing them as a
* pair keeps the tab icon and the home-screen icon from ever showing two
* different images.
*/
async setFavicon(admin: User, files: Record<FaviconSize, Buffer>): Promise<BrandingView> {
const sizes = Object.entries(files).map(([declared, bytes]) => {
const size = this.assertUsablePng(bytes, 512);
const expected = Number(declared);
if (size.width !== expected || size.height !== expected) {
throw new BadRequestException({ code: 'branding_favicon_not_square' });
}
return { expected: expected as FaviconSize, bytes, size };
});
for (const entry of sizes) {
await this.storage.save(BrandingService.faviconKey(entry.expected), entry.bytes);
}
// The 32px variant identifies the pair — it is what the tab shows.
const small = sizes.find((entry) => entry.expected === 32)!;
await this.settings.set(INSTANCE_KEYS.favicon, this.assetOf(small.bytes, small.size), admin.id);
await this.record(admin, 'favicon', 'set');
return this.view();
}
async clearFavicon(admin: User): Promise<BrandingView> {
await this.storage.remove(BrandingService.faviconKey(32));
await this.storage.remove(BrandingService.faviconKey(180));
await this.settings.set(INSTANCE_KEYS.favicon, null, admin.id);
await this.record(admin, 'favicon', 'cleared');
return this.view();
}
/** The bytes to serve for a logo variant, or null when none is stored. */
logoBytes(variant: LogoVariant): Promise<Buffer | null> {
return this.storage.read(BrandingService.logoKey(variant));
}
/**
* The favicon bytes: the uploaded one, else the shipped default. The
* `<link rel="icon">` in index.html is static, so this route must always
* answer with an image a 404 there would leave the browser's generic
* icon for good.
*/
async faviconBytes(size: FaviconSize): Promise<{ bytes: Buffer; uploaded: boolean }> {
const stored = await this.storage.read(BrandingService.faviconKey(size));
if (stored) return { bytes: stored, uploaded: true };
const bytes = await readFile(join(__dirname, '../../assets', `default-favicon-${size}.png`));
return { bytes, uploaded: false };
}
/** The pond's own branding, defaulted — one place reads the settings blob. */
async pondBranding(pondId: string): Promise<PondBranding> {
const pond = await this.prisma.pond.findUnique({
where: { id: pondId },
select: { settings: true },
});
if (!pond) throw new NotFoundException();
return pondSettingsSchema.parse(pond.settings ?? {}).branding;
}
private async writePondBranding(
actor: User,
pondId: string,
next: PondBranding,
asset: 'logo' | 'logoDark' | 'favicon',
change: 'set' | 'cleared',
): Promise<PondBranding> {
const pond = await this.prisma.pond.findUniqueOrThrow({
where: { id: pondId },
select: { settings: true },
});
const settings = pondSettingsSchema.parse(pond.settings ?? {});
await this.prisma.pond.update({
where: { id: pondId },
data: { settings: { ...settings, branding: next } as object },
});
await this.audit.record({
action: 'branding.changed',
actorId: actor.id,
targetType: 'pond',
targetId: pondId,
details: { scope: 'pond', pondId, asset, change },
});
return next;
}
/**
* A pond logo, charged to the pond's storage quota (issue #307).
*
* Without the charge, branding would be a way around the quota and
* replacing a logo repeatedly would let a pond admin consume disk with no
* ceiling. Charged BEFORE the write, like attachments, so a race never
* leaves bytes on the volume without a reservation; the bytes a replaced
* asset frees are released first, so re-uploading the same logo is free
* rather than cumulative.
*/
async setPondLogo(
actor: User,
pond: { id: string; ownerId: string },
variant: LogoVariant,
bytes: Buffer,
): Promise<PondBranding> {
const size = this.assertUsablePng(bytes, MAX_LOGO_EDGE);
const current = await this.pondBranding(pond.id);
const previous = variant === 'dark' ? current.logoDark : current.logo;
await this.chargeQuota(pond, bytes.length, previous);
await this.storage.save(BrandingService.pondLogoKey(pond.id, variant), bytes);
const asset = this.assetOf(bytes, size);
return this.writePondBranding(
actor,
pond.id,
variant === 'dark' ? { ...current, logoDark: asset } : { ...current, logo: asset },
variant === 'dark' ? 'logoDark' : 'logo',
'set',
);
}
async clearPondLogo(
actor: User,
pond: { id: string; ownerId: string },
variant: LogoVariant,
): Promise<PondBranding> {
const current = await this.pondBranding(pond.id);
const previous = variant === 'dark' ? current.logoDark : current.logo;
await this.storage.remove(BrandingService.pondLogoKey(pond.id, variant));
if (previous?.byteSize) await this.quotas.release(pond.id, previous.byteSize);
return this.writePondBranding(
actor,
pond.id,
variant === 'dark' ? { ...current, logoDark: null } : { ...current, logo: null },
variant === 'dark' ? 'logoDark' : 'logo',
'cleared',
);
}
async setPondFavicon(
actor: User,
pond: { id: string; ownerId: string },
files: Record<FaviconSize, Buffer>,
): Promise<PondBranding> {
const checked = Object.entries(files).map(([declared, bytes]) => {
const size = this.assertUsablePng(bytes, 512);
const expected = Number(declared);
if (size.width !== expected || size.height !== expected) {
throw new BadRequestException({ code: 'branding_favicon_not_square' });
}
return { expected: expected as FaviconSize, bytes, size };
});
const current = await this.pondBranding(pond.id);
const total = checked.reduce((sum, entry) => sum + entry.bytes.length, 0);
await this.chargeQuota(pond, total, current.favicon);
for (const entry of checked) {
await this.storage.save(BrandingService.pondFaviconKey(pond.id, entry.expected), entry.bytes);
}
const small = checked.find((entry) => entry.expected === 32)!;
// The pair is charged together, so the stored size is the pair's — that
// is what a later release has to give back.
const asset = { ...this.assetOf(small.bytes, small.size), byteSize: total };
return this.writePondBranding(actor, pond.id, { ...current, favicon: asset }, 'favicon', 'set');
}
async clearPondFavicon(
actor: User,
pond: { id: string; ownerId: string },
): Promise<PondBranding> {
const current = await this.pondBranding(pond.id);
for (const size of FAVICON_SIZES) {
await this.storage.remove(BrandingService.pondFaviconKey(pond.id, size));
}
if (current.favicon?.byteSize) await this.quotas.release(pond.id, current.favicon.byteSize);
return this.writePondBranding(
actor,
pond.id,
{ ...current, favicon: null },
'favicon',
'cleared',
);
}
/** Bytes for a pond asset null when the pond has none at that slot, which
* is what makes the caller fall back to the instance level. */
pondLogoBytes(pondId: string, variant: LogoVariant): Promise<Buffer | null> {
return this.storage.read(BrandingService.pondLogoKey(pondId, variant));
}
pondFaviconBytes(pondId: string, size: FaviconSize): Promise<Buffer | null> {
return this.storage.read(BrandingService.pondFaviconKey(pondId, size));
}
/** Removes every branding file of a pond (issue #307's purge obligation). */
async removePondAssets(pondId: string): Promise<void> {
for (const key of BrandingService.pondKeys(pondId)) await this.storage.remove(key);
}
private record(
admin: User,
asset: 'logo' | 'logoDark' | 'favicon',
action: 'set' | 'cleared',
): Promise<unknown> {
// `scope` is here from the start so the pond-level change (#307) is the
// same event with a different scope, not a second id in the catalogue.
return this.audit.record({
action: 'branding.changed',
actorId: admin.id,
targetType: 'setting',
targetId: `instance.${asset}`,
details: { scope: 'instance', asset, change: action },
});
}
}

View File

@ -0,0 +1,256 @@
import { mkdtemp, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { deflateSync } from 'node:zlib';
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { AuthTokensService } from '../auth/auth-tokens.service';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import {
createTestPrisma,
deletePondsWhere,
grantOwnerAdmin,
hasTestDb,
uniqueSuffix,
} from '../testing/test-db';
import { TrashService } from '../trash/trash.service';
import { UsersService } from '../users/users.service';
import { BrandingService } from './branding.service';
import { BrandingStorageService } from './branding-storage.service';
const crcTable = Array.from({ length: 256 }, (_, n) => {
let c = n;
for (let k = 0; k < 8; k += 1) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
return c >>> 0;
});
function crc32(buf: Buffer): number {
let c = 0xffffffff;
for (const byte of buf) c = crcTable[(c ^ byte) & 0xff]! ^ (c >>> 8);
return (c ^ 0xffffffff) >>> 0;
}
function chunk(type: string, data: Buffer): Buffer {
const length = Buffer.alloc(4);
length.writeUInt32BE(data.length);
const body = Buffer.concat([Buffer.from(type, 'ascii'), data]);
const crc = Buffer.alloc(4);
crc.writeUInt32BE(crc32(body));
return Buffer.concat([length, body, crc]);
}
/** A real PNG — the api reads the IHDR, so the header has to be genuine. */
function png(size: number): Buffer {
const ihdr = Buffer.alloc(13);
ihdr.writeUInt32BE(size, 0);
ihdr.writeUInt32BE(size, 4);
ihdr[8] = 8;
ihdr[9] = 6;
return Buffer.concat([
Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]),
chunk('IHDR', ihdr),
chunk('IDAT', deflateSync(Buffer.alloc(size * (size * 4 + 1)))),
chunk('IEND', Buffer.alloc(0)),
]);
}
describe.skipIf(!hasTestDb)('pond branding (e2e, issue #307)', () => {
let app: INestApplication;
let prisma: PrismaClient;
let storage: BrandingStorageService;
let brandingDir: string;
const suffix = uniqueSuffix();
const password = 'teichmarke mit eigenem logo 1';
const owner = { username: `pb-${suffix}` };
const member = { username: `pbm-${suffix}` };
let ownerCookie: string;
let memberCookie: string;
let pondId: string;
const api = () => request(app.getHttpServer());
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
brandingDir = await mkdtemp(join(tmpdir(), 'dorfteich-pondbranding-'));
process.env.BRANDING_DIR = brandingDir;
app = await createTestApp();
storage = app.get(BrandingStorageService);
const users = app.get(UsersService);
const tokens = app.get(AuthTokensService);
// Verification through the endpoint, not `markEmailVerified`: only this
// path creates the personal pond these tests brand.
const verify = async (userId: string): Promise<void> => {
await api()
.post('/api/v1/auth/verify-email')
.send({ token: await tokens.issue(userId, 'EMAIL_VERIFICATION', 600) })
.expect(204);
};
const ownerUser = await users.createUser({
username: owner.username,
email: `${owner.username}@example.org`,
displayName: `Pond Branding Owner ${suffix}`,
password,
locale: 'en',
});
await verify(ownerUser.id);
const memberUser = await users.createUser({
username: member.username,
email: `${member.username}@example.org`,
displayName: `Pond Branding Member ${suffix}`,
password,
locale: 'en',
});
await verify(memberUser.id);
const login = async (username: string): Promise<string> =>
sessionCookieOf(
await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: username, password })
.expect(200),
);
ownerCookie = await login(owner.username);
memberCookie = await login(member.username);
pondId = (
await prisma.pond.findFirstOrThrow({ where: { ownerId: ownerUser.id, type: 'PERSONAL' } })
).id;
// A reader on the same pond: may see it, may not administer it. Through
// the API, not a raw row — the permission cache would not see the row
// (the documented rule for grants in tests).
await api()
.post(`/api/v1/ponds/${pondId}/grants`)
.set('Cookie', ownerCookie)
.send({
subjectType: 'user',
subjectId: memberUser.id,
role: 'reader',
scopeType: 'pond',
effect: 'allow',
})
.expect(201);
});
afterAll(async () => {
await prisma.roleGrant.deleteMany({
where: { pond: { owner: { username: { contains: suffix } } } },
});
await deletePondsWhere(prisma, { owner: { username: { contains: suffix } } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();
await rm(brandingDir, { recursive: true, force: true });
delete process.env.BRANDING_DIR;
});
it('stores a pond logo, reports it, and serves it under the pond scope', async () => {
const view = await api()
.post(`/api/v1/ponds/${pondId}/branding/logo?variant=light`)
.set('Cookie', ownerCookie)
.attach('file', png(64), 'logo.png')
.expect(201);
expect(view.body.logo).toMatchObject({ width: 64, height: 64 });
const served = await api()
.get(`/api/v1/branding/logo?variant=light&pond=${pondId}`)
.expect(200);
expect(served.headers['content-type']).toContain('image/png');
// Without the pond scope the instance level answers — 404 here, since no
// instance logo is set. The two levels never leak into each other.
await api().get('/api/v1/branding/logo?variant=light').expect(404);
});
it('charges the pond quota and gives the bytes back when the logo is replaced', async () => {
const usageOf = async (): Promise<number> =>
Number(
(
await prisma.pondUsage.findUnique({
where: { pondId },
select: { storageBytesUsed: true },
})
)?.storageBytesUsed ?? 0,
);
const before = await usageOf();
const big = png(120);
await api()
.post(`/api/v1/ponds/${pondId}/branding/logo?variant=dark`)
.set('Cookie', ownerCookie)
.attach('file', big, 'logo.png')
.expect(201);
const afterUpload = await usageOf();
expect(afterUpload).toBe(before + big.length);
// Replacing releases the old reservation first — otherwise re-uploading
// the same logo would eat the quota again and again.
await api()
.post(`/api/v1/ponds/${pondId}/branding/logo?variant=dark`)
.set('Cookie', ownerCookie)
.attach('file', big, 'logo.png')
.expect(201);
expect(await usageOf()).toBe(afterUpload);
await api()
.delete(`/api/v1/ponds/${pondId}/branding/logo?variant=dark`)
.set('Cookie', ownerCookie)
.expect(200);
expect(await usageOf()).toBe(before);
});
it('refuses SVG at the pond level too — the rules do not relax for a pond admin', async () => {
const res = await api()
.post(`/api/v1/ponds/${pondId}/branding/logo?variant=light`)
.set('Cookie', ownerCookie)
.attach('file', Buffer.from('<svg xmlns="x"><script/></svg>'), 'x.png')
.expect(400);
expect(res.body.code).toBe('branding_svg_rejected');
});
it('lets a member read the pond branding but not change it', async () => {
await api().get(`/api/v1/ponds/${pondId}/branding`).set('Cookie', memberCookie).expect(200);
await api()
.post(`/api/v1/ponds/${pondId}/branding/logo?variant=light`)
.set('Cookie', memberCookie)
.attach('file', png(32), 'x.png')
.expect(403);
await api()
.delete(`/api/v1/ponds/${pondId}/branding/favicon`)
.set('Cookie', memberCookie)
.expect(403);
});
it('purging the pond removes its branding files', async () => {
// A pond of its own, so the purge does not take the shared fixture with it.
const ownerRow = await prisma.user.findFirstOrThrow({ where: { username: owner.username } });
const created = await prisma.pond.create({
data: {
name: `Purge Branding ${suffix}`,
slug: `purge-branding-${suffix}`,
type: 'SHARED',
ownerId: ownerRow.id,
},
});
// Raw grant row, before this pond's first permission query — the
// documented exception to "grants through the API".
await grantOwnerAdmin(prisma, created.id, ownerRow.id);
await api()
.post(`/api/v1/ponds/${created.id}/branding/logo?variant=light`)
.set('Cookie', ownerCookie)
.attach('file', png(48), 'logo.png')
.expect(201);
expect(await storage.read(BrandingService.pondLogoKey(created.id, 'light'))).not.toBeNull();
await prisma.pond.update({ where: { id: created.id }, data: { deletedAt: new Date() } });
const trash = app.get(TrashService);
await trash.purgePondNow(ownerRow, created.id);
// The purge standard is absolute: after it nothing referencing the pond
// survives — rows OR files.
expect(await storage.read(BrandingService.pondLogoKey(created.id, 'light'))).toBeNull();
});
});

View File

@ -0,0 +1,74 @@
import { INestApplication } from '@nestjs/common';
import { Test } from '@nestjs/testing';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { AppModule } from '../app.module';
// Boots the AppModule without a database (like health.e2e.test.ts): the
// middleware under test runs before any route logic, so the always-on
// healthz endpoint is a representative response (issue #197).
describe('security response headers & CORS (e2e, issue #197)', () => {
let app: INestApplication;
const appOrigin = 'http://localhost:5173'; // APP_BASE_URL default origin
beforeAll(async () => {
process.env.NODE_ENV = 'test';
process.env.DATABASE_URL ??= 'postgresql://nobody:nothing@127.0.0.1:59999/absent';
const moduleRef = await Test.createTestingModule({ imports: [AppModule] }).compile();
app = moduleRef.createNestApplication();
app.setGlobalPrefix('api/v1');
await app.init();
});
afterAll(async () => {
await app.close();
});
it('stamps the full header set on a representative response', async () => {
const res = await request(app.getHttpServer()).get('/api/v1/healthz').expect(200);
expect(res.headers['strict-transport-security']).toBe('max-age=31536000');
expect(res.headers['x-content-type-options']).toBe('nosniff');
expect(res.headers['referrer-policy']).toBe('no-referrer');
// SAMEORIGIN, not DENY — the plugin sandbox frame is embedded
// same-origin (plugins.e2e.db.test.ts asserts the frame side).
expect(res.headers['x-frame-options']).toBe('SAMEORIGIN');
expect(res.headers['permissions-policy']).toBe(
'camera=(), microphone=(), geolocation=(), payment=(), usb=()',
);
});
it('stamps the headers on error responses too (unknown route)', async () => {
const res = await request(app.getHttpServer()).get('/api/v1/does-not-exist').expect(404);
expect(res.headers['x-content-type-options']).toBe('nosniff');
expect(res.headers['x-frame-options']).toBe('SAMEORIGIN');
});
it('grants a foreign origin nothing (no ACAO), while varying on Origin', async () => {
const res = await request(app.getHttpServer())
.get('/api/v1/healthz')
.set('Origin', 'https://attacker.example')
.expect(200);
expect(res.headers['access-control-allow-origin']).toBeUndefined();
expect(res.headers['access-control-allow-credentials']).toBeUndefined();
expect(res.headers.vary).toContain('Origin');
});
it("echoes only the app's own origin, with the credentials rule stated", async () => {
const res = await request(app.getHttpServer())
.get('/api/v1/healthz')
.set('Origin', appOrigin)
.expect(200);
expect(res.headers['access-control-allow-origin']).toBe(appOrigin);
expect(res.headers['access-control-allow-credentials']).toBe('true');
});
it('leaves a foreign preflight ungranted (no CORS response headers)', async () => {
const res = await request(app.getHttpServer())
.options('/api/v1/healthz')
.set('Origin', 'https://attacker.example')
.set('Access-Control-Request-Method', 'POST');
expect(res.headers['access-control-allow-origin']).toBeUndefined();
expect(res.headers['access-control-allow-methods']).toBeUndefined();
});
});

View File

@ -0,0 +1,54 @@
import { Injectable, NestMiddleware } from '@nestjs/common';
import type { NextFunction, Request, Response } from 'express';
import { AppConfig } from '../config/app-config.service';
/**
* Security response headers and the CORS stance for every api response
* (issue #197). Hand-rolled instead of `helmet`: the header set is small
* enough to own, every value below is a deliberate decision, and the api
* gains no transitive dependency. Wired via the AppModule's
* MiddlewareConsumer so the test harness (createTestApp) exercises the
* exact production middleware rationale per header in
* docs/architecture/security.md §Security response headers & CORS.
*/
@Injectable()
export class SecurityHeadersMiddleware implements NestMiddleware {
/** The one origin the SPA is served from; the only origin CORS ever echoes. */
private readonly allowedOrigin: string;
constructor(config: AppConfig) {
this.allowedOrigin = new URL(config.env.APP_BASE_URL).origin;
}
use(req: Request, res: Response, next: NextFunction): void {
// No includeSubDomains: the api cannot speak for sibling subdomains it
// does not control (e.g. a support desk on the same apex). Browsers
// ignore HSTS over plain http, so sending it unconditionally is safe.
res.setHeader('Strict-Transport-Security', 'max-age=31536000');
res.setHeader('X-Content-Type-Options', 'nosniff');
// Page paths are permission-scoped knowledge — leak them to no one.
res.setHeader('Referrer-Policy', 'no-referrer');
// SAMEORIGIN, deliberately not DENY: the plugin sandbox (ADR 0008)
// embeds /api/v1/plugins/<id>/<version>/frame same-origin, and the
// frame's own CSP carries no frame-ancestors — this header governs.
res.setHeader('X-Frame-Options', 'SAMEORIGIN');
// Deny the powerful features outright; nothing in the app uses them.
res.setHeader(
'Permissions-Policy',
'camera=(), microphone=(), geolocation=(), payment=(), usb=()',
);
// CORS: no foreign origin is granted anything — only the app's own
// origin is ever echoed (where browsers do not consult CORS anyway, as
// same-origin; the echo states the decision rather than enabling a
// caller). Same-origin requests never preflight, so no OPTIONS
// handling is needed. Vary on every response keeps caches honest.
res.vary('Origin');
if (req.headers.origin === this.allowedOrigin) {
res.setHeader('Access-Control-Allow-Origin', this.allowedOrigin);
res.setHeader('Access-Control-Allow-Credentials', 'true');
}
next();
}
}

View File

@ -0,0 +1,152 @@
import { createHash } from 'node:crypto';
import { INestApplication, InternalServerErrorException } from '@nestjs/common';
import { PrismaClient, User } from '@prisma/client';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { AuthTokensService } from '../auth/auth-tokens.service';
import { createTestApp } from '../testing/test-app';
import { createTestPrisma, deletePondsWhere, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
import { FileStorageService } from './file-storage.service';
import { FilesService } from './files.service';
const PNG_SIGNATURE = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
const pngBuffer = (payload: string): Buffer => Buffer.concat([PNG_SIGNATURE, Buffer.from(payload)]);
const sha256 = (buffer: Buffer): string => createHash('sha256').update(buffer).digest('hex');
/**
* Attachment integrity (issue #199): uploads store the SHA-256 of the
* written bytes, downloads verify it and fail closed (audited) on mismatch,
* and the nightly backfill hashes pre-#199 rows idempotently, reporting
* unreadable files instead of skipping them.
*/
describe.skipIf(!hasTestDb)('attachment integrity (e2e, issue #199)', () => {
let app: INestApplication;
let prisma: PrismaClient;
let files: FilesService;
let storage: FileStorageService;
let user: User;
let pondId: string;
const suffix = uniqueSuffix();
async function uploadPng(payload: string): Promise<{ id: string; bytes: Buffer }> {
const bytes = pngBuffer(payload);
const view = await files.upload(user, pondId, {
buffer: bytes,
size: bytes.length,
originalname: `${payload}.png`,
});
return { id: view.id, bytes };
}
beforeAll(async () => {
prisma = createTestPrisma();
app = await createTestApp();
files = app.get(FilesService);
storage = app.get(FileStorageService);
const users = app.get(UsersService);
user = await users.createUser({
username: `ines-integrity-${suffix}`,
email: `ines-integrity-${suffix}@example.org`,
displayName: `Ines Integrity ${suffix}`,
password: 'jedes byte bleibt wie es war 1',
locale: 'en',
});
// Verification via the endpoint (not markEmailVerified) because only the
// endpoint creates the personal pond the uploads go into.
const token = await app.get(AuthTokensService).issue(user.id, 'EMAIL_VERIFICATION', 600);
await request(app.getHttpServer())
.post('/api/v1/auth/verify-email')
.send({ token })
.expect(204);
const pond = await prisma.pond.findFirstOrThrow({ where: { ownerId: user.id } });
pondId = pond.id;
});
afterAll(async () => {
await prisma.auditEntry.deleteMany({
where: { action: 'file.integrity_failed', details: { path: ['pondId'], equals: pondId } },
});
await prisma.attachment.deleteMany({ where: { pondId } });
const where = { pond: { owner: { username: { contains: suffix } } } };
await prisma.roleGrant.deleteMany({ where });
await deletePondsWhere(prisma, { owner: { username: { contains: suffix } } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();
});
it('stores the hash of the written bytes at upload', async () => {
const { id, bytes } = await uploadPng('honest-upload');
const row = await prisma.attachment.findUniqueOrThrow({ where: { id } });
expect(row.sha256).toBe(sha256(bytes));
});
it('serves an intact file and fails closed, audited, on a tampered one', async () => {
const { id, bytes } = await uploadPng('will-be-tampered');
// Intact: the download succeeds and streams the exact bytes.
const intact = await files.download(null, id, { actorId: null, sessionKey: 'anon' });
const chunks: Buffer[] = [];
for await (const chunk of intact.stream) chunks.push(chunk as Buffer);
expect(Buffer.concat(chunks).equals(bytes)).toBe(true);
// Tampered on disk (row untouched): fail closed with the dedicated code.
await storage.save(pondId, id, pngBuffer('evil-replacement'));
const failure = await files
.download(null, id, { actorId: null, sessionKey: 'anon' })
.catch((error: unknown) => error);
expect(failure).toBeInstanceOf(InternalServerErrorException);
expect((failure as InternalServerErrorException).getResponse()).toMatchObject({
code: 'attachment_integrity_failure',
});
// The mismatch is on the audit trail with both hashes.
const audit = await prisma.auditEntry.findFirst({
where: { action: 'file.integrity_failed', targetId: id },
});
expect(audit).not.toBeNull();
expect(audit!.details).toMatchObject({
expected: sha256(bytes),
actual: sha256(pngBuffer('evil-replacement')),
});
});
it('backfills missing hashes idempotently and reports unreadable files', async () => {
const readable = await uploadPng('backfill-me');
const unreadable = await uploadPng('bytes-will-vanish');
await prisma.attachment.updateMany({
where: { id: { in: [readable.id, unreadable.id] } },
data: { sha256: null },
});
await storage.delete(pondId, unreadable.id);
// A null-hash row is served unverified (pre-#199 status quo).
const unverified = await files.download(null, readable.id, {
actorId: null,
sessionKey: 'anon',
});
expect(unverified.attachment.sha256).toBeNull();
const first = await files.backfillHashes();
expect(first.hashed).toBeGreaterThanOrEqual(1);
expect(first.unreadable).toBeGreaterThanOrEqual(1);
const rehashed = await prisma.attachment.findUniqueOrThrow({ where: { id: readable.id } });
expect(rehashed.sha256).toBe(sha256(readable.bytes));
// The unreadable row keeps its null hash — reported, retried next run,
// never silently marked done.
const vanished = await prisma.attachment.findUniqueOrThrow({ where: { id: unreadable.id } });
expect(vanished.sha256).toBeNull();
// Idempotent: a second run finds nothing new to hash here.
const second = await files.backfillHashes();
const third = await prisma.attachment.findUniqueOrThrow({ where: { id: readable.id } });
expect(third.sha256).toBe(sha256(readable.bytes));
expect(second.unreadable).toBeGreaterThanOrEqual(1);
});
});

View File

@ -1,5 +1,5 @@
import { createReadStream } from 'node:fs';
import { access, mkdir, rm, writeFile } from 'node:fs/promises';
import { access, mkdir, readdir, readFile, rm, stat, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import type { Readable } from 'node:stream';
@ -30,6 +30,13 @@ export class FileStorageService {
return createReadStream(this.pathFor(pondId, fileId));
}
/** The complete stored bytes. Used where the caller must see the whole
* object before serving a single byte of it integrity verification
* (issue #199) cannot work on a stream that is already leaving. */
read(pondId: string, fileId: string): Promise<Buffer> {
return readFile(this.pathFor(pondId, fileId));
}
/** Whether the file's bytes are actually on disk. Used by the pond export to
* skip an attachment whose bytes are missing (data drift) rather than crash
* the archive stream (issue #65). */
@ -46,4 +53,37 @@ export class FileStorageService {
async delete(pondId: string, fileId: string): Promise<void> {
await rm(this.pathFor(pondId, fileId), { force: true });
}
/**
* Every stored file with its modification time, for the orphan sweep's
* volumedatabase direction (issue #194, ADR 0011). A missing uploads
* directory is an empty volume, not an error.
*/
async listStored(): Promise<{ pondId: string; fileId: string; mtimeMs: number }[]> {
const root = this.config.env.UPLOADS_DIR;
const result: { pondId: string; fileId: string; mtimeMs: number }[] = [];
let pondDirs: string[];
try {
pondDirs = await readdir(root);
} catch {
return result;
}
for (const pondId of pondDirs) {
let files: string[];
try {
files = await readdir(join(root, pondId));
} catch {
continue; // not a directory or vanished mid-walk
}
for (const fileId of files) {
try {
const info = await stat(join(root, pondId, fileId));
if (info.isFile()) result.push({ pondId, fileId, mtimeMs: info.mtimeMs });
} catch {
// vanished mid-walk — the next sweep sees the truth
}
}
}
return result;
}
}

View File

@ -27,6 +27,7 @@ import {
RequiresPagePermission,
RequiresPondRole,
} from '../permissions/permission.decorators';
import { readActorOf } from '../read-trail/read-actor';
import { FilesService } from './files.service';
@ -90,14 +91,18 @@ export class FilesController {
@Req() request: AuthedRequest,
@Res({ passthrough: true }) response: Response,
): Promise<StreamableFile> {
const { attachment, stream, inline } = await this.files.download(request.user ?? null, fileId);
const { attachment, stream, inline, downloadName } = await this.files.download(
request.user ?? null,
fileId,
readActorOf(request),
);
response.set('X-Content-Type-Options', 'nosniff');
// Attachments are immutable — a new upload always gets a new id.
response.set('Cache-Control', 'private, max-age=31536000, immutable');
const kind = inline ? 'inline' : 'attachment';
return new StreamableFile(stream, {
type: attachment.mimeType,
disposition: `${kind}; filename="${encodeURIComponent(attachment.fileName)}"`,
disposition: `${kind}; filename="${encodeURIComponent(downloadName)}"`,
});
}

View File

@ -301,7 +301,6 @@ describe.skipIf(!hasTestDb)('files (e2e, issue #27)', () => {
await api().get(`/api/v1/media/${uploaded.body.id}`).set('Cookie', ownerCookie).expect(200);
const stillThere = await prisma.attachment.findUnique({ where: { id: uploaded.body.id } });
expect(stillThere).not.toBeNull();
expect(stillThere?.deletedAt).toBeNull();
});
it('lists a page attachment for the page and links it (#61)', async () => {
@ -328,6 +327,121 @@ describe.skipIf(!hasTestDb)('files (e2e, issue #27)', () => {
expect(item.pageTitle).toBe(`Page Files ${suffix}`);
});
it('prefixes downloads of classified attachments; unset pageId fails closed (issue #212)', async () => {
const page = await api()
.post(`/api/v1/ponds/${pondId}/pages`)
.set('Cookie', ownerCookie)
.send({ title: `Classified Files ${suffix}` })
.expect(201);
const uploaded = await api()
.post(`/api/v1/pages/${page.body.id}/files`)
.set('Cookie', ownerCookie)
.attach('file', Buffer.from('%PDF-1.4 classified content'), 'geheim.pdf')
.expect(201);
// Unclassified page: unchanged filename.
const openServed = await api()
.get(`/api/v1/media/${uploaded.body.id}`)
.set('Cookie', ownerCookie)
.buffer(true)
.parse(binaryParser as unknown as ParseCallback)
.expect(200);
expect(openServed.headers['content-disposition']).toContain('filename="geheim.pdf"');
// Classified page: the documented VS-NfD_ prefix.
await prisma.page.update({
where: { id: page.body.id as string },
data: { classification: 'VS_NFD' },
});
const served = await api()
.get(`/api/v1/media/${uploaded.body.id}`)
.set('Cookie', ownerCookie)
.buffer(true)
.parse(binaryParser as unknown as ParseCallback)
.expect(200);
expect(served.headers['content-disposition']).toContain('filename="VS-NfD_geheim.pdf"');
// pageId unset (paste-then-insert): fails closed to the pond's highest
// level — the pond now contains a classified page, so the orphan upload
// is served with the prefix too.
const orphan = await api()
.post(`/api/v1/ponds/${pondId}/files`)
.set('Cookie', ownerCookie)
.attach('file', Buffer.from('%PDF-1.4 orphan bytes'), 'lose-datei.pdf')
.expect(201);
const orphanServed = await api()
.get(`/api/v1/media/${orphan.body.id}`)
.set('Cookie', ownerCookie)
.buffer(true)
.parse(binaryParser as unknown as ParseCallback)
.expect(200);
expect(orphanServed.headers['content-disposition']).toContain(
'filename="VS-NfD_lose-datei.pdf"',
);
// Back to all-open: the orphan serves unprefixed again.
await prisma.page.update({
where: { id: page.body.id as string },
data: { classification: 'UNCLASSIFIED' },
});
const openOrphan = await api()
.get(`/api/v1/media/${orphan.body.id}`)
.set('Cookie', ownerCookie)
.buffer(true)
.parse(binaryParser as unknown as ParseCallback)
.expect(200);
expect(openOrphan.headers['content-disposition']).toContain('filename="lose-datei.pdf"');
});
it('blocks uploads to classified pages server-side when the policy says so (issue #213)', async () => {
const settings = app.get(InstanceSettingsService);
const page = await api()
.post(`/api/v1/ponds/${pondId}/pages`)
.set('Cookie', ownerCookie)
.send({ title: `Blocked Uploads ${suffix}` })
.expect(201);
await prisma.page.update({
where: { id: page.body.id as string },
data: { classification: 'VS_NFD' },
});
// Default policy `warn`: the upload is allowed (the UI shows the notice).
await api()
.post(`/api/v1/pages/${page.body.id}/files`)
.set('Cookie', ownerCookie)
.attach('file', Buffer.from('%PDF-1.4 warned upload'), 'warned.pdf')
.expect(201);
await settings.set('classification.uploadPolicy', 'block', 'test');
try {
// Enforced server-side, not only in the UI.
const blocked = await api()
.post(`/api/v1/pages/${page.body.id}/files`)
.set('Cookie', ownerCookie)
.attach('file', Buffer.from('%PDF-1.4 blocked upload'), 'blocked.pdf')
.expect(403);
expect((blocked.body as { code: string }).code).toBe('classified_upload_blocked');
// Unclassified pages stay uploadable under `block`.
const open = await api()
.post(`/api/v1/ponds/${pondId}/pages`)
.set('Cookie', ownerCookie)
.send({ title: `Open Uploads ${suffix}` })
.expect(201);
await api()
.post(`/api/v1/pages/${open.body.id}/files`)
.set('Cookie', ownerCookie)
.attach('file', Buffer.from('%PDF-1.4 open upload'), 'open.pdf')
.expect(201);
} finally {
await settings.set('classification.uploadPolicy', 'warn', 'test');
await prisma.instanceSetting.deleteMany({
where: { key: 'classification.uploadPolicy' },
});
}
});
it('pond file manager reports usage, orphans, and page links (#61)', async () => {
const page = await api()
.post(`/api/v1/ponds/${pondId}/pages`)

View File

@ -1,16 +1,43 @@
import { Module } from '@nestjs/common';
import { Module, OnModuleInit } from '@nestjs/common';
import { CommonModule } from '../common/common.module';
import { PondsModule } from '../ponds/ponds.module';
import { QuotasModule } from '../quotas/quotas.module';
import { SchedulerModule } from '../scheduler/scheduler.module';
import { SchedulerService } from '../scheduler/scheduler.service';
import { FileStorageService } from './file-storage.service';
import { FilesController } from './files.controller';
import { FilesService } from './files.service';
import { OrphanSweepService } from './orphan-sweep.service';
/** Nightly, per operations.md's maintenance-jobs table (issue #194). */
const ORPHAN_SWEEP_CADENCE_SECONDS = 24 * 60 * 60;
@Module({
imports: [PondsModule, QuotasModule],
imports: [CommonModule, PondsModule, QuotasModule, SchedulerModule],
controllers: [FilesController],
providers: [FilesService, FileStorageService],
providers: [FilesService, FileStorageService, OrphanSweepService],
exports: [FileStorageService, FilesService],
})
export class FilesModule {}
export class FilesModule implements OnModuleInit {
constructor(
private readonly scheduler: SchedulerService,
private readonly sweep: OrphanSweepService,
private readonly files: FilesService,
) {}
onModuleInit(): void {
this.scheduler.register({
name: 'orphan-file-sweep',
cadenceSeconds: ORPHAN_SWEEP_CADENCE_SECONDS,
run: async () => {
await this.sweep.sweep();
// Same nightly volume walk, same domain: hash rows that predate
// #199 until none remain (idempotent, bounded batch) — a separate
// scheduled job would outlive its purpose.
await this.files.backfillHashes();
},
});
}
}

View File

@ -1,9 +1,11 @@
import { randomUUID } from 'node:crypto';
import type { Readable } from 'node:stream';
import { createHash, randomUUID } from 'node:crypto';
import { Readable } from 'node:stream';
import {
BadRequestException,
ForbiddenException,
Injectable,
InternalServerErrorException,
NotFoundException,
PayloadTooLargeException,
} from '@nestjs/common';
@ -12,15 +14,19 @@ import {
AttachmentListItemView,
AttachmentView,
PondFilesView,
PageClassification,
SVG_MIME_TYPE,
classificationFilenamePrefix,
fileExtension,
isImageMimeType,
} from '@dorfteich/shared';
import { Attachment, User } from '@prisma/client';
import { PinoLogger } from 'nestjs-pino';
import { AuditService } from '../audit/audit.service';
import { PrismaService } from '../prisma/prisma.service';
import { QuotaService } from '../quotas/quota.service';
import { ReadTrailService, type ReadActor } from '../read-trail/read-trail.service';
import { InstanceSettingsService } from '../settings/instance-settings.service';
import { FileStorageService } from './file-storage.service';
@ -34,6 +40,11 @@ export interface FileDownload {
* else office files, PDFs, and SVG is always sent as a download so it
* can never execute inline (ADR 0011, security.md §Uploads). */
inline: boolean;
/** The filename for the Content-Disposition (issue #212, ADR 0022): the
* original name, prefixed `VS-NfD_` when the attachment's effective
* classification is vs_nfd the one marker an arbitrary binary can
* carry. The file's CONTENT stays unmarked (documented residual risk). */
downloadName: string;
}
/** What the upload bytes resolved to after allowlist + SVG handling. */
@ -51,6 +62,8 @@ export class FilesService {
private readonly quotas: QuotaService,
private readonly storage: FileStorageService,
private readonly settings: InstanceSettingsService,
private readonly audit: AuditService,
private readonly readTrail: ReadTrailService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(FilesService.name);
@ -153,6 +166,9 @@ export class FilesService {
sizeBytes,
storagePath: `${pond.id}/${id}`,
uploadedBy: user.id,
// Integrity hash (issue #199): computed from the exact in-memory
// bytes that were just written — never by re-reading the disk.
sha256: createHash('sha256').update(resolved.buffer).digest('hex'),
},
});
this.logger.info(
@ -192,23 +208,140 @@ export class FilesService {
): Promise<AttachmentView> {
const page = await this.prisma.page.findFirst({ where: { id: pageId } });
if (!page) throw new NotFoundException();
// Attaching to a classified page (issue #213, ADR 0022): the file will
// inherit a classification its content cannot carry (#212). The UI warns;
// the instance can harden the warning into a server-side block — enforced
// HERE, not only client-side.
if (page.classification === 'VS_NFD') {
const policy = await this.settings.get('classification.uploadPolicy');
if (policy === 'block') {
throw new ForbiddenException({ code: 'classified_upload_blocked' });
}
}
return this.upload(user, page.pondId, file, page.id);
}
async download(_user: User | null, id: string): Promise<FileDownload> {
/**
* Serve an attachment, verifying its integrity first (issue #199): the
* whole object is read and hashed BEFORE the first byte leaves a stream
* cannot be un-sent, so verification must precede serving. Memory is
* bounded by the `max_file_bytes` quota that gated the upload. A mismatch
* fails closed with its own error code and lands in the audit trail (a
* security event, not content activity); the operator's move is a restore
* from backup (runbook). Rows that predate #199 (sha256 still null until
* the nightly backfill reaches them) are served unverified that is the
* pre-#199 status quo, not a downgrade.
*/
async download(_user: User | null, id: string, read: ReadActor): Promise<FileDownload> {
const attachment = await this.prisma.attachment.findFirst({ where: { id } });
if (!attachment) throw new NotFoundException();
const buffer = await this.storage.read(attachment.pondId, attachment.id).catch(() => null);
if (!buffer) throw new NotFoundException();
if (attachment.sha256) {
const actual = createHash('sha256').update(buffer).digest('hex');
if (actual !== attachment.sha256) {
await this.audit.record({
action: 'file.integrity_failed',
targetType: 'attachment',
targetId: attachment.id,
details: { pondId: attachment.pondId, expected: attachment.sha256, actual },
});
throw new InternalServerErrorException({ code: 'attachment_integrity_failure' });
}
}
const classification = await this.effectiveClassification(attachment);
// Read trail (issue #222): a download whose effective classification is
// vs_nfd (#212 semantics — page level, pond max when page-less) is a read
// of classified content. `pageId` may be null for pond-level files; the
// attachment id in `details` keeps the object identifiable.
if (classification === 'vs_nfd') {
await this.readTrail.record({
...read,
pageId: attachment.pageId,
pondId: attachment.pondId,
channel: 'attachment',
details: { attachmentId: attachment.id },
});
}
return {
attachment,
stream: this.storage.createReadStream(attachment.pondId, attachment.id),
stream: Readable.from(buffer),
inline: isImageMimeType(attachment.mimeType),
downloadName: `${classificationFilenamePrefix(classification)}${attachment.fileName}`,
};
}
/**
* The classification an attachment inherits (issue #212, ADR 0022): its
* page's level. An attachment whose `pageId` is still unset
* (paste-then-insert, pond-level files) FAILS CLOSED to the highest level
* of any live page in its pond it could belong to any of them, so it is
* treated as classified as the most classified candidate. In an all-open
* pond that is `unclassified`, so nothing gets marked noise.
*/
private async effectiveClassification(attachment: Attachment): Promise<PageClassification> {
if (attachment.pageId) {
const page = await this.prisma.page.findUnique({
where: { id: attachment.pageId },
select: { classification: true },
});
if (page) return page.classification.toLowerCase() as PageClassification;
// Page row gone but link set (race with purge): fall through to the
// pond-wide fail-closed answer below.
}
const classified = await this.prisma.page.findFirst({
where: { pondId: attachment.pondId, deletedAt: null, classification: 'VS_NFD' },
select: { id: true },
});
return classified ? 'vs_nfd' : 'unclassified';
}
/**
* Hash attachments that predate #199 (sha256 null), a bounded batch per
* nightly run until none remain idempotent by construction (hashed rows
* stop matching). An unreadable file is reported (log + count) and left
* null so the next run retries it; the orphan sweep is the mechanism that
* eventually explains truly missing bytes.
*/
async backfillHashes(limit = 1000): Promise<{ hashed: number; unreadable: number }> {
const rows = await this.prisma.attachment.findMany({
where: { sha256: null },
select: { id: true, pondId: true },
take: limit,
});
let hashed = 0;
let unreadable = 0;
for (const row of rows) {
let buffer: Buffer;
try {
buffer = await this.storage.read(row.pondId, row.id);
} catch (error) {
unreadable += 1;
this.logger.error(
{ attachmentId: row.id, pondId: row.pondId, err: error },
'attachment unreadable during hash backfill; will retry next run',
);
continue;
}
await this.prisma.attachment.update({
where: { id: row.id },
data: { sha256: createHash('sha256').update(buffer).digest('hex') },
});
hashed += 1;
}
if (rows.length > 0) {
this.logger.info(
{ hashed, unreadable, batch: rows.length, batchLimit: limit },
'audit: attachment hash backfill progress',
);
}
return { hashed, unreadable };
}
/** Attachments linked to a page, for its attachments section (#61). */
async listForPage(pageId: string): Promise<AttachmentListItemView[]> {
const rows = await this.prisma.attachment.findMany({
where: { pageId, deletedAt: null },
where: { pageId },
orderBy: { createdAt: 'desc' },
include: { uploader: true, page: true },
});
@ -219,7 +352,7 @@ export class FilesService {
async listForPond(pondId: string): Promise<PondFilesView> {
const [rows, usage, storageBytesLimit] = await Promise.all([
this.prisma.attachment.findMany({
where: { pondId, deletedAt: null },
where: { pondId },
orderBy: { createdAt: 'desc' },
include: { uploader: true, page: true },
}),

View File

@ -0,0 +1,179 @@
import { existsSync } from 'node:fs';
import { utimes, writeFile, mkdir } from 'node:fs/promises';
import { join } from 'node:path';
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
import { OrphanSweepService } from './orphan-sweep.service';
const HOUR = 60 * 60 * 1000;
/**
* Orphan-file sweep (issue #194): unclaimed attachments past the grace
* period are reclaimed (row, file, quota), fresh ones are protected
* (paste-then-insert), claimed ones are never touched the page
* attachments panel is a legitimate reference and stray files without a
* database row disappear once old enough.
*/
describe.skipIf(!hasTestDb)('orphan file sweep (e2e, issue #194)', () => {
let app: INestApplication;
let prisma: PrismaClient;
const suffix = uniqueSuffix();
const password = 'orphan sweep pass 1';
const ids: Record<string, string> = {};
const cookies: Record<string, string> = {};
let pondId: string;
let pageId: string;
const api = () => request(app.getHttpServer());
const fileOnDisk = (fondId: string, fileId: string) =>
join(process.env.UPLOADS_DIR!, fondId, fileId);
async function makeUser(handle: string, siteAdmin = false): Promise<void> {
const users = app.get(UsersService);
const username = `os-${handle}-${suffix}`;
const user = await users.createUser({
username,
email: `${username}@example.org`,
displayName: `Sweep ${handle}`,
password,
locale: 'en',
});
ids[handle] = user.id;
await users.markEmailVerified(user.id);
if (siteAdmin) {
await prisma.user.update({ where: { id: user.id }, data: { isSiteAdmin: true } });
}
cookies[handle] = sessionCookieOf(
await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: username, password })
.expect(200),
);
}
/** A real upload via the pond route (pageId stays null = unclaimed). */
async function uploadUnclaimed(name: string): Promise<string> {
const res = await api()
.post(`/api/v1/ponds/${pondId}/files`)
.set('Cookie', cookies.owner!)
.attach('file', Buffer.from(`bytes of ${name}`), name)
.expect(201);
return res.body.id as string;
}
function backdate(attachmentId: string, ageMs: number): Promise<unknown> {
return prisma.attachment.update({
where: { id: attachmentId },
data: { createdAt: new Date(Date.now() - ageMs) },
});
}
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
app = await createTestApp();
await makeUser('owner');
await makeUser('admin', true);
await api()
.put(`/api/v1/admin/quotas/user/${ids.owner!}/additional_ponds`)
.set('Cookie', cookies.admin!)
.send({ value: 5 })
.expect(200);
const pond = await api()
.post('/api/v1/ponds')
.set('Cookie', cookies.owner!)
.send({ name: `Sweep Pond ${suffix}` })
.expect(201);
pondId = pond.body.id;
const page = await api()
.post(`/api/v1/ponds/${pondId}/pages`)
.set('Cookie', cookies.owner!)
.send({ title: `Sweep Page ${suffix}` })
.expect(201);
pageId = page.body.id;
});
afterAll(async () => {
const all = Object.values(ids);
await prisma.quotaOverride.deleteMany({ where: { subjectId: { in: all } } });
await prisma.auditEntry.deleteMany({ where: { actorId: { in: all } } });
const ponds = await prisma.pond.findMany({
where: { ownerId: { in: all } },
select: { id: true },
});
const pondIds = ponds.map((p) => p.id);
await prisma.attachment.deleteMany({ where: { pondId: { in: pondIds } } });
await prisma.page.deleteMany({ where: { pondId: { in: pondIds } } });
await prisma.pond.deleteMany({ where: { id: { in: pondIds } } });
await prisma.watch.deleteMany({ where: { userId: { in: all } } });
await prisma.session.deleteMany({ where: { userId: { in: all } } });
await prisma.userIdentity.deleteMany({ where: { userId: { in: all } } });
await prisma.user.deleteMany({ where: { id: { in: all } } });
await prisma.$disconnect();
await app.close();
});
it('reclaims unclaimed attachments past the grace period, protects fresh and claimed ones', async () => {
const oldUnclaimed = await uploadUnclaimed('old-unclaimed.txt');
const freshUnclaimed = await uploadUnclaimed('fresh-unclaimed.txt');
const oldClaimed = await api()
.post(`/api/v1/pages/${pageId}/files`)
.set('Cookie', cookies.owner!)
.attach('file', Buffer.from('panel asset'), 'panel-asset.txt')
.expect(201);
await backdate(oldUnclaimed, 25 * HOUR);
await backdate(oldClaimed.body.id, 25 * HOUR);
const usageBefore = await prisma.pondUsage.findUnique({ where: { pondId } });
const reclaimedBytes = (
await prisma.attachment.findUniqueOrThrow({
where: { id: oldUnclaimed },
})
).sizeBytes;
const result = await app.get(OrphanSweepService).sweep();
expect(result.reclaimed).toBeGreaterThanOrEqual(1);
// The old unclaimed upload is gone: row, file, quota.
expect(await prisma.attachment.findUnique({ where: { id: oldUnclaimed } })).toBeNull();
expect(existsSync(fileOnDisk(pondId, oldUnclaimed))).toBe(false);
const usageAfter = await prisma.pondUsage.findUnique({ where: { pondId } });
expect(Number(usageBefore!.storageBytesUsed) - Number(usageAfter!.storageBytesUsed)).toBe(
reclaimedBytes,
);
// The fresh unclaimed upload survives (paste-then-insert grace).
expect(await prisma.attachment.findUnique({ where: { id: freshUnclaimed } })).not.toBeNull();
expect(existsSync(fileOnDisk(pondId, freshUnclaimed))).toBe(true);
// The claimed panel asset survives despite its age — never swept.
expect(
await prisma.attachment.findUnique({ where: { id: oldClaimed.body.id } }),
).not.toBeNull();
expect(existsSync(fileOnDisk(pondId, oldClaimed.body.id))).toBe(true);
});
it('removes stray files without a database row once they are old enough', async () => {
const dir = join(process.env.UPLOADS_DIR!, pondId);
await mkdir(dir, { recursive: true });
const oldStray = join(dir, `stray-old-${suffix}`);
const freshStray = join(dir, `stray-fresh-${suffix}`);
await writeFile(oldStray, 'stray bytes');
await writeFile(freshStray, 'stray bytes');
const past = new Date(Date.now() - 25 * HOUR);
await utimes(oldStray, past, past);
const result = await app.get(OrphanSweepService).sweep();
expect(existsSync(oldStray)).toBe(false);
expect(existsSync(freshStray)).toBe(true);
expect(result.strays).toBeGreaterThanOrEqual(1);
});
});

View File

@ -0,0 +1,81 @@
import { Injectable } from '@nestjs/common';
import { PinoLogger } from 'nestjs-pino';
import { ClockService } from '../common/clock.service';
import { PrismaService } from '../prisma/prisma.service';
import { QuotaService } from '../quotas/quota.service';
import { FileStorageService } from './file-storage.service';
/**
* Nightly orphan-file sweep (issue #194, ADR 0011) with two directions:
*
* 1. UNCLAIMED ROWS: an attachment whose `pageId` is still null after the
* grace period was claimed by nothing not by a collab persist (which
* claims every embedded image, apps/collab persistence), not by a page
* upload (#61), not by an import (`linkAttachmentsToPage`). The pond
* file manager flags exactly these as orphans; the sweep reclaims them
* (row + file + quota). The grace period protects paste-then-insert:
* an upload is unclaimed until the ~2 s-debounced persist runs.
*
* 2. STRAY FILES: bytes on the uploads volume without a database row
* (volumeDB drift, e.g. a crash between file write and row insert).
* Removed once older than the grace period; no quota to correct the
* reservation was rolled back with the failed upload.
*
* Deliberately NOT swept: claimed attachments whose page content no longer
* references them. The page attachments panel lists claimed files as
* user-managed objects inserting into the document is optional there
* so "not embedded" is not "unused"; auto-deleting would destroy panel
* assets. Humans clean those up in the panel or the pond file manager.
*/
const GRACE_MS = 24 * 60 * 60 * 1000;
@Injectable()
export class OrphanSweepService {
constructor(
private readonly prisma: PrismaService,
private readonly storage: FileStorageService,
private readonly quotas: QuotaService,
private readonly clock: ClockService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(OrphanSweepService.name);
}
async sweep(): Promise<{ reclaimed: number; strays: number }> {
const cutoff = this.clock.now().getTime() - GRACE_MS;
const unclaimed = await this.prisma.attachment.findMany({
where: { pageId: null, createdAt: { lte: new Date(cutoff) } },
});
for (const attachment of unclaimed) {
// File first (idempotent), then row + quota — a crash in between
// leaves a row the next sweep finishes, never untracked bytes.
await this.storage.delete(attachment.pondId, attachment.id);
await this.prisma.attachment.deleteMany({ where: { id: attachment.id } });
await this.quotas.release(attachment.pondId, attachment.sizeBytes);
this.logger.info(
{ attachmentId: attachment.id, pondId: attachment.pondId },
'audit: orphaned attachment reclaimed',
);
}
let strays = 0;
const stored = await this.storage.listStored();
const ids = new Set(
(await this.prisma.attachment.findMany({ select: { id: true } })).map((a) => a.id),
);
for (const file of stored) {
if (ids.has(file.fileId) || file.mtimeMs > cutoff) continue;
await this.storage.delete(file.pondId, file.fileId);
strays += 1;
this.logger.info(
{ fileId: file.fileId, pondId: file.pondId },
'audit: stray file without database row removed',
);
}
return { reclaimed: unclaimed.length, strays };
}
}

View File

@ -0,0 +1,55 @@
import { mkdir, readFile, rm, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import { Injectable } from '@nestjs/common';
import type { FontUploadFormat } from '@dorfteich/shared';
import { AppConfig } from '../config/app-config.service';
/**
* Filesystem binding for operator-uploaded fonts (issue #303, ADR 0016 §#303).
*
* The layout mirrors the baked-in catalog `<slug>/<slug>-<weight>.woff2`
* so the PDF exporter's `@font-face` builder needs no special case beyond
* choosing the directory.
*
* That directory is `CUSTOM_FONTS_DIR`, NOT `FONTS_DIR`: the latter is baked
* into the image, so anything written there disappears on the next deploy and
* never reaches a backup. This one is a sibling of the uploads and plugins
* mounts and travels in the restore set (`apps/backup/src/data-dirs.ts`).
*/
@Injectable()
export class CustomFontStorageService {
constructor(private readonly config: AppConfig) {}
private dirFor(slug: string): string {
return join(this.config.env.CUSTOM_FONTS_DIR, slug);
}
fileNameFor(slug: string, weight: number, format: FontUploadFormat): string {
return `${slug}-${weight}.${format}`;
}
pathFor(slug: string, weight: number, format: FontUploadFormat): string {
return join(this.dirFor(slug), this.fileNameFor(slug, weight, format));
}
async save(slug: string, weight: number, format: FontUploadFormat, bytes: Buffer): Promise<void> {
await mkdir(this.dirFor(slug), { recursive: true });
await writeFile(this.pathFor(slug, weight, format), bytes);
}
read(slug: string, weight: number, format: FontUploadFormat): Promise<Buffer> {
return readFile(this.pathFor(slug, weight, format));
}
/** Removes the family's whole directory. Missing is fine deletion must
* stay idempotent so a half-failed upload can still be cleaned up. */
async deleteFamily(slug: string): Promise<void> {
await rm(this.dirFor(slug), { recursive: true, force: true });
}
async deleteWeight(slug: string, weight: number, format: FontUploadFormat): Promise<void> {
await rm(this.pathFor(slug, weight, format), { force: true });
}
}

View File

@ -0,0 +1,164 @@
import {
BadRequestException,
Controller,
Delete,
Get,
HttpCode,
Param,
Post,
Req,
Res,
UploadedFiles,
UseGuards,
UseInterceptors,
} from '@nestjs/common';
import { AnyFilesInterceptor } from '@nestjs/platform-express';
import {
CustomFontView,
FONT_WEIGHTS,
MAX_FONT_FILE_BYTES,
createCustomFontInputSchema,
} from '@dorfteich/shared';
import type { Response } from 'express';
import { SiteAdminGuard } from '../admin/site-admin.guard';
import { AuthedRequest, Public } from '../auth/auth.guard';
import { AuthenticatedOnly } from '../permissions/permission.decorators';
import { CustomFontStorageService } from './custom-font-storage.service';
import { CustomFontsService, WeightUpload } from './custom-fonts.service';
/** Multipart field names: `woff2-<weight>` and the optional `woff-<weight>`. */
const FILE_FIELD = /^(woff2|woff)-(\d{3})$/;
function parseUploads(files: Express.Multer.File[] | undefined): WeightUpload[] {
const byWeight = new Map<number, WeightUpload>();
for (const file of files ?? []) {
const match = FILE_FIELD.exec(file.fieldname);
if (!match) throw new BadRequestException({ code: 'font_unexpected_field' });
const weight = Number(match[2]);
if (!(FONT_WEIGHTS as readonly number[]).includes(weight)) {
throw new BadRequestException({ code: 'font_weight_invalid' });
}
const entry = byWeight.get(weight) ?? { weight, woff2: Buffer.alloc(0) };
if (match[1] === 'woff2') entry.woff2 = file.buffer;
else entry.woff = file.buffer;
byWeight.set(weight, entry);
}
// A WOFF without its WOFF2 would produce a weight the PDF path cannot
// embed — the exporter reads WOFF2 only.
for (const entry of byWeight.values()) {
if (entry.woff2.length === 0) throw new BadRequestException({ code: 'font_woff2_missing' });
}
return [...byWeight.values()].sort((a, b) => a.weight - b.weight);
}
/** Site-Admin management of operator-uploaded fonts (issue #303). */
@Controller('admin/fonts')
@UseGuards(SiteAdminGuard)
export class CustomFontsAdminController {
constructor(private readonly fonts: CustomFontsService) {}
@Get()
list(): Promise<CustomFontView[]> {
return this.fonts.list();
}
@Post()
@UseInterceptors(AnyFilesInterceptor({ limits: { fileSize: MAX_FONT_FILE_BYTES } }))
async create(
@Req() request: AuthedRequest,
@UploadedFiles() files: Express.Multer.File[] | undefined,
): Promise<CustomFontView> {
// The metadata rides as ordinary multipart fields next to the files.
const input = createCustomFontInputSchema.parse({
family: request.body?.family,
category: request.body?.category,
licence: request.body?.licence,
licenceUrl: request.body?.licenceUrl || null,
});
return this.fonts.create(request.user!, input, parseUploads(files));
}
@Post(':id/weights')
@UseInterceptors(AnyFilesInterceptor({ limits: { fileSize: MAX_FONT_FILE_BYTES } }))
async addWeight(
@Param('id') id: string,
@Req() request: AuthedRequest,
@UploadedFiles() files: Express.Multer.File[] | undefined,
): Promise<CustomFontView> {
const uploads = parseUploads(files);
if (uploads.length !== 1) throw new BadRequestException({ code: 'font_one_weight_expected' });
return this.fonts.addWeight(request.user!, id, uploads[0]!);
}
/** How many live ponds still use the family — shown before deleting. */
@Get(':id/usage')
async usage(@Param('id') id: string): Promise<{ pondsAffected: number }> {
const font = (await this.fonts.list()).find((entry) => entry.id === id);
if (!font) throw new BadRequestException({ code: 'not_found' });
return { pondsAffected: await this.fonts.pondsUsing(font.family) };
}
@Delete(':id')
@HttpCode(204)
async remove(@Param('id') id: string, @Req() request: AuthedRequest): Promise<void> {
await this.fonts.remove(request.user!, id);
}
}
/**
* Reading side of the uploaded fonts: the family list every signed-in user
* needs, and the bytes themselves.
*
* The listing is NOT site-admin-gated (issue #304): every signed-in user picks
* fonts in their pond's Appearance settings, reads the licence page, and needs
* the `@font-face` rules injected the admin list at `/admin/fonts` carries
* the same data, so gating this one would only force a second, admin-only UI.
*
* The file route is unauthenticated on purpose: a font is referenced from CSS,
* and the login screen carries the pond-independent chrome an authenticated
* font URL would simply not load. The bytes are branding, not content.
*/
@Controller('fonts/custom')
export class CustomFontsFileController {
constructor(
private readonly storage: CustomFontStorageService,
private readonly fonts: CustomFontsService,
) {}
// Explicit access declaration, as every route needs (issue #52's fence
// `route-permissions.e2e.db.test.ts`): a session, no further permission —
// the list says which families exist, which is what the pickers offer.
@AuthenticatedOnly()
@Get()
list(): Promise<CustomFontView[]> {
return this.fonts.list();
}
@Public()
@Get(':slug/:file')
async serve(
@Param('slug') slug: string,
@Param('file') file: string,
@Res() res: Response,
): Promise<void> {
const match = /^([a-z0-9-]+)-(\d{3})\.(woff2|woff)$/.exec(file);
// The slug must match the file's own prefix, so the path cannot be used
// to reach a different family's directory.
if (!match || match[1] !== slug) throw new BadRequestException({ code: 'not_found' });
const known = (await this.fonts.list()).find((entry) => entry.slug === slug);
if (!known) throw new BadRequestException({ code: 'not_found' });
const format = match[3] as 'woff2' | 'woff';
const bytes = await this.storage
.read(slug, Number(match[2]), format)
.catch(() => Promise.reject(new BadRequestException({ code: 'not_found' })));
res.setHeader('Content-Type', format === 'woff2' ? 'font/woff2' : 'font/woff');
// Slug + weight + format identify the bytes; a changed family is a new
// upload under a new id, so a long lifetime is safe.
res.setHeader('Cache-Control', 'public, max-age=31536000, immutable');
res.send(bytes);
}
}

View File

@ -0,0 +1,244 @@
import { mkdtemp, readFile, rm } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, deletePondsWhere, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
/** Smallest bytes that pass the magic check — the api never parses further. */
const woff2 = (): Buffer => Buffer.concat([Buffer.from('wOF2'), Buffer.alloc(64)]);
const woff = (): Buffer => Buffer.concat([Buffer.from('wOFF'), Buffer.alloc(64)]);
describe.skipIf(!hasTestDb)('custom fonts (e2e, issue #303)', () => {
let app: INestApplication;
let prisma: PrismaClient;
let fontsDir: string;
const suffix = uniqueSuffix();
const password = 'schriftverwaltung mit stil 1';
const admin = { username: `fa-${suffix}`, displayName: `Font Admin ${suffix}` };
const plain = { username: `fp-${suffix}`, displayName: `Font Plain ${suffix}` };
let adminCookie: string;
let plainCookie: string;
const api = () => request(app.getHttpServer());
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
// A real directory so the storage layer is exercised, not mocked — the
// point of this suite is that bytes actually land somewhere retrievable.
fontsDir = await mkdtemp(join(tmpdir(), 'dorfteich-fonts-'));
process.env.CUSTOM_FONTS_DIR = fontsDir;
app = await createTestApp();
const users = app.get(UsersService);
const adminUser = await users.createUser({
username: admin.username,
email: `${admin.username}@example.org`,
displayName: admin.displayName,
password,
locale: 'en',
});
await users.markEmailVerified(adminUser.id);
await prisma.user.update({ where: { id: adminUser.id }, data: { isSiteAdmin: true } });
// additional_ponds defaults to 0 (ADR 0011) and the instance default is
// never raised — the usage test needs a pond, so grant an override.
await prisma.quotaOverride.create({
data: {
subjectType: 'USER',
subjectId: adminUser.id,
quotaKey: 'additional_ponds',
value: 10,
},
});
const plainUser = await users.createUser({
username: plain.username,
email: `${plain.username}@example.org`,
displayName: plain.displayName,
password,
locale: 'en',
});
await users.markEmailVerified(plainUser.id);
const login = async (username: string): Promise<string> =>
sessionCookieOf(
await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: username, password })
.expect(200),
);
adminCookie = await login(admin.username);
plainCookie = await login(plain.username);
});
afterAll(async () => {
await prisma.customFont.deleteMany({});
const ids = (
await prisma.user.findMany({
where: { username: { contains: suffix } },
select: { id: true },
})
).map((row) => row.id);
await prisma.quotaOverride.deleteMany({ where: { subjectId: { in: ids } } });
await deletePondsWhere(prisma, { owner: { username: { contains: suffix } } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();
await rm(fontsDir, { recursive: true, force: true });
delete process.env.CUSTOM_FONTS_DIR;
});
it('uploads a family, writes the bytes, and serves them back', async () => {
const created = await api()
.post('/api/v1/admin/fonts')
.set('Cookie', adminCookie)
.field('family', `Hausschrift ${suffix}`)
.field('category', 'serif')
.field('licence', 'Commercial — Foundry XY')
.attach('woff2-400', woff2(), 'x.woff2')
.attach('woff-400', woff(), 'x.woff')
.expect(201);
expect(created.body.weights).toEqual([400]);
expect(created.body.licence).toBe('Commercial — Foundry XY');
const slug = created.body.slug as string;
// The bytes are really on disk, in the catalog's layout.
const onDisk = await readFile(join(fontsDir, slug, `${slug}-400.woff2`));
expect(onDisk.subarray(0, 4).toString()).toBe('wOF2');
// …and reachable without a session: a font is fetched from CSS.
const served = await api().get(`/api/v1/fonts/custom/${slug}/${slug}-400.woff2`).expect(200);
expect(served.headers['content-type']).toContain('font/woff2');
});
it('rejects a file that is not a font, whatever it is called', async () => {
const res = await api()
.post('/api/v1/admin/fonts')
.set('Cookie', adminCookie)
.field('family', `Fake ${suffix}`)
.field('category', 'sans-serif')
.field('licence', 'X')
.attach('woff2-400', Buffer.from('\x89PNG\r\n\x1a\n and more'), 'evil.woff2')
.expect(400);
expect(res.body.code).toBe('font_file_not_a_font');
});
it('refuses a family name that a catalog font already owns', async () => {
const res = await api()
.post('/api/v1/admin/fonts')
.set('Cookie', adminCookie)
.field('family', 'Roboto')
.field('category', 'sans-serif')
.field('licence', 'X')
.attach('woff2-400', woff2(), 'x.woff2')
.expect(409);
expect(res.body.code).toBe('font_family_reserved');
});
it('refuses a weight whose WOFF2 is missing', async () => {
const res = await api()
.post('/api/v1/admin/fonts')
.set('Cookie', adminCookie)
.field('family', `NurWoff ${suffix}`)
.field('category', 'sans-serif')
.field('licence', 'X')
.attach('woff-400', woff(), 'x.woff')
.expect(400);
expect(res.body.code).toBe('font_woff2_missing');
});
/**
* Issue #304: an ordinary member picks fonts in their pond's Appearance
* settings and reads the licence page, so the family list cannot be
* Site-Admin-only only the management routes are.
*/
it('lets any signed-in user read the family list, but nobody anonymous', async () => {
await api()
.post('/api/v1/admin/fonts')
.set('Cookie', adminCookie)
.field('family', `Leseschrift ${suffix}`)
.field('category', 'monospace')
.field('licence', 'Read me')
.attach('woff2-500', woff2(), 'x.woff2')
.expect(201);
const listed = await api().get('/api/v1/fonts/custom').set('Cookie', plainCookie).expect(200);
const seen = (listed.body as { family: string; weights: number[] }[]).find(
(font) => font.family === `Leseschrift ${suffix}`,
);
expect(seen?.weights).toEqual([500]);
await api().get('/api/v1/fonts/custom').expect(401);
});
it('keeps every management route away from a non-admin', async () => {
await api().get('/api/v1/admin/fonts').set('Cookie', plainCookie).expect(403);
await api()
.post('/api/v1/admin/fonts')
.set('Cookie', plainCookie)
.field('family', `Nope ${suffix}`)
.field('category', 'serif')
.field('licence', 'X')
.attach('woff2-400', woff2(), 'x.woff2')
.expect(403);
});
it('counts the ponds a family is used by, and deletion leaves them working', async () => {
const created = await api()
.post('/api/v1/admin/fonts')
.set('Cookie', adminCookie)
.field('family', `Zählschrift ${suffix}`)
.field('category', 'sans-serif')
.field('licence', 'X')
.attach('woff2-400', woff2(), 'x.woff2')
.expect(201);
const pond = await api()
.post('/api/v1/ponds')
.set('Cookie', adminCookie)
.send({ name: `Schriftteich ${suffix}` })
.expect(201);
await api()
.patch(`/api/v1/ponds/${pond.body.id}`)
.set('Cookie', adminCookie)
.send({ fonts: { body: { family: `Zählschrift ${suffix}`, weight: 400 } } })
.expect(200);
const usage = await api()
.get(`/api/v1/admin/fonts/${created.body.id}/usage`)
.set('Cookie', adminCookie)
.expect(200);
expect(usage.body.pondsAffected).toBe(1);
// Deletion is never blocked by usage.
await api()
.delete(`/api/v1/admin/fonts/${created.body.id}`)
.set('Cookie', adminCookie)
.expect(204);
// The pond still resolves — it keeps the stored family name and falls
// back to the system stack, rather than breaking.
const after = await api()
.get(`/api/v1/ponds/${pond.body.slug}`)
.set('Cookie', adminCookie)
.expect(200);
expect(after.body.settings.fonts.body.family).toBe(`Zählschrift ${suffix}`);
expect(
await api().get('/api/v1/admin/fonts').set('Cookie', adminCookie).expect(200),
).toBeTruthy();
const audit = await prisma.auditEntry.findFirst({
where: { action: 'font.deleted', targetId: created.body.id },
});
expect(audit).not.toBeNull();
expect(audit!.details).toMatchObject({ pondsAffected: 1 });
});
});

View File

@ -0,0 +1,249 @@
import {
BadRequestException,
ConflictException,
Injectable,
NotFoundException,
} from '@nestjs/common';
import {
CreateCustomFontInput,
CustomFontView,
FONT_CATALOG,
FontCategory,
FontUploadFormat,
MAX_FONT_FILE_BYTES,
MAX_FONT_WEIGHTS,
fontSlug,
hasFontMagic,
} from '@dorfteich/shared';
import { User } from '@prisma/client';
import { PinoLogger } from 'nestjs-pino';
import { AuditService } from '../audit/audit.service';
import { PrismaService } from '../prisma/prisma.service';
import { CustomFontStorageService } from './custom-font-storage.service';
/** One weight's bytes as they arrive from the controller. */
export interface WeightUpload {
weight: number;
woff2: Buffer;
woff?: Buffer;
}
/**
* Operator-uploaded font families (issue #303, ADR 0016 §#303).
*
* Site-Admin-only, additive to the compile-time catalog, and deliberately
* incurious about the files: the api validates the magic number and the size
* and then stores the bytes. Family, category and licence come from the form.
*/
@Injectable()
export class CustomFontsService {
constructor(
private readonly prisma: PrismaService,
private readonly storage: CustomFontStorageService,
private readonly audit: AuditService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(CustomFontsService.name);
}
/**
* Rejects bytes that are not what they claim to be, before anything is
* written. Deliberately the ONLY inspection: parsing the font would gain
* metadata the form already carries, at the price of a known
* memory-safety surface (ADR 0016 §#303).
*/
private assertUsableFont(bytes: Buffer, format: FontUploadFormat): void {
if (bytes.length === 0) throw new BadRequestException({ code: 'font_file_empty' });
if (bytes.length > MAX_FONT_FILE_BYTES) {
throw new BadRequestException({ code: 'font_file_too_large' });
}
if (!hasFontMagic(bytes, format)) {
throw new BadRequestException({ code: 'font_file_not_a_font' });
}
}
/**
* A custom family must not collide with a catalog one, by name or by slug:
* a pond stores `fonts.<slot>.family` as a plain string, so two families
* answering to the same name would make the PDF path embed whichever file
* it happened to find.
*/
private async assertNameIsFree(family: string, slug: string): Promise<void> {
const catalogHit = FONT_CATALOG.some(
(entry) => entry.family === family || fontSlug(entry.family) === slug,
);
if (catalogHit) throw new ConflictException({ code: 'font_family_reserved' });
const existing = await this.prisma.customFont.findFirst({
where: { OR: [{ family }, { slug }] },
select: { id: true },
});
if (existing) throw new ConflictException({ code: 'font_family_exists' });
}
private viewOf(font: {
id: string;
family: string;
slug: string;
category: string;
licence: string;
licenceUrl: string | null;
createdAt: Date;
weights: { weight: number }[];
}): CustomFontView {
return {
id: font.id,
family: font.family,
slug: font.slug,
category: font.category as FontCategory,
licence: font.licence,
licenceUrl: font.licenceUrl,
weights: font.weights.map((row) => row.weight).sort((a, b) => a - b),
createdAt: font.createdAt.toISOString(),
};
}
async list(): Promise<CustomFontView[]> {
const fonts = await this.prisma.customFont.findMany({
orderBy: { family: 'asc' },
include: { weights: { select: { weight: true } } },
});
return fonts.map((font) => this.viewOf(font));
}
async create(
admin: User,
input: CreateCustomFontInput,
uploads: WeightUpload[],
): Promise<CustomFontView> {
if (uploads.length === 0) throw new BadRequestException({ code: 'font_no_weights' });
if (uploads.length > MAX_FONT_WEIGHTS) {
throw new BadRequestException({ code: 'font_too_many_weights' });
}
for (const upload of uploads) {
this.assertUsableFont(upload.woff2, 'woff2');
if (upload.woff) this.assertUsableFont(upload.woff, 'woff');
}
const slug = fontSlug(input.family);
if (!slug) throw new BadRequestException({ code: 'font_family_unusable' });
await this.assertNameIsFree(input.family, slug);
// Row first, then bytes: a row without files is repairable (re-upload the
// weight), while files without a row would be invisible litter.
const font = await this.prisma.customFont.create({
data: {
family: input.family,
slug,
category: input.category,
licence: input.licence,
licenceUrl: input.licenceUrl,
uploadedBy: admin.id,
weights: {
create: uploads.map((upload) => ({
weight: upload.weight,
hasWoff: Boolean(upload.woff),
byteSize: upload.woff2.length,
})),
},
},
include: { weights: { select: { weight: true } } },
});
for (const upload of uploads) {
await this.storage.save(slug, upload.weight, 'woff2', upload.woff2);
if (upload.woff) await this.storage.save(slug, upload.weight, 'woff', upload.woff);
}
await this.audit.record({
action: 'font.uploaded',
actorId: admin.id,
targetType: 'font',
targetId: font.id,
details: { family: font.family },
});
return this.viewOf(font);
}
async addWeight(admin: User, fontId: string, upload: WeightUpload): Promise<CustomFontView> {
this.assertUsableFont(upload.woff2, 'woff2');
if (upload.woff) this.assertUsableFont(upload.woff, 'woff');
const font = await this.prisma.customFont.findUnique({
where: { id: fontId },
include: { weights: { select: { weight: true } } },
});
if (!font) throw new NotFoundException();
if (font.weights.length >= MAX_FONT_WEIGHTS) {
throw new BadRequestException({ code: 'font_too_many_weights' });
}
if (font.weights.some((row) => row.weight === upload.weight)) {
throw new ConflictException({ code: 'font_weight_exists' });
}
await this.prisma.customFontWeight.create({
data: {
fontId,
weight: upload.weight,
hasWoff: Boolean(upload.woff),
byteSize: upload.woff2.length,
},
});
await this.storage.save(font.slug, upload.weight, 'woff2', upload.woff2);
if (upload.woff) await this.storage.save(font.slug, upload.weight, 'woff', upload.woff);
await this.audit.record({
action: 'font.uploaded',
actorId: admin.id,
targetType: 'font',
targetId: fontId,
details: { family: font.family, weight: upload.weight },
});
const updated = await this.prisma.customFont.findUniqueOrThrow({
where: { id: fontId },
include: { weights: { select: { weight: true } } },
});
return this.viewOf(updated);
}
/**
* How many live ponds still name this family in any of their three font
* slots. Shown before deletion those ponds keep working (an unknown
* family falls back to the system stack) but they visibly change.
*/
async pondsUsing(family: string): Promise<number> {
const rows = await this.prisma.$queryRaw<{ count: bigint }[]>`
SELECT count(*)::bigint AS count
FROM ponds
WHERE deleted_at IS NULL
AND (settings #>> '{fonts,heading,family}' = ${family}
OR settings #>> '{fonts,body,family}' = ${family}
OR settings #>> '{fonts,mono,family}' = ${family})
`;
return Number(rows[0]?.count ?? 0);
}
/**
* Deletion is never blocked by usage. `fontStack` already yields the system
* fallback for an unknown family, so affected ponds degrade rather than
* break, and re-uploading the family restores them but the count travels
* into the audit entry so the change is not silent.
*/
async remove(admin: User, fontId: string): Promise<void> {
const font = await this.prisma.customFont.findUnique({ where: { id: fontId } });
if (!font) throw new NotFoundException();
const pondsAffected = await this.pondsUsing(font.family);
await this.prisma.customFont.delete({ where: { id: fontId } });
await this.storage.deleteFamily(font.slug);
await this.audit.record({
action: 'font.deleted',
actorId: admin.id,
targetType: 'font',
targetId: fontId,
details: { family: font.family, pondsAffected },
});
this.logger.info({ fontId, family: font.family, pondsAffected }, 'custom font deleted');
}
}

View File

@ -0,0 +1,14 @@
import { Module } from '@nestjs/common';
import { CustomFontStorageService } from './custom-font-storage.service';
import { CustomFontsAdminController, CustomFontsFileController } from './custom-fonts.controller';
import { CustomFontsService } from './custom-fonts.service';
/** Operator-uploaded fonts (issue #303, ADR 0016 §#303). Exports the service
* so the PDF exporter can resolve a pond's font to a custom family. */
@Module({
controllers: [CustomFontsAdminController, CustomFontsFileController],
providers: [CustomFontsService, CustomFontStorageService],
exports: [CustomFontsService, CustomFontStorageService],
})
export class FontsModule {}

View File

@ -152,7 +152,14 @@ export class GrantsService {
* pond_admin only at pond scope for a user subject, no extra admins on a
* personal pond, and scope/subject must exist here. Rejects duplicates.
*/
async createGrant(user: User, pondId: string, grant: Grant): Promise<GrantView> {
async createGrant(
user: User,
pondId: string,
grant: Grant,
// `idp` when the claim mapping writes (issue #217): the row is marked
// as mapping-owned and the audit entry names the origin.
options: { origin?: 'manual' | 'idp' } = {},
): Promise<GrantView> {
const pond = await this.requireLivePond(pondId);
const invalid = grantValidationError(grant, {
@ -169,7 +176,7 @@ export class GrantsService {
if (existing) throw new ConflictException({ code: 'grant_exists' });
const created = await this.prisma.roleGrant.create({
data: { pondId, createdBy: user.id, ...columns },
data: { pondId, createdBy: user.id, origin: options.origin ?? 'manual', ...columns },
});
await this.accessChanged(pondId);
await this.audit.record({
@ -185,6 +192,7 @@ export class GrantsService {
scope: grant.scopeType,
scopeId: grant.scopeId,
effect: grant.effect,
...(options.origin === 'idp' ? { origin: 'idp_mapping' } : {}),
},
});
return GrantsService.viewOf(created);
@ -195,7 +203,12 @@ export class GrantsService {
* grant is protected deleting it would leave the pond unmanageable
* (only a Site Admin could recover it).
*/
async deleteGrant(user: User, pondId: string, grantId: string): Promise<void> {
async deleteGrant(
user: User,
pondId: string,
grantId: string,
options: { origin?: 'manual' | 'idp' } = {},
): Promise<void> {
const grant = await this.prisma.roleGrant.findFirst({ where: { id: grantId, pondId } });
if (!grant) throw new NotFoundException();
@ -213,7 +226,12 @@ export class GrantsService {
actorId: user.id,
targetType: 'pond',
targetId: pondId,
details: { grantId, subjectId: grant.subjectId, role: grant.role },
details: {
grantId,
subjectId: grant.subjectId,
role: grant.role,
...(options.origin === 'idp' ? { origin: 'idp_mapping' } : {}),
},
});
}

View File

@ -1,10 +1,12 @@
import deErrors from '@dorfteich/shared/i18n/de/errors.json';
import deLegal from '@dorfteich/shared/i18n/de/legal.json';
import deMails from '@dorfteich/shared/i18n/de/mails.json';
import dePonds from '@dorfteich/shared/i18n/de/ponds.json';
import deTasks from '@dorfteich/shared/i18n/de/tasks.json';
import enErrors from '@dorfteich/shared/i18n/en/errors.json';
import enLegal from '@dorfteich/shared/i18n/en/legal.json';
import enMails from '@dorfteich/shared/i18n/en/mails.json';
import enPonds from '@dorfteich/shared/i18n/en/ponds.json';
import enTasks from '@dorfteich/shared/i18n/en/tasks.json';
import { createInstance, type i18n as I18n } from 'i18next';
@ -17,8 +19,8 @@ export const apiI18n: I18n = createInstance();
void apiI18n.init({
resources: {
en: { errors: enErrors, mails: enMails, legal: enLegal, tasks: enTasks },
de: { errors: deErrors, mails: deMails, legal: deLegal, tasks: deTasks },
en: { errors: enErrors, mails: enMails, legal: enLegal, tasks: enTasks, ponds: enPonds },
de: { errors: deErrors, mails: deMails, legal: deLegal, tasks: deTasks, ponds: dePonds },
},
fallbackLng: 'en',
supportedLngs: ['de', 'en'],

View File

@ -0,0 +1,39 @@
import { describe, expect, it } from 'vitest';
import { markClassifiedMarkdown, parseClassifiedMarkdown } from './classified-markdown';
const MARKING = 'VS NUR FÜR DEN DIENSTGEBRAUCH';
describe('classified markdown marking (issue #210)', () => {
it('wraps a classified page in frontmatter and top+bottom imprint', () => {
const marked = markClassifiedMarkdown('# Title\n\nBody.\n', 'vs_nfd');
expect(marked).toBe(
`---\nclassification: vs_nfd\n---\n\n${MARKING}\n\n# Title\n\nBody.\n\n${MARKING}\n`,
);
});
it('leaves unclassified markdown untouched', () => {
expect(markClassifiedMarkdown('# Title\n\nBody.\n', 'unclassified')).toBe('# Title\n\nBody.\n');
});
it('parse is the inverse of mark', () => {
const original = '# Title\n\nBody.\n';
const { markdown, classification } = parseClassifiedMarkdown(
markClassifiedMarkdown(original, 'vs_nfd'),
);
expect(classification).toBe('vs_nfd');
expect(markdown).toBe(original);
});
it('passes documents without our frontmatter through unchanged', () => {
for (const raw of [
'# Plain\n\nNo frontmatter.\n',
'---\ntitle: Foreign frontmatter\ntags: [a]\n---\n\n# Doc\n',
`${MARKING}\n\nJust an imprint line without frontmatter.\n`,
]) {
const { markdown, classification } = parseClassifiedMarkdown(raw);
expect(classification).toBeNull();
expect(markdown).toBe(raw);
}
});
});

View File

@ -0,0 +1,43 @@
import { PageClassification, classificationMarking } from '@dorfteich/shared';
/**
* VS-NfD marking of exported Markdown (issue #210, ADR 0022): a classified
* page's `.md` carries the level machine-readably in YAML frontmatter AND
* human-visibly as the marking line at the top and bottom of the file.
* Unclassified pages pass through untouched no marking, no frontmatter.
*/
export function markClassifiedMarkdown(
markdown: string,
classification: PageClassification,
): string {
const marking = classificationMarking(classification);
if (!marking) return markdown;
return `---\nclassification: ${classification}\n---\n\n${marking}\n\n${markdown.trimEnd()}\n\n${marking}\n`;
}
/**
* Inverse of {@link markClassifiedMarkdown} for the import side: recognizes
* exactly the frontmatter block we generate (a lone `classification:` key)
* and the marking lines around the body, so a round-trip re-import yields
* the original content and the page starts at the imported level (content
* must not escape its marking by traveling through a ZIP). Anything else
* foreign frontmatter, hand-written documents passes through unchanged.
*/
export function parseClassifiedMarkdown(raw: string): {
markdown: string;
classification: PageClassification | null;
} {
const match = raw.match(/^---\nclassification: (vs_nfd|unclassified)\n---\n\n/);
if (!match) return { markdown: raw, classification: null };
const classification = match[1] as PageClassification;
let body = raw.slice(match[0].length);
const marking = classificationMarking(classification);
if (marking) {
if (body.startsWith(`${marking}\n\n`)) body = body.slice(marking.length + 2);
const trimmed = body.trimEnd();
if (trimmed.endsWith(`\n\n${marking}`)) {
body = `${trimmed.slice(0, -(marking.length + 2)).trimEnd()}\n`;
}
}
return { markdown: body, classification };
}

View File

@ -5,7 +5,7 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { AuthTokensService } from '../auth/auth-tokens.service';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { createTestPrisma, deletePondsWhere, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
import { ConversionJobService } from './conversion-job.service';
@ -108,7 +108,7 @@ describe.skipIf(!hasTestDb)('conversion job queue (e2e, issue #62)', () => {
// grant); clear those before the users they reference.
const where = { pond: { owner: { username: { contains: suffix } } } };
await prisma.roleGrant.deleteMany({ where });
await prisma.pond.deleteMany({ where: { owner: { username: { contains: suffix } } } });
await deletePondsWhere(prisma, { owner: { username: { contains: suffix } } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();

View File

@ -3,11 +3,15 @@ import { ConversionJob, ConversionJobStatus as PrismaStatus } from '@prisma/clie
import { ConversionJobStatus, ConversionJobView } from '@dorfteich/shared';
import { PinoLogger } from 'nestjs-pino';
import { ClockService } from '../common/clock.service';
import { PrismaService } from '../prisma/prisma.service';
import { InstanceSettingsService } from '../settings/instance-settings.service';
import { ConversionWorker } from './conversion-worker.service';
import { MAX_CONVERSION_INPUT_BYTES } from './pandoc.converter';
const MS_PER_DAY = 24 * 60 * 60 * 1000;
export interface EnqueueConversion {
ownerId: string;
kind: string;
@ -50,6 +54,8 @@ export class ConversionJobService {
constructor(
private readonly prisma: PrismaService,
private readonly worker: ConversionWorker,
private readonly settings: InstanceSettingsService,
private readonly clock: ClockService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(ConversionJobService.name);
@ -106,6 +112,35 @@ export class ConversionJobService {
};
}
/**
* Null the raw payload bytes of jobs that finished longer ago than
* `conversion.payloadRetentionDays` (issue #233) every kind, input AND
* result. Only terminal jobs are touched: a PENDING row and a crashed
* RUNNING row awaiting stale-lock recovery keep their input so the worker
* can still (re)process them. The row itself survives for status/audit;
* `updatedAt` marks completion because a terminal row is never written
* again (the payload guard below keeps this run from re-matching rows).
*/
async pruneExpiredPayloads(): Promise<number> {
const retentionDays = await this.settings.get('conversion.payloadRetentionDays');
const cutoff = new Date(this.clock.now().getTime() - retentionDays * MS_PER_DAY);
const result = await this.prisma.conversionJob.updateMany({
where: {
status: { in: ['SUCCEEDED', 'FAILED'] },
updatedAt: { lt: cutoff },
OR: [{ input: { not: null } }, { result: { not: null } }],
},
data: { input: null, result: null, resultMimeType: null },
});
if (result.count > 0) {
this.logger.info(
{ pruned: result.count, cutoff: cutoff.toISOString(), retentionDays },
'audit: conversion job payloads pruned',
);
}
return result.count;
}
private async ownedJob(id: string, userId: string): Promise<ConversionJob> {
const job = await this.prisma.conversionJob.findFirst({ where: { id, ownerId: userId } });
if (!job) throw new NotFoundException();

View File

@ -0,0 +1,120 @@
import { INestApplication } from '@nestjs/common';
import { PrismaClient } from '@prisma/client';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { createTestApp } from '../testing/test-app';
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
import { ConversionJobService } from './conversion-job.service';
const DAY = 24 * 60 * 60 * 1000;
/**
* Conversion payload retention (issue #233): finished jobs past
* `conversion.payloadRetentionDays` lose their raw input/result bytes while
* the row survives for status; pending and stale-RUNNING rows (the worker's
* lock-recovery path) keep their payload untouched.
*/
describe.skipIf(!hasTestDb)('conversion payload prune (e2e, issue #233)', () => {
let app: INestApplication;
let prisma: PrismaClient;
let ownerId: string;
const suffix = uniqueSuffix();
const bytes = () => new Uint8Array(Buffer.from(`payload-${suffix}`));
async function jobRow(
status: 'PENDING' | 'RUNNING' | 'SUCCEEDED' | 'FAILED',
ageDays: number,
lockedAt: Date | null = null,
) {
return prisma.conversionJob.create({
data: {
ownerId,
kind: `test_prune_${suffix}`,
sourceFormat: 'markdown',
targetFormat: 'html',
input: bytes(),
status,
result: status === 'SUCCEEDED' ? bytes() : null,
resultMimeType: status === 'SUCCEEDED' ? 'text/html' : null,
errorCode: status === 'FAILED' ? 'conversion_failed' : null,
lockedAt,
updatedAt: new Date(Date.now() - ageDays * DAY),
},
});
}
beforeAll(async () => {
prisma = createTestPrisma();
// A short period so ages are unambiguous; written straight to the row
// BEFORE the app boots (the settings cache is in-process and fills on
// first read). The key is cleaned afterAll.
await prisma.instanceSetting.upsert({
where: { key: 'conversion.payloadRetentionDays' },
create: { key: 'conversion.payloadRetentionDays', value: 10 },
update: { value: 10 },
});
app = await createTestApp();
const user = await app.get(UsersService).createUser({
username: `pia-prune-${suffix}`,
email: `pia-prune-${suffix}@example.org`,
displayName: `Pia Prune ${suffix}`,
password: 'bytes verschwinden fristgerecht 1',
locale: 'en',
});
ownerId = user.id;
});
afterAll(async () => {
await prisma.instanceSetting.deleteMany({
where: { key: 'conversion.payloadRetentionDays' },
});
await prisma.conversionJob.deleteMany({ where: { kind: `test_prune_${suffix}` } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();
});
it('prunes finished jobs past the period, keeps everything else', async () => {
const oldSucceeded = await jobRow('SUCCEEDED', 15);
const oldFailed = await jobRow('FAILED', 15);
const freshSucceeded = await jobRow('SUCCEEDED', 5);
const oldPending = await jobRow('PENDING', 15);
// A crashed run the worker's stale-lock recovery will pick up again —
// its input must survive or the retry would fail (#233 acceptance).
const oldStaleRunning = await jobRow('RUNNING', 15, new Date(Date.now() - 15 * DAY));
// >=: the shared suite database may hold other files' aged rows.
const pruned = await app.get(ConversionJobService).pruneExpiredPayloads();
expect(pruned).toBeGreaterThanOrEqual(2);
const byId = new Map(
(await prisma.conversionJob.findMany({ where: { kind: `test_prune_${suffix}` } })).map(
(job) => [job.id, job],
),
);
// The finished rows survive with status and error code, only bytes-free.
expect(byId.get(oldSucceeded.id)).toMatchObject({
status: 'SUCCEEDED',
input: null,
result: null,
resultMimeType: null,
});
expect(byId.get(oldFailed.id)).toMatchObject({
status: 'FAILED',
errorCode: 'conversion_failed',
input: null,
result: null,
});
expect(byId.get(freshSucceeded.id)!.input).not.toBeNull();
expect(byId.get(freshSucceeded.id)!.result).not.toBeNull();
expect(byId.get(oldPending.id)!.input).not.toBeNull();
expect(byId.get(oldStaleRunning.id)!.input).not.toBeNull();
});
it('is a no-op when nothing is due', async () => {
expect(await app.get(ConversionJobService).pruneExpiredPayloads()).toBe(0);
});
});

View File

@ -1,3 +1,6 @@
import { readFile } from 'node:fs/promises';
import { join } from 'node:path';
import { Injectable, OnModuleDestroy, OnModuleInit } from '@nestjs/common';
import { ModuleRef } from '@nestjs/core';
import { ConversionJob } from '@prisma/client';
@ -14,7 +17,12 @@ import {
} from './data-export.constants';
import { GotenbergRenderer, RenderError } from './gotenberg.renderer';
import { IMPORT_PROCESSOR, ImportProcessor, isImportKind } from './import.constants';
import { ConversionError, ConversionResult, PandocConverter } from './pandoc.converter';
import {
ConversionError,
ConversionResult,
conversionInputOf,
PandocConverter,
} from './pandoc.converter';
/** How often the worker sweeps for pending jobs on its own the safety net
* that makes a queued conversion survive an API restart even if no new
@ -59,12 +67,36 @@ export class ConversionWorker implements OnModuleInit, OnModuleDestroy {
const result: ConversionResult = await this.converter.convert({
from: job.sourceFormat,
to: job.targetFormat,
input: Buffer.from(job.input),
input: Buffer.from(conversionInputOf(job)),
standalone: job.standalone,
referenceDoc: await this.classifiedReferenceDoc(job),
});
return { bytes: result.output, mimeType: result.mimeType };
}
/**
* The classified reference document for a marked docx/odt export (issue
* #209, ADR 0022): pandoc copies its header/footer which carry the
* VS-NfD marking into the output, so the marking repeats on every page
* in Word/LibreOffice and is not deletable body text. Only present when
* the enqueue put a `marking` into the job options; the binaries ship in
* `apps/api/assets/` (see `scripts/gen-classified-reference-docs.mjs`).
*/
private async classifiedReferenceDoc(
job: ConversionJob,
): Promise<{ name: string; bytes: Buffer } | undefined> {
const marked = Boolean((job.options as { marking?: string } | null)?.marking);
if (!marked || (job.targetFormat !== 'docx' && job.targetFormat !== 'odt')) return undefined;
const name = `reference-vs-nfd.${job.targetFormat}`;
const cached = this.referenceDocs.get(name);
if (cached) return { name, bytes: cached };
const bytes = await readFile(join(__dirname, '../../assets', name));
this.referenceDocs.set(name, bytes);
return { name, bytes };
}
private readonly referenceDocs = new Map<string, Buffer>();
onModuleInit(): void {
if (this.config.env.NODE_ENV === 'test') return; // tests drive drain() directly
this.timer = setInterval(() => this.drainSafely(), SWEEP_MS);
@ -156,8 +188,14 @@ export class ConversionWorker implements OnModuleInit, OnModuleDestroy {
.build(job);
expiresAt = new Date(Date.now() + DATA_EXPORT_TTL_MS);
} else if (job.targetFormat === 'pdf') {
// A classified page's export carries its marking as a job option
// (issue #208) — Gotenberg repeats it in header/footer of every page.
const marking = (job.options as { marking?: string } | null)?.marking ?? null;
output = {
bytes: await this.renderer.renderHtmlToPdf(Buffer.from(job.input).toString('utf8')),
bytes: await this.renderer.renderHtmlToPdf(
Buffer.from(conversionInputOf(job)).toString('utf8'),
{ marking },
),
mimeType: 'application/pdf',
};
} else {

View File

@ -95,7 +95,14 @@ export class DataExportService implements DataExportProcessor {
orderBy: { slug: 'asc' },
});
for (const pond of ponds) {
await this.exports.appendPondMarkdown(archive, user, pond, `ponds/${pond.slug}/`);
// The build runs in the conversion worker, outside any request — the
// read-trail session key (#222) is the job itself: `job:<id>` names the
// one download this build feeds, so the dedup window (#223) has a
// stable, honest key.
await this.exports.appendPondMarkdown(archive, user, pond, `ponds/${pond.slug}/`, {
actorId: user.id,
sessionKey: `job:${job.id}`,
});
}
await archive.finalize();

View File

@ -1,12 +1,21 @@
import { Body, Controller, Get, Param, Post, Req, Res } from '@nestjs/common';
import { ConversionJobView, PageExportInput, pageExportInputSchema } from '@dorfteich/shared';
import { Body, Controller, Get, Param, Post, Req, Res, UseGuards } from '@nestjs/common';
import {
ConversionJobView,
PageExportInput,
PondArchivePreview,
pageExportInputSchema,
} from '@dorfteich/shared';
import type { Response } from 'express';
import { AuthedRequest } from '../auth/auth.guard';
import { ZodValidationPipe } from '../common/zod-validation.pipe';
import { RequiresPagePermission, RequiresPondRole } from '../permissions/permission.decorators';
import { readActorOf } from '../read-trail/read-actor';
import { SiteAdminGuard } from '../admin/site-admin.guard';
import { ExportService } from './export.service';
import { PondArchiveService } from './pond-archive.service';
/**
* Export endpoints (ADR 0009, issue #65): a whole pond as a ZIP of Markdown and
@ -15,7 +24,41 @@ import { ExportService } from './export.service';
*/
@Controller()
export class ExportController {
constructor(private readonly exports: ExportService) {}
constructor(
private readonly exports: ExportService,
private readonly archives: PondArchiveService,
) {}
/**
* How much of the pond this requester's archive would contain (issue #305).
* Asked before the download so the UI can name the number of omitted pages:
* an archive silently missing content is worse than no archive, because it
* ends the search.
*/
@Get('ponds/:pondId/archive/preview')
@RequiresPondRole('pond_admin', { idParam: 'pondId' })
archivePreview(
@Param('pondId') pondId: string,
@Req() request: AuthedRequest,
): Promise<PondArchivePreview> {
return this.archives.preview(request.user!, pondId, false);
}
/**
* The full archive: every readable page, EVERY attachment, and a versioned
* manifest with settings, labels, comments and the hierarchy (issue #305).
* Pond-Admin, because it is the deletion flow's last resort a reader who
* wants their own copy has the Markdown export.
*/
@Get('ponds/:pondId/archive')
@RequiresPondRole('pond_admin', { idParam: 'pondId' })
async pondArchive(
@Param('pondId') pondId: string,
@Req() request: AuthedRequest,
@Res() response: Response,
): Promise<void> {
await this.archives.stream(request.user!, pondId, response, readActorOf(request), false);
}
/** Streamed ZIP of the pond's readable pages as Markdown (+ `media/`). The
* `reader` role is "may see the pond"; the service filters to readable pages,
@ -27,7 +70,7 @@ export class ExportController {
@Req() request: AuthedRequest,
@Res() response: Response,
): Promise<void> {
await this.exports.streamPondMarkdownZip(request.user!, pondId, response);
await this.exports.streamPondMarkdownZip(request.user!, pondId, response, readActorOf(request));
}
/** Enqueue a `.docx`/`.odt` export of one page; poll `GET /jobs/:id` and
@ -39,6 +82,42 @@ export class ExportController {
@Body(new ZodValidationPipe(pageExportInputSchema)) input: PageExportInput,
@Req() request: AuthedRequest,
): Promise<ConversionJobView> {
return this.exports.enqueuePageExport(request.user!, pageId, input.format);
return this.exports.enqueuePageExport(
request.user!,
pageId,
input.format,
readActorOf(request),
);
}
}
/**
* The Site Admin's archive from the purge dialog (issue #305, #193).
*
* Separate controller because it must NOT carry `@RequiresPondRole`: a Site
* Admin purging a trashed pond is usually not a member of it, and the last
* archive before an irreversible purge must not depend on that. It is
* therefore complete by construction the read filter is skipped.
*/
@Controller('admin/trash')
@UseGuards(SiteAdminGuard)
export class PondArchiveAdminController {
constructor(private readonly archives: PondArchiveService) {}
@Get('ponds/:pondId/archive/preview')
archivePreview(
@Param('pondId') pondId: string,
@Req() request: AuthedRequest,
): Promise<PondArchivePreview> {
return this.archives.preview(request.user!, pondId, true);
}
@Get('ponds/:pondId/archive')
async archive(
@Param('pondId') pondId: string,
@Req() request: AuthedRequest,
@Res() response: Response,
): Promise<void> {
await this.archives.stream(request.user!, pondId, response, readActorOf(request), true);
}
}

View File

@ -1,6 +1,8 @@
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import { classificationMarking } from '@dorfteich/shared';
import { strFromU8, unzipSync } from 'fflate';
import { beforeAll, describe, expect, it, TestContext } from 'vitest';
import { AppConfig } from '../config/app-config.service';
@ -8,6 +10,8 @@ import { AppConfig } from '../config/app-config.service';
import { markdownForDocument } from './export-markdown';
import { PandocServerConverter } from './pandoc.converter';
const MARKING = classificationMarking('vs_nfd')!;
/**
* Export fidelity regression (issue #69, ADR 0009): exports the committed
* Markdown corpus to `.docx`/`.odt` through the real pinned pandoc and reads
@ -70,4 +74,68 @@ describe('export fidelity corpus (real pandoc, issue #69)', () => {
});
}
}
// The classified reference documents (issue #209, ADR 0022): a marked
// export must carry the VS-NfD marking in the document's own header/footer
// definition (repeats per page in Word/LibreOffice, not deletable body
// text); an unmarked export must not. Asserted structurally against the
// real pinned pandoc; the body round-trip above stays untouched by the
// reference doc (headers are outside the content pandoc reads back).
it('a marked docx export carries the marking in header1.xml/footer1.xml; unmarked does not', async (ctx: TestContext) => {
if (!reachable) ctx.skip();
const referenceDoc = {
name: 'reference-vs-nfd.docx',
bytes: readFileSync(join(process.cwd(), 'assets/reference-vs-nfd.docx')),
};
const marked = await converter.convert({
from: 'gfm',
to: 'docx',
input: Buffer.from('# Marked\n\nbody', 'utf8'),
standalone: true,
referenceDoc,
});
const parts = unzipSync(new Uint8Array(marked.output));
const header = strFromU8(parts['word/header1.xml']!);
const footer = strFromU8(parts['word/footer1.xml']!);
expect(header).toContain(MARKING);
expect(footer).toContain(MARKING);
expect(strFromU8(parts['word/document.xml']!)).toContain('headerReference');
const unmarked = await converter.convert({
from: 'gfm',
to: 'docx',
input: Buffer.from('# Open\n\nbody', 'utf8'),
standalone: true,
});
const openParts = unzipSync(new Uint8Array(unmarked.output));
expect(openParts['word/header1.xml']).toBeUndefined();
});
it('a marked odt export carries the marking in its master-page header/footer; unmarked does not', async (ctx: TestContext) => {
if (!reachable) ctx.skip();
const referenceDoc = {
name: 'reference-vs-nfd.odt',
bytes: readFileSync(join(process.cwd(), 'assets/reference-vs-nfd.odt')),
};
const marked = await converter.convert({
from: 'gfm',
to: 'odt',
input: Buffer.from('# Marked\n\nbody', 'utf8'),
standalone: true,
referenceDoc,
});
const styles = strFromU8(unzipSync(new Uint8Array(marked.output))['styles.xml']!);
expect(styles).toContain('<style:header>');
const occurrences = styles.split(MARKING).length - 1;
expect(occurrences).toBeGreaterThanOrEqual(2); // header + footer
const unmarked = await converter.convert({
from: 'gfm',
to: 'odt',
input: Buffer.from('# Open\n\nbody', 'utf8'),
standalone: true,
});
const openStyles = strFromU8(unzipSync(new Uint8Array(unmarked.output))['styles.xml']!);
expect(openStyles).not.toContain(MARKING);
});
});

View File

@ -31,8 +31,10 @@ const PNG_BASE64 =
class RecordingConverter extends PandocConverter {
lastInput = '';
lastReferenceDoc: string | null = null;
convert(request: ConversionRequest): Promise<ConversionResult> {
this.lastInput = request.input.toString('utf8');
this.lastReferenceDoc = request.referenceDoc?.name ?? null;
return Promise.resolve({ output: Buffer.from('OFFICE-BYTES'), mimeType: 'application/x-test' });
}
reachable(): Promise<boolean> {
@ -42,9 +44,11 @@ class RecordingConverter extends PandocConverter {
class RecordingRenderer extends GotenbergRenderer {
lastHtml = '';
lastMarking: string | null = null;
failWith: RenderError | null = null;
renderHtmlToPdf(html: string): Promise<Buffer> {
renderHtmlToPdf(html: string, options?: { marking?: string | null }): Promise<Buffer> {
this.lastHtml = html;
this.lastMarking = options?.marking ?? null;
if (this.failWith) return Promise.reject(this.failWith);
return Promise.resolve(Buffer.from('%PDF-1.7 fake'));
}
@ -189,6 +193,123 @@ describe.skipIf(!hasTestDb)('export (e2e, issue #65)', () => {
expect(target).toContain(`![dot](media/${image.id}.png)`);
});
it('marks classified pages in the pond ZIP with frontmatter+imprint, ships a manifest, and round-trips (#210)', async () => {
const marking = 'VS NUR FÜR DEN DIENSTGEBRAUCH';
const classifiedSlug = await seedPage(
personalPondId,
'Zip Classified',
'# Zip Classified\n\nclassified body text',
);
const openSlug = await seedPage(personalPondId, 'Zip Open', '# Zip Open\n\nopen body text');
await prisma.page.updateMany({
where: { pondId: personalPondId, slug: classifiedSlug },
data: { classification: 'VS_NFD' },
});
const res = await api()
.get(`/api/v1/ponds/${personalPondId}/export/markdown`)
.set('Cookie', ownerCookie)
.buffer(true)
.parse((r, cb) => {
const chunks: Buffer[] = [];
r.on('data', (c: Buffer) => chunks.push(c));
r.on('end', () => cb(null, Buffer.concat(chunks)));
})
.expect(200);
const entries = zipEntries(res.body as Buffer);
// Machine-readable frontmatter AND the visible imprint, top and bottom.
const marked = Buffer.from(entries[`${classifiedSlug}.md`]!).toString('utf8');
expect(marked.startsWith(`---\nclassification: vs_nfd\n---\n\n${marking}\n\n`)).toBe(true);
expect(marked.trimEnd().endsWith(marking)).toBe(true);
// Unclassified files are unchanged: no frontmatter, no imprint.
const open = Buffer.from(entries[`${openSlug}.md`]!).toString('utf8');
expect(open).not.toContain('classification:');
expect(open).not.toContain(marking);
// The manifest lists every file with its level and states the highest once.
const manifest = JSON.parse(Buffer.from(entries['manifest.json']!).toString('utf8')) as {
classification: string;
files: { path: string; classification: string }[];
};
expect(manifest.classification).toBe('vs_nfd');
expect(manifest.files).toContainEqual({
path: `${classifiedSlug}.md`,
classification: 'vs_nfd',
});
expect(manifest.files).toContainEqual({
path: `${openSlug}.md`,
classification: 'unclassified',
});
// Round-trip: re-importing the marked file must not confuse the importer —
// the page starts at the imported level, the body carries neither the
// frontmatter nor the imprint lines.
const imported = await api()
.post(`/api/v1/ponds/${personalPondId}/import`)
.set('Cookie', ownerCookie)
.attach('file', Buffer.from(marked, 'utf8'), 'reimported-classified.md')
.expect(201);
expect(imported.body.status).toBe('succeeded');
const reimported = await prisma.page.findUniqueOrThrow({
where: { id: imported.body.resultPageId as string },
});
expect(reimported.classification).toBe('VS_NFD');
const cache = await prisma.pageContentCache.findUniqueOrThrow({
where: { pageId: reimported.id },
});
expect(cache.markdown).toContain('classified body text');
expect(cache.markdown).not.toContain(marking);
expect(cache.markdown).not.toContain('classification:');
});
it('adds a classification companion for classified media in the ZIP (issue #212)', async () => {
const image = await files.upload({ id: ownerId } as never, personalPondId, {
buffer: Buffer.from(PNG_BASE64, 'base64'),
size: 70,
originalname: 'secret-dot.png',
});
const slug = await seedPage(
personalPondId,
'Zip Media Classified',
`# Zip Media Classified\n\n![dot](${image.id})`,
);
await prisma.page.updateMany({
where: { pondId: personalPondId, slug },
data: { classification: 'VS_NFD' },
});
const res = await api()
.get(`/api/v1/ponds/${personalPondId}/export/markdown`)
.set('Cookie', ownerCookie)
.buffer(true)
.parse((r, cb) => {
const chunks: Buffer[] = [];
r.on('data', (c: Buffer) => chunks.push(c));
r.on('end', () => cb(null, Buffer.concat(chunks)));
})
.expect(200);
const entries = zipEntries(res.body as Buffer);
// Media inherits the highest referencing page's level: sibling companion
// carries the full marking; the manifest lists the media file's level.
const companion = entries[`media/${image.id}.png.classification.txt`];
expect(companion).toBeDefined();
expect(Buffer.from(companion!).toString('utf8')).toContain('VS NUR FÜR DEN DIENSTGEBRAUCH');
const manifest = JSON.parse(Buffer.from(entries['manifest.json']!).toString('utf8')) as {
files: { path: string; classification: string }[];
};
expect(manifest.files).toContainEqual({
path: `media/${image.id}.png`,
classification: 'vs_nfd',
});
await prisma.page.updateMany({
where: { pondId: personalPondId, slug },
data: { classification: 'UNCLASSIFIED' },
});
});
it('skips an attachment whose bytes are missing on disk instead of crashing', async () => {
// An attachment row with no file (data drift): upload then remove the bytes.
const image = await files.upload({ id: ownerId } as never, personalPondId, {
@ -327,6 +448,30 @@ describe.skipIf(!hasTestDb)('export (e2e, issue #65)', () => {
.set('Cookie', ownerCookie)
.expect(200);
expect(result.text).toBe('OFFICE-BYTES');
// An unclassified page converts without a reference doc (issue #209).
expect(fake.lastReferenceDoc).toBeNull();
});
it('hands pandoc the classified reference doc for a marked page (issue #209)', async () => {
const slug = await seedPage(personalPondId, 'Classified Docx', '# Classified Docx\n\nbody');
const page = await prisma.page.findFirstOrThrow({
where: { pondId: personalPondId, slug },
});
await prisma.page.update({ where: { id: page.id }, data: { classification: 'VS_NFD' } });
const enqueued = await api()
.post(`/api/v1/pages/${page.id}/export`)
.set('Cookie', ownerCookie)
.send({ format: 'docx' })
.expect(201);
await worker.drain();
expect(fake.lastReferenceDoc).toBe('reference-vs-nfd.docx');
const done = await api()
.get(`/api/v1/jobs/${enqueued.body.id}`)
.set('Cookie', ownerCookie)
.expect(200);
expect(done.body.status).toBe('succeeded');
});
it('exports a page to PDF: content + image inlined, font CSS, via Gotenberg', async () => {
@ -377,6 +522,35 @@ describe.skipIf(!hasTestDb)('export (e2e, issue #65)', () => {
.expect(200);
expect(result.headers['content-type']).toContain('application/pdf');
expect((result.body as Buffer).toString('utf8')).toContain('%PDF');
// An unclassified page renders without any marking option (issue #208).
expect(renderer.lastMarking).toBeNull();
});
it('hands the VS-NfD marking of a classified page to the renderer (issue #208)', async () => {
const slug = await seedPage(
personalPondId,
'Classified Pdf',
'# Classified Pdf\n\nbody',
'<p>Classified body.</p>',
);
const page = await prisma.page.findFirstOrThrow({
where: { pondId: personalPondId, slug },
});
await prisma.page.update({ where: { id: page.id }, data: { classification: 'VS_NFD' } });
const enqueued = await api()
.post(`/api/v1/pages/${page.id}/export`)
.set('Cookie', ownerCookie)
.send({ format: 'pdf' })
.expect(201);
await worker.drain();
expect(renderer.lastMarking).toBe('VS NUR FÜR DEN DIENSTGEBRAUCH');
const done = await api()
.get(`/api/v1/jobs/${enqueued.body.id}`)
.set('Cookie', ownerCookie)
.expect(200);
expect(done.body.status).toBe('succeeded');
});
it('inlines active section-style plugin CSS into the PDF html (#75)', async () => {

View File

@ -6,7 +6,12 @@ import {
ConversionJobView,
ExportFormat,
PondFonts,
customFontEntries,
fontSlug,
PageClassification,
classificationMarking,
classificationRank,
highestClassification,
pondSettingsSchema,
} from '@dorfteich/shared';
import { User } from '@prisma/client';
@ -16,11 +21,14 @@ import { PinoLogger } from 'nestjs-pino';
import { AppConfig } from '../config/app-config.service';
import { FileStorageService } from '../files/file-storage.service';
import { CustomFontsService } from '../fonts/custom-fonts.service';
import { PermissionService } from '../permissions/permission.service';
import { PluginFallbackRenderer } from '../plugins/plugin-fallback-renderer';
import { PluginsService } from '../plugins/plugins.service';
import { PrismaService } from '../prisma/prisma.service';
import { ReadTrailService, type ReadActor } from '../read-trail/read-trail.service';
import { markClassifiedMarkdown } from './classified-markdown';
import { ConversionJobService } from './conversion-job.service';
import {
imageExtension,
@ -47,6 +55,8 @@ export class ExportService {
private readonly plugins: PluginsService,
private readonly fallbacks: PluginFallbackRenderer,
private readonly config: AppConfig,
private readonly customFonts: CustomFontsService,
private readonly readTrail: ReadTrailService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(ExportService.name);
@ -59,7 +69,12 @@ export class ExportService {
* already checked the requester may see the pond; here we filter to the pages
* they may actually read.
*/
async streamPondMarkdownZip(user: User, pondId: string, res: Response): Promise<void> {
async streamPondMarkdownZip(
user: User,
pondId: string,
res: Response,
read: ReadActor,
): Promise<void> {
const pond = await this.prisma.pond.findFirst({ where: { id: pondId, deletedAt: null } });
if (!pond) throw new NotFoundException();
@ -72,7 +87,7 @@ export class ExportService {
res.destroy(error);
});
archive.pipe(res);
await this.appendPondMarkdown(archive, user, pond);
await this.appendPondMarkdown(archive, user, pond, '', read);
await archive.finalize();
}
@ -88,7 +103,8 @@ export class ExportService {
archive: archiver.Archiver,
user: User,
pond: { id: string; slug: string },
prefix = '',
prefix: string,
read: ReadActor,
): Promise<void> {
const pondId = pond.id;
const pages = await this.prisma.page.findMany({
@ -108,6 +124,21 @@ export class ExportService {
const readablePages = pages.filter((page) => readableIds.has(page.id));
const readableSlugs = new Set(readablePages.map((page) => page.slug));
// Read trail (issue #222): the ZIP is a bulk-egress channel — one event
// per classified page it will contain, recorded BEFORE any classified
// bytes enter the stream, so a failed write aborts the download while the
// evidence is still complete (ADR 0023).
for (const page of readablePages) {
if (page.classification !== 'VS_NFD') continue;
await this.readTrail.record({
...read,
pageId: page.id,
pondId,
channel: 'export',
details: { format: 'markdown_zip' },
});
}
// Every image referenced by a readable page — resolved to attachments that
// still exist in this pond, so the media directory matches the rewrites.
const referenced = new Set<string>();
@ -117,7 +148,7 @@ export class ExportService {
const attachmentRows =
referenced.size > 0
? await this.prisma.attachment.findMany({
where: { id: { in: [...referenced] }, pondId, deletedAt: null },
where: { id: { in: [...referenced] }, pondId },
select: { id: true, mimeType: true },
})
: [];
@ -131,15 +162,64 @@ export class ExportService {
attachments.map((a) => [a.id, `${a.id}.${imageExtension(a.mimeType)}`]),
);
// Media inherits the highest classification among the readable pages that
// reference it (fail-closed, ADR 0022 — a shared image is as classified
// as its most classified use).
const mediaClassification = new Map<string, PageClassification>();
for (const page of readablePages) {
const markdown = markdownForZip(
page.contentCache?.markdown ?? '',
readableSlugs,
mediaNameById,
const level = page.classification.toLowerCase() as PageClassification;
for (const id of imageFileIds(page.contentCache?.markdown ?? '')) {
const current = mediaClassification.get(id) ?? 'unclassified';
if (classificationRank(level) > classificationRank(current)) {
mediaClassification.set(id, level);
}
}
}
const manifestFiles: { path: string; classification: PageClassification }[] = [];
for (const page of readablePages) {
const level = page.classification.toLowerCase() as PageClassification;
// A classified page's file carries the level in YAML frontmatter and
// the marking line at top and bottom (#210); unclassified files are
// byte-identical to the pre-#210 export.
const markdown = markClassifiedMarkdown(
markdownForZip(page.contentCache?.markdown ?? '', readableSlugs, mediaNameById),
level,
);
// Page slugs are unique within a pond, so `<slug>.md` never collides.
archive.append(markdown, { name: `${prefix}${page.slug}.md` });
manifestFiles.push({ path: `${prefix}${page.slug}.md`, classification: level });
}
for (const attachment of attachments) {
const mediaLevel = mediaClassification.get(attachment.id) ?? 'unclassified';
manifestFiles.push({
path: `${prefix}media/${mediaNameById.get(attachment.id)!}`,
classification: mediaLevel,
});
// Companion file for classified media (issue #212): the binary itself
// cannot carry the marking, so a sibling text file states it — it
// survives unpacking and copying, where the manifest may be dropped.
const mediaMarking = classificationMarking(mediaLevel);
if (mediaMarking) {
archive.append(`${mediaMarking}\n`, {
name: `${prefix}media/${mediaNameById.get(attachment.id)!}.classification.txt`,
});
}
}
// The archive-level manifest (#210): every file with its level, and the
// highest level contained stated once — the bulk-egress channel stays
// machine-checkable even after the ZIP is unpacked and copied onward.
archive.append(
JSON.stringify(
{
classification: highestClassification(manifestFiles.map((f) => f.classification)),
files: manifestFiles,
},
null,
2,
),
{ name: `${prefix}manifest.json` },
);
for (const attachment of attachments) {
const stream = this.storage.createReadStream(pond.id, attachment.id);
// Defence in depth: a file removed between the existence check and the
@ -168,14 +248,18 @@ export class ExportService {
user: User,
pageId: string,
format: ExportFormat,
read: ReadActor,
): Promise<ConversionJobView> {
if (format === 'pdf') return this.enqueuePdfExport(user, pageId);
if (format === 'pdf') return this.enqueuePdfExport(user, pageId, read);
const page = await this.prisma.page.findFirst({
where: { id: pageId, deletedAt: null },
include: { contentCache: { select: { markdown: true } } },
});
if (!page) throw new NotFoundException();
// Read trail (issue #222): recorded at enqueue — the user's action; the
// worker's later conversion is machinery, not a second read.
await this.recordClassifiedExport(page, read, format);
// Plugin blocks degrade to their fallback text and sections to quoted
// blocks first (#79) — GFM knows neither construct, and pandoc would
@ -184,6 +268,10 @@ export class ExportService {
const dataUriById = await this.inlineImages(page.pondId, imageFileIds(markdown));
const document = markdownForDocument(markdown, dataUriById);
// A classified page's export records its marking as a job option (#209):
// the worker then hands pandoc the classified reference document whose
// header/footer carry the marking on every page in Word/LibreOffice.
const marking = classificationMarking(page.classification.toLowerCase() as PageClassification);
const job = await this.jobs.enqueue({
ownerId: user.id,
kind: `export_${format}`,
@ -191,6 +279,7 @@ export class ExportService {
to: format,
input: Buffer.from(document, 'utf8'),
standalone: true,
...(marking ? { options: { marking } } : {}),
});
this.logger.info(
{ jobId: job.id, pageId, format, userId: user.id },
@ -206,7 +295,27 @@ export class ExportService {
* the job input; the worker sends it to Gotenberg (`html → pdf`). Building it
* up front keeps the job a plain bytebyte render the worker can retry.
*/
private async enqueuePdfExport(user: User, pageId: string): Promise<ConversionJobView> {
/** One `export` event for a classified page leaving as a document (#222). */
private async recordClassifiedExport(
page: { id: string; pondId: string; classification: string },
read: ReadActor,
format: ExportFormat,
): Promise<void> {
if (page.classification !== 'VS_NFD') return;
await this.readTrail.record({
...read,
pageId: page.id,
pondId: page.pondId,
channel: 'export',
details: { format },
});
}
private async enqueuePdfExport(
user: User,
pageId: string,
read: ReadActor,
): Promise<ConversionJobView> {
const page = await this.prisma.page.findFirst({
where: { id: pageId, deletedAt: null },
include: {
@ -215,6 +324,7 @@ export class ExportService {
},
});
if (!page) throw new NotFoundException();
await this.recordClassifiedExport(page, read, 'pdf');
const fonts = pondSettingsSchema.parse(page.pond.settings ?? {}).fonts;
// Plugin blocks first become their best static form (#79: stored SVG
@ -226,12 +336,19 @@ export class ExportService {
pondName: page.pond.name,
bodyHtml,
fonts,
// Both the rules and the stack need the uploaded families: embedding a
// face the stack never names would render the system font (issue #304).
customFonts: customFontEntries(await this.customFonts.list()),
fontFaceCss: await this.fontFaceCss(fonts),
// Styled sections keep their look in the PDF (#75); a pond without
// active style plugins contributes an empty string.
sectionStyleCss: await this.plugins.sectionStyleCssForPond(page.pondId),
});
// The VS-NfD marking (issue #208, ADR 0022) travels as a job option so
// the worker can hand it to Gotenberg's per-page header/footer templates
// — an unclassified page carries none and renders exactly as before.
const marking = classificationMarking(page.classification.toLowerCase() as PageClassification);
const job = await this.jobs.enqueue({
ownerId: user.id,
kind: 'export_pdf',
@ -239,6 +356,7 @@ export class ExportService {
to: 'pdf',
input: Buffer.from(html, 'utf8'),
standalone: true,
...(marking ? { options: { marking } } : {}),
});
this.logger.info(
{ jobId: job.id, pageId, format: 'pdf', userId: user.id },
@ -259,12 +377,18 @@ export class ExportService {
});
}
/** Base64 `@font-face` rules for the pond's three fonts, read from the
* catalog baked into the image (ADR 0016). A font file that is absent (a
* native dev run without `FONTS_DIR` populated) is skipped the render falls
* back to the system stack rather than failing. */
/** Base64 `@font-face` rules for the pond's three fonts. Catalog families
* come from the directory baked into the image (ADR 0016); operator-uploaded
* ones from `CUSTOM_FONTS_DIR` (issue #303) same on-disk layout, so only
* the base directory differs. A font file that is absent (a native dev run
* without `FONTS_DIR` populated, or a family deleted between the settings
* write and the export) is skipped: the render falls back to the system
* stack rather than failing. */
private async fontFaceCss(fonts: PondFonts): Promise<string> {
const slots = [fonts.heading, fonts.body, fonts.mono];
const customSlugs = new Map(
(await this.customFonts.list()).map((font) => [font.family, font.slug]),
);
// Dedup identical family+weight so a doc that repeats a font embeds it once.
const seen = new Set<string>();
const faces: string[] = [];
@ -272,8 +396,10 @@ export class ExportService {
const key = `${slot.family}:${slot.weight}`;
if (seen.has(key)) continue;
seen.add(key);
const slug = fontSlug(slot.family);
const file = join(this.config.env.FONTS_DIR, slug, `${slug}-${slot.weight}.woff2`);
const customSlug = customSlugs.get(slot.family);
const slug = customSlug ?? fontSlug(slot.family);
const baseDir = customSlug ? this.config.env.CUSTOM_FONTS_DIR : this.config.env.FONTS_DIR;
const file = join(baseDir, slug, `${slug}-${slot.weight}.woff2`);
try {
const bytes = await readFile(file);
faces.push(
@ -281,7 +407,7 @@ export class ExportService {
` src: url('data:font/woff2;base64,${bytes.toString('base64')}') format('woff2'); }`,
);
} catch {
this.logger.warn({ font: key }, 'pdf export: catalog font file missing, using fallback');
this.logger.warn({ font: key }, 'pdf export: font file missing, using fallback');
}
}
return faces.join('\n');
@ -293,7 +419,7 @@ export class ExportService {
const dataUriById = new Map<string, string>();
if (ids.length === 0) return dataUriById;
const attachments = await this.prisma.attachment.findMany({
where: { id: { in: ids }, pondId, deletedAt: null },
where: { id: { in: ids }, pondId },
select: { id: true, mimeType: true },
});
for (const attachment of attachments) {

View File

@ -27,6 +27,40 @@ const FOOTER_HTML =
'<span class="pageNumber"></span> / <span class="totalPages"></span>' +
'</div></body></html>';
function escapeHtml(value: string): string {
return value.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
}
/** Per-page running header carrying the VS-NfD marking (issue #208,
* ADR 0022) rendered by Gotenberg's Chromium header template on EVERY
* page, so a printed/filed PDF stays marked even as single sheets. */
function markingHeaderHtml(marking: string): string {
return (
'<html><head><style>body{margin:0;font:bold 9px system-ui;color:#000;width:100%;' +
'letter-spacing:0.08em;}div{text-align:center;}</style></head><body><div>' +
escapeHtml(marking) +
'</div></body></html>'
);
}
/** Footer variant with the marking next to the existing page numbers. */
function markingFooterHtml(marking: string): string {
return (
'<html><head><style>body{margin:0;font:9px system-ui;color:#64748b;width:100%;}' +
'div{text-align:center;}b{color:#000;letter-spacing:0.08em;}</style></head><body><div>' +
`<b>${escapeHtml(marking)}</b> · ` +
'<span class="pageNumber"></span> / <span class="totalPages"></span>' +
'</div></body></html>'
);
}
/** Options for a render; `marking` = the classification wording to repeat
* in header and footer of every page, `null`/absent = no marking and an
* output byte-identical in layout to the pre-#208 renderer. */
export interface RenderPdfOptions {
marking?: string | null;
}
/** Per ADR 0009: a single render may run for at most 60 s. */
const RENDER_TIMEOUT_MS = 60_000;
@ -38,7 +72,7 @@ const RENDER_TIMEOUT_MS = 60_000;
*/
export abstract class GotenbergRenderer {
/** Render a standalone HTML document (fonts/images already inlined) to PDF. */
abstract renderHtmlToPdf(html: string): Promise<Buffer>;
abstract renderHtmlToPdf(html: string, options?: RenderPdfOptions): Promise<Buffer>;
abstract reachable(): Promise<boolean>;
}
@ -63,16 +97,33 @@ export class GotenbergHttpRenderer extends GotenbergRenderer {
}
}
async renderHtmlToPdf(html: string): Promise<Buffer> {
async renderHtmlToPdf(html: string, options: RenderPdfOptions = {}): Promise<Buffer> {
const marking = options.marking ?? null;
const form = new FormData();
// Gotenberg's Chromium route requires the main document to be `index.html`.
form.append('files', new Blob([html], { type: 'text/html' }), 'index.html');
form.append('files', new Blob([FOOTER_HTML], { type: 'text/html' }), 'footer.html');
// A marked page (issue #208) gets the classification as a per-page
// running header AND next to the page numbers in the footer; without a
// marking the forms are exactly the pre-#208 ones (unchanged output).
if (marking) {
form.append(
'files',
new Blob([markingHeaderHtml(marking)], { type: 'text/html' }),
'header.html',
);
form.append(
'files',
new Blob([markingFooterHtml(marking)], { type: 'text/html' }),
'footer.html',
);
} else {
form.append('files', new Blob([FOOTER_HTML], { type: 'text/html' }), 'footer.html');
}
// Page geometry: A4 with room at the bottom for the page-number footer. The
// document's own `@page`/print CSS controls the rest of the layout.
form.append('paperWidth', '8.27');
form.append('paperHeight', '11.7');
form.append('marginTop', '0.6');
form.append('marginTop', marking ? '0.8' : '0.6');
form.append('marginBottom', '0.8');
form.append('marginLeft', '0.7');
form.append('marginRight', '0.7');

View File

@ -1,18 +1,21 @@
import { Module, OnModuleInit } from '@nestjs/common';
import { CommonModule } from '../common/common.module';
import { FilesModule } from '../files/files.module';
import { FontsModule } from '../fonts/fonts.module';
import { LabelsModule } from '../labels/labels.module';
import { PagesModule } from '../pages/pages.module';
import { PluginsModule } from '../plugins/plugins.module';
import { SchedulerModule } from '../scheduler/scheduler.module';
import { SchedulerService } from '../scheduler/scheduler.service';
import { SettingsModule } from '../settings/settings.module';
import { ConversionJobService } from './conversion-job.service';
import { ConversionWorker } from './conversion-worker.service';
import { DATA_EXPORT_PROCESSOR } from './data-export.constants';
import { DataExportController } from './data-export.controller';
import { DataExportService } from './data-export.service';
import { ExportController } from './export.controller';
import { ExportController, PondArchiveAdminController } from './export.controller';
import { ExportService } from './export.service';
import { GotenbergHttpRenderer, GotenbergRenderer } from './gotenberg.renderer';
import { IMPORT_PROCESSOR } from './import.constants';
@ -20,24 +23,45 @@ import { ImportController } from './import.controller';
import { ImportService } from './import.service';
import { JobsController } from './jobs.controller';
import { PandocConverter, PandocServerConverter } from './pandoc.converter';
import { PondArchiveService } from './pond-archive.service';
/** How often expired data-export payloads are purged (#68). Hourly is ample:
* the link's own expiry check already stops downloads the moment it lapses. */
const EXPORT_PURGE_CADENCE_SECONDS = 60 * 60;
/** Daily, per operations.md's maintenance-jobs table (issue #233): the
* payload retention works in days, so a tighter cadence buys nothing. */
const PAYLOAD_PRUNE_CADENCE_SECONDS = 24 * 60 * 60;
/**
* Import/export orchestration (ADR 0009): the conversion job queue, its worker,
* the pandoc-server client (#62), the document import pipeline (#63), the
* feature exports (#65/#67), and the GDPR account data export (#68).
*/
@Module({
imports: [FilesModule, LabelsModule, PagesModule, PluginsModule, SchedulerModule],
controllers: [JobsController, ImportController, ExportController, DataExportController],
imports: [
CommonModule,
FilesModule,
FontsModule,
LabelsModule,
PagesModule,
PluginsModule,
SchedulerModule,
SettingsModule,
],
controllers: [
JobsController,
ImportController,
ExportController,
PondArchiveAdminController,
DataExportController,
],
providers: [
ConversionJobService,
ConversionWorker,
ImportService,
ExportService,
PondArchiveService,
DataExportService,
// The worker resolves the import pipeline through this token (never the
// class), so its file does not import the import service's (avoids a cycle).
@ -56,6 +80,7 @@ export class ImportExportModule implements OnModuleInit {
constructor(
private readonly scheduler: SchedulerService,
private readonly dataExport: DataExportService,
private readonly jobs: ConversionJobService,
) {}
onModuleInit(): void {
@ -64,5 +89,10 @@ export class ImportExportModule implements OnModuleInit {
cadenceSeconds: EXPORT_PURGE_CADENCE_SECONDS,
run: () => this.dataExport.purgeExpired().then(() => undefined),
});
this.scheduler.register({
name: 'conversion-payload-prune',
cadenceSeconds: PAYLOAD_PRUNE_CADENCE_SECONDS,
run: () => this.jobs.pruneExpiredPayloads().then(() => undefined),
});
}
}

View File

@ -24,6 +24,7 @@ import { PagesService } from '../pages/pages.service';
import { docToState, emptyPageState } from '../pages/yjs-content';
import { PrismaService } from '../prisma/prisma.service';
import { parseClassifiedMarkdown } from './classified-markdown';
import { ImportProcessor } from './import.constants';
import {
ASSET_PLACEHOLDER_PREFIX,
@ -32,7 +33,7 @@ import {
parseVaultZip,
planVaultImport,
} from './obsidian-vault';
import { ConversionError, PandocConverter } from './pandoc.converter';
import { ConversionError, conversionInputOf, PandocConverter } from './pandoc.converter';
import { ConversionJobService } from './conversion-job.service';
/** Source format per accepted upload extension (ADR 0009). `md` is our own
@ -270,7 +271,7 @@ export class ImportService implements ImportProcessor {
const rawMarkdown = await convertImportedDocument(
this.converter,
job.sourceFormat,
Buffer.from(job.input),
Buffer.from(conversionInputOf(job)),
);
const page = await this.createPageFromMarkdown(user, job.pondId, rawMarkdown, job.sourceName);
await this.prisma.conversionJob.update({
@ -321,7 +322,7 @@ export class ImportService implements ImportProcessor {
let plan: VaultImportPlan;
let assetBytes: Map<string, { data: Uint8Array; name: string }>;
try {
const zip = new Uint8Array(job.input);
const zip = new Uint8Array(conversionInputOf(job));
plan = planVaultImport(zip, {
frontmatterMode: options.frontmatterMode,
existingSlugs: new Set(existing.map((row) => row.slug)),
@ -544,12 +545,24 @@ export class ImportService implements ImportProcessor {
// file ids); track what we create so a later failure can be rolled back.
const storedFileIds: string[] = [];
try {
const markdown = await this.storeEmbeddedImages(rawMarkdown, user, pondId, storedFileIds);
// Our own classified export wraps the content in frontmatter + marking
// lines (#210) — strip them and carry the level into the new page, so
// a round-trip neither duplicates the marking nor loses it.
const { markdown: unwrapped, classification } = parseClassifiedMarkdown(rawMarkdown);
const markdown = await this.storeEmbeddedImages(unwrapped, user, pondId, storedFileIds);
const json = markdownToDoc(markdown).toJSON() as unknown as PmNode;
const { title, doc } = this.splitTitle(json, sourceName);
const state = docToState(Node.fromJSON(editorSchema, doc));
const page = await this.pages.createWithState(user, pondId, title, state);
const page = await this.pages.createWithState(
user,
pondId,
title,
state,
null,
undefined,
classification,
);
await this.files.linkAttachmentsToPage(storedFileIds, page.id);
return page;
} catch (error) {

View File

@ -17,6 +17,11 @@ export interface ConversionRequest {
/** Line-wrapping of the writer's output. Import uses `none` so a paragraph
* stays on one line (no soft breaks inside image alt text or links). */
wrap?: 'none' | 'auto' | 'preserve';
/** Reference document for the docx/odt writers (issue #209, ADR 0022):
* pandoc copies its page setup including the header/footer that carry
* the VS-NfD marking into the output. Sent to pandoc-server as an
* in-request file plus the `reference-doc` option. */
referenceDoc?: { name: string; bytes: Buffer };
}
export interface ConversionResult {
@ -50,6 +55,17 @@ export class ConversionError extends Error {
}
}
/** The job's input bytes. Since #233 the column is nullable the retention
* job prunes finished jobs' payloads. It never touches PENDING/RUNNING rows
* (incl. stale-lock recovery), so a claimed job without input was re-queued
* by hand; fail it finally instead of crashing the worker. */
export function conversionInputOf(job: { input: Uint8Array | null }): Uint8Array {
if (!job.input) {
throw new ConversionError('conversion_failed', false, 'input payload was pruned');
}
return job.input;
}
/** Server-side conversion limits (ADR 0009). Input is checked before the
* sidecar call; output is capped while reading the response so a runaway
* conversion can't exhaust memory. */
@ -130,6 +146,14 @@ export class PandocServerConverter extends PandocConverter {
// so these are only present when the import pipeline sets them.
...(request.embedResources ? { 'embed-resources': true } : {}),
...(request.wrap ? { wrap: request.wrap } : {}),
...(request.referenceDoc
? {
'reference-doc': request.referenceDoc.name,
files: {
[request.referenceDoc.name]: request.referenceDoc.bytes.toString('base64'),
},
}
: {}),
}),
signal: controller.signal,
});

View File

@ -0,0 +1,51 @@
import { DEFAULT_FONTS, customFontEntries } from '@dorfteich/shared';
import { describe, expect, it } from 'vitest';
import { buildPdfHtml } from './pdf-html';
const CUSTOM = customFontEntries([
{
id: 'f1',
family: 'Corporate Grotesk',
slug: 'corporate-grotesk',
category: 'sans-serif',
licence: 'Bought from Foundry X',
licenceUrl: null,
weights: [400, 700],
createdAt: '2026-08-01T00:00:00.000Z',
},
]);
function base(family: string): Parameters<typeof buildPdfHtml>[0] {
return {
title: 'T',
pondName: 'P',
bodyHtml: '<p>x</p>',
fonts: { ...DEFAULT_FONTS, body: { family, weight: 400 } },
fontFaceCss: `@font-face { font-family: '${family}'; src: url('data:font/woff2;base64,AA'); }`,
};
}
describe('buildPdfHtml font stacks (issues #303/#304)', () => {
it('names an operator-uploaded family in the CSS stack when it is known', () => {
const html = buildPdfHtml({ ...base('Corporate Grotesk'), customFonts: CUSTOM });
expect(html).toContain("--font-body: 'Corporate Grotesk',");
});
/**
* The regression this pins: the `@font-face` rule for a custom family was
* embedded, but `fontStack` not knowing the family produced the bare
* system fallback, so the rule was never referenced and the PDF rendered in
* the system font while everything reported success.
*/
it('would fall back to the system stack without the uploaded families', () => {
const html = buildPdfHtml(base('Corporate Grotesk'));
expect(html).not.toContain("'Corporate Grotesk',");
expect(html).toContain('--font-body: system-ui');
});
it('leaves catalog families working without any uploaded ones', () => {
const html = buildPdfHtml(base('Lora'));
expect(html).toContain("--font-body: 'Lora', Georgia");
});
});

View File

@ -1,4 +1,4 @@
import { PondFonts, fontStack } from '@dorfteich/shared';
import { FontCatalogEntry, PondFonts, fontStack } from '@dorfteich/shared';
export interface PdfHtmlParams {
title: string;
@ -8,6 +8,12 @@ export interface PdfHtmlParams {
fonts: PondFonts;
/** Pre-built `@font-face` rules (base64 WOFF2) for the pond's fonts. */
fontFaceCss: string;
/** The instance's operator-uploaded families (issue #303), so a pond set to
* one gets it NAMED in the `font-family` stack. Without them `fontStack`
* cannot tell a custom family from a typo and yields the bare system
* fallback the `@font-face` rule would then be embedded but never
* referenced, and the PDF would silently render in the system font. */
customFonts?: readonly FontCatalogEntry[];
/** The pond's active section-style plugin CSS (issue #75), already validated
* at install time (scoped selectors, no external fetches, no `</style>`).
* Sections of a disabled plugin render neutrally their class matches
@ -35,6 +41,7 @@ function escapeHtml(value: string): string {
*/
export function buildPdfHtml(params: PdfHtmlParams): string {
const { fonts } = params;
const extra = params.customFonts ?? [];
return `<!doctype html>
<html lang="en">
<head>
@ -44,9 +51,9 @@ export function buildPdfHtml(params: PdfHtmlParams): string {
${params.fontFaceCss}
@page { size: A4; }
:root {
--font-heading: ${fontStack(fonts.heading.family)};
--font-body: ${fontStack(fonts.body.family)};
--font-mono: ${fontStack(fonts.mono.family)};
--font-heading: ${fontStack(fonts.heading.family, extra)};
--font-body: ${fontStack(fonts.body.family, extra)};
--font-mono: ${fontStack(fonts.mono.family, extra)};
}
html { font-size: 11pt; }
body {

View File

@ -1,4 +1,4 @@
import { DEFAULT_FONTS, PondFonts } from '@dorfteich/shared';
import { DEFAULT_FONTS, PondFonts, classificationMarking } from '@dorfteich/shared';
import { PDFParse } from 'pdf-parse';
import { beforeAll, describe, expect, it, TestContext } from 'vitest';
@ -22,12 +22,12 @@ const renderer = new GotenbergHttpRenderer({ env: { GOTENBERG_URL } } as unknown
let reachable = false;
/** Extract the concatenated text and page count from PDF bytes. */
async function readPdf(pdf: Buffer): Promise<{ text: string; pages: number }> {
/** Extract the concatenated text, per-page texts and page count from PDF bytes. */
async function readPdf(pdf: Buffer): Promise<{ text: string; pages: number; pageTexts: string[] }> {
const parser = new PDFParse({ data: new Uint8Array(pdf) });
try {
const result = await parser.getText();
return { text: result.text, pages: result.total };
return { text: result.text, pages: result.total, pageTexts: result.pages.map((p) => p.text) };
} finally {
await parser.destroy();
}
@ -69,5 +69,36 @@ describe('PDF export smoke (real Gotenberg, issue #69)', () => {
// regression would blow well past that.
expect(pages).toBeGreaterThanOrEqual(2);
expect(pages).toBeLessThanOrEqual(3);
// An unmarked render carries no classification anywhere (issue #208:
// unclassified pages produce an unchanged PDF).
expect(text).not.toContain('DIENSTGEBRAUCH');
});
it('repeats the VS-NfD marking in header and footer of EVERY page (issue #208)', async (ctx: TestContext) => {
if (!reachable) ctx.skip();
const marking = classificationMarking('vs_nfd')!;
const html = buildPdfHtml({
title: 'Marked Fidelity Report',
pondName: 'Fidelity Pond',
bodyHtml:
'<p>First page of classified content.</p>' +
'<div style="page-break-before: always"></div>' +
'<p>Second page of classified content.</p>',
fonts: DEFAULT_FONTS as PondFonts,
fontFaceCss: '',
});
const pdf = await renderer.renderHtmlToPdf(html, { marking });
const { pages, pageTexts } = await readPdf(pdf);
expect(pages).toBeGreaterThanOrEqual(2);
for (const pageText of pageTexts) {
// Once from the running header, once from the footer next to the
// page numbers — on every single page.
const occurrences = pageText.split(marking).length - 1;
expect(occurrences).toBe(2);
}
// The document-level header keeps working alongside the marking.
expect(pageTexts[0]).toContain('Marked Fidelity Report');
expect(pageTexts[0]).toContain('Fidelity Pond');
});
});

View File

@ -0,0 +1,250 @@
import { INestApplication } from '@nestjs/common';
import { PondArchiveManifest } from '@dorfteich/shared';
import { PrismaClient } from '@prisma/client';
import { unzipSync } from 'fflate';
import request from 'supertest';
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
import { AuthTokensService } from '../auth/auth-tokens.service';
import { FilesService } from '../files/files.service';
import { createTestApp, sessionCookieOf } from '../testing/test-app';
import { createTestPrisma, deletePondsWhere, hasTestDb, uniqueSuffix } from '../testing/test-db';
import { UsersService } from '../users/users.service';
const PNG_BASE64 =
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==';
function entries(buffer: Buffer): Record<string, Uint8Array> {
return unzipSync(new Uint8Array(buffer));
}
/** supertest parses text by default — a ZIP has to be collected as bytes. */
function asBinary(req: request.Test): request.Test {
return req.parse((res, cb) => {
const chunks: Buffer[] = [];
res.on('data', (chunk: Buffer) => chunks.push(chunk));
res.on('end', () => cb(null, Buffer.concat(chunks)));
});
}
function manifestOf(buffer: Buffer): PondArchiveManifest {
const raw = entries(buffer)['manifest.json'];
return JSON.parse(Buffer.from(raw!).toString('utf8')) as PondArchiveManifest;
}
/**
* The full pond archive (issue #305). What separates it from the Markdown
* export is exactly what is asserted here: EVERY attachment travels, not only
* the embedded ones, and the manifest carries what Markdown cannot settings,
* labels, comments and the hierarchy.
*/
describe.skipIf(!hasTestDb)('pond archive (e2e, issue #305)', () => {
let app: INestApplication;
let prisma: PrismaClient;
let files: FilesService;
const suffix = uniqueSuffix();
const password = 'archiviere den ganzen teich 1';
const owner = { username: `arch-${suffix}` };
const admin = { username: `archadm-${suffix}` };
let ownerId: string;
let ownerCookie: string;
let adminCookie: string;
let pondId: string;
let parentPageId: string;
const api = () => request(app.getHttpServer());
beforeAll(async () => {
prisma = createTestPrisma();
await prisma.rateLimit.deleteMany({});
app = await createTestApp();
files = app.get(FilesService);
const users = app.get(UsersService);
const tokens = app.get(AuthTokensService);
const ownerUser = await users.createUser({
username: owner.username,
email: `${owner.username}@example.org`,
displayName: `Archive Owner ${suffix}`,
password,
locale: 'en',
});
ownerId = ownerUser.id;
await api()
.post('/api/v1/auth/verify-email')
.send({ token: await tokens.issue(ownerUser.id, 'EMAIL_VERIFICATION', 600) })
.expect(204);
ownerCookie = sessionCookieOf(
await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: owner.username, password })
.expect(200),
);
const adminUser = await users.createUser({
username: admin.username,
email: `${admin.username}@example.org`,
displayName: `Archive Admin ${suffix}`,
password,
locale: 'en',
});
await users.markEmailVerified(adminUser.id);
await prisma.user.update({ where: { id: adminUser.id }, data: { isSiteAdmin: true } });
adminCookie = sessionCookieOf(
await api()
.post('/api/v1/auth/login')
.send({ usernameOrEmail: admin.username, password })
.expect(200),
);
pondId = (await prisma.pond.findFirstOrThrow({ where: { ownerId, type: 'PERSONAL' } })).id;
// A parent and a child page, so the hierarchy has something to state.
const parent = await prisma.page.create({
data: {
pondId,
title: 'Archive Parent',
slug: 'archive-parent',
ydocState: new Uint8Array(),
sortKey: 'a',
createdBy: ownerId,
contentCache: {
create: { plainText: 'Parent body', markdown: 'Parent body', html: '', outline: [] },
},
},
});
parentPageId = parent.id;
await prisma.page.create({
data: {
pondId,
parentId: parent.id,
title: 'Archive Child',
slug: 'archive-child',
ydocState: new Uint8Array(),
sortKey: 'b',
createdBy: ownerId,
contentCache: {
create: { plainText: 'Child body', markdown: 'Child body', html: '', outline: [] },
},
},
});
const label = await prisma.label.create({
data: { pondId, name: `Archive Label ${suffix}`, color: '#2f6f4f' },
});
await prisma.pageLabel.create({ data: { pageId: parent.id, labelId: label.id } });
await prisma.comment.create({
data: { pageId: parent.id, authorId: ownerId, body: 'A remark worth keeping.' },
});
});
afterAll(async () => {
const where = { pond: { owner: { username: { contains: suffix } } } };
await prisma.comment.deleteMany({ where: { page: where } });
await prisma.attachment.deleteMany({ where });
await prisma.pageLabel.deleteMany({ where: { page: where } });
await prisma.label.deleteMany({ where });
await prisma.page.deleteMany({ where });
await prisma.roleGrant.deleteMany({ where });
await deletePondsWhere(prisma, { owner: { username: { contains: suffix } } });
await prisma.user.deleteMany({ where: { username: { contains: suffix } } });
await prisma.$disconnect();
await app.close();
});
it('contains EVERY attachment, not only the embedded ones', async () => {
// The gap this whole issue exists for: an attachment nobody embedded
// would vanish unnoticed with the Markdown export.
const orphan = await files.upload({ id: ownerId } as never, pondId, {
buffer: Buffer.from(PNG_BASE64, 'base64'),
size: 70,
originalname: 'never-embedded.png',
} as never);
const res = await asBinary(api().get(`/api/v1/ponds/${pondId}/archive`))
.set('Cookie', ownerCookie)
.expect(200);
expect(res.headers['content-type']).toContain('application/zip');
const names = Object.keys(entries(res.body as Buffer));
expect(names).toContain('manifest.json');
expect(names).toContain('README.txt');
expect(names).toContain('pages/archive-parent.md');
expect(names).toContain('pages/archive-child.md');
expect(names.some((name) => name.startsWith(`media/${orphan.id}.`))).toBe(true);
const manifest = manifestOf(res.body as Buffer);
expect(manifest.attachments.map((a) => a.id)).toContain(orphan.id);
// The Markdown export would have shipped no media at all here.
expect(manifest.attachments.length).toBeGreaterThan(0);
});
it('states the hierarchy, labels, comments and settings in the manifest', async () => {
const res = await asBinary(api().get(`/api/v1/ponds/${pondId}/archive`))
.set('Cookie', ownerCookie)
.expect(200);
const manifest = manifestOf(res.body as Buffer);
expect(manifest.kind).toBe('dorfteich-pond-archive');
expect(manifest.formatVersion).toBe(1);
expect(manifest.complete).toBe(true);
expect(manifest.omittedPages).toBe(0);
const child = manifest.pages.find((page) => page.slug === 'archive-child');
// The hierarchy is exactly what a folder of Markdown cannot express.
expect(child?.parentId).toBe(parentPageId);
expect(manifest.labels.some((label) => label.name.includes(suffix))).toBe(true);
expect(manifest.comments.map((comment) => comment.body)).toContain('A remark worth keeping.');
// A display name, not an account id — the archive outlives the account.
expect(manifest.comments[0]?.author).toContain('Archive Owner');
// Pond settings ride along; fonts are always present through the schema
// defaults, so their presence proves the settings object is real.
expect(manifest.pond.settings).toHaveProperty('fonts');
});
it('tells a requester before the download how much they would get', async () => {
const preview = await api()
.get(`/api/v1/ponds/${pondId}/archive/preview`)
.set('Cookie', ownerCookie)
.expect(200);
expect(preview.body.omittedPages).toBe(0);
expect(preview.body.includedPages).toBe(preview.body.totalPages);
expect(preview.body.complete).toBe(true);
});
it('audits the download with counts and completeness', async () => {
await asBinary(api().get(`/api/v1/ponds/${pondId}/archive`))
.set('Cookie', ownerCookie)
.expect(200);
const entry = await prisma.auditEntry.findFirst({
where: { action: 'pond.archived', targetId: pondId },
orderBy: { at: 'desc' },
});
expect(entry).not.toBeNull();
expect(entry!.details).toMatchObject({ complete: true, omittedPages: 0 });
});
it('gives the Site Admin a complete archive without pond membership', async () => {
// The purge dialog's archive must not depend on which ponds the operator
// happens to be a member of — this admin is a member of none.
const preview = await api()
.get(`/api/v1/admin/trash/ponds/${pondId}/archive/preview`)
.set('Cookie', adminCookie)
.expect(200);
expect(preview.body.complete).toBe(true);
expect(preview.body.omittedPages).toBe(0);
const res = await asBinary(api().get(`/api/v1/admin/trash/ponds/${pondId}/archive`))
.set('Cookie', adminCookie)
.expect(200);
expect(manifestOf(res.body as Buffer).pages.length).toBe(preview.body.totalPages);
});
it('keeps the admin archive away from an ordinary pond admin', async () => {
await api()
.get(`/api/v1/admin/trash/ponds/${pondId}/archive`)
.set('Cookie', ownerCookie)
.expect(403);
});
});

View File

@ -0,0 +1,395 @@
import { Injectable, NotFoundException } from '@nestjs/common';
import {
PageClassification,
PondArchiveManifest,
PondArchivePreview,
POND_ARCHIVE_FORMAT_VERSION,
classificationMarking,
highestClassification,
pondSettingsSchema,
} from '@dorfteich/shared';
import { User } from '@prisma/client';
import archiver from 'archiver';
import type { Response } from 'express';
import { PinoLogger } from 'nestjs-pino';
import { AuditService } from '../audit/audit.service';
import { FileStorageService } from '../files/file-storage.service';
import { PermissionService } from '../permissions/permission.service';
import { PrismaService } from '../prisma/prisma.service';
import { ReadTrailService, type ReadActor } from '../read-trail/read-trail.service';
import { markClassifiedMarkdown } from './classified-markdown';
import { imageExtension, markdownForZip } from './export-markdown';
/** The plain-text note that travels inside the ZIP. The manifest says the
* same thing machine-readably, but a person unpacking a folder of Markdown
* a year from now reads the file lying next to it and must not believe
* they are holding a one-click restore. */
const README = `Dorfteich pond archive (format version ${POND_ARCHIVE_FORMAT_VERSION})
This is a PRESERVATION archive, not a backup you can re-import: Dorfteich has
no importer for it yet. Everything needed to write one later is here and
documented see manifest.json and docs/architecture/pond-archive-format.md in
the Dorfteich repository.
manifest.json pond settings, labels, page hierarchy, comments, attachment
metadata, and the classification of every file
pages/ one Markdown file per page
media/ EVERY attachment of the pond, not only the embedded ones
If manifest.json states "complete": false, the archive was produced by someone
who could not read every page of the pond; "omittedPages" says how many are
missing.
`;
/**
* The full pond archive offered before a pond is deleted (issue #305).
*
* Distinct from the Markdown export (`exportPond`, #65) on purpose: that one
* ships the pages plus the images they embed, which as a LAST resort is not
* enough an attachment nobody embedded would vanish unnoticed. This one adds
* every attachment and a machine-readable sidecar of the things Markdown
* cannot carry: settings, labels, comments and the page hierarchy.
*
* Re-import is deliberately out of scope. The archive is a preservation
* format: complete, versioned and documented, so an importer can be written
* later without guesswork.
*/
@Injectable()
export class PondArchiveService {
constructor(
private readonly prisma: PrismaService,
private readonly permissions: PermissionService,
private readonly storage: FileStorageService,
private readonly readTrail: ReadTrailService,
private readonly audit: AuditService,
private readonly logger: PinoLogger,
) {
this.logger.setContext(PondArchiveService.name);
}
/**
* Pages in the pond and how many of them this requester may read.
*
* The UI states the difference BEFORE the download: an archive silently
* missing content is worse than no archive, because it ends the search.
* A site admin archiving from the purge dialog reads everything, so their
* preview says nothing is omitted.
*/
async preview(user: User, pondId: string, unfiltered: boolean): Promise<PondArchivePreview> {
const pond = await this.loadPond(pondId);
const pages = await this.readablePages(user, pond.id, unfiltered);
const total = await this.prisma.page.count({ where: { pondId: pond.id, deletedAt: null } });
return {
totalPages: total,
includedPages: pages.length,
omittedPages: total - pages.length,
// "Complete" is a statement about the RESULT, not about the route: a
// pond admin who may read every page gets a complete archive too. Only
// an archive that actually leaves pages out is incomplete.
complete: total === pages.length,
};
}
/** The pond, or 404 — the caller's permission is checked by the route. */
private async loadPond(pondId: string): Promise<{ id: string; slug: string; name: string }> {
// Deliberately including trashed ponds: the purge dialog archives a pond
// that is already in the trash, which is the last moment it exists.
const pond = await this.prisma.pond.findUnique({
where: { id: pondId },
select: { id: true, slug: true, name: true },
});
if (!pond) throw new NotFoundException();
return pond;
}
private async readablePages(
user: User,
pondId: string,
unfiltered: boolean,
): Promise<
{
id: string;
slug: string;
title: string;
parentId: string | null;
sortKey: string;
classification: string;
createdAt: Date;
updatedAt: Date;
labels: { labelId: string }[];
contentCache: { markdown: string } | null;
}[]
> {
const pages = await this.prisma.page.findMany({
where: { pondId, deletedAt: null },
orderBy: { title: 'asc' },
select: {
id: true,
slug: true,
title: true,
parentId: true,
sortKey: true,
classification: true,
createdAt: true,
updatedAt: true,
labels: { select: { labelId: true } },
contentCache: { select: { markdown: true } },
},
});
if (unfiltered) return pages;
const readable = await this.permissions.filterPages(
user,
pondId,
pages.map((page) => ({ id: page.id, labelIds: page.labels.map((l) => l.labelId) })),
'read',
);
return pages.filter((page) => readable.has(page.id));
}
/**
* Stream the archive.
*
* `unfiltered` is the Site-Admin path from the purge dialog: it skips the
* read filter, because the last archive before an irreversible purge must
* not depend on which pages the operator happens to be a member of.
* Whether the RESULT is complete is a separate question, answered by
* comparing what went in with what exists.
*/
async stream(
user: User,
pondId: string,
res: Response,
read: ReadActor,
unfiltered: boolean,
): Promise<void> {
const pond = await this.loadPond(pondId);
const pages = await this.readablePages(user, pond.id, unfiltered);
const totalPages = await this.prisma.page.count({
where: { pondId: pond.id, deletedAt: null },
});
const complete = totalPages === pages.length;
// Read trail (ADR 0023, issue #222's property): one `export` event per
// classified page BEFORE any classified byte enters the stream, so a
// failed write aborts the download with the evidence intact. The added
// attachments carry their page's classification and are covered by the
// same events — they never travel without their page.
for (const page of pages) {
if (page.classification !== 'VS_NFD') continue;
await this.readTrail.record({
...read,
pageId: page.id,
pondId: pond.id,
channel: 'export',
details: { format: 'pond_archive' },
});
}
const [settingsRow, labels, comments, attachmentRows] = await Promise.all([
this.prisma.pond.findUniqueOrThrow({
where: { id: pond.id },
select: { name: true, slug: true, type: true, settings: true, createdAt: true },
}),
this.prisma.label.findMany({
where: { pondId: pond.id },
select: { id: true, name: true, color: true, parentId: true },
orderBy: { name: 'asc' },
}),
this.prisma.comment.findMany({
where: { page: { pondId: pond.id, deletedAt: null } },
orderBy: { createdAt: 'asc' },
select: {
id: true,
pageId: true,
parentId: true,
body: true,
createdAt: true,
editedAt: true,
resolvedAt: true,
author: { select: { displayName: true } },
},
}),
// EVERY attachment of the pond (issue #305) — not only the embedded
// ones the Markdown export ships.
this.prisma.attachment.findMany({
where: { pondId: pond.id },
select: {
id: true,
pageId: true,
fileName: true,
mimeType: true,
sizeBytes: true,
sha256: true,
createdAt: true,
},
orderBy: { createdAt: 'asc' },
}),
]);
const includedPageIds = new Set(pages.map((page) => page.id));
// An attachment of a page the requester cannot read stays out — the same
// rule the pages follow. Pond-level attachments (no page) are included:
// nothing narrower than the pond governs them.
const visibleAttachments = attachmentRows.filter(
(row) => !row.pageId || includedPageIds.has(row.pageId),
);
const onDisk = await Promise.all(
visibleAttachments.map((row) => this.storage.exists(pond.id, row.id)),
);
const attachments = visibleAttachments.filter((_, index) => onDisk[index]);
const classificationByPage = new Map(
pages.map((page) => [page.id, page.classification.toLowerCase() as PageClassification]),
);
const mediaName = new Map(
attachments.map((row) => [row.id, `${row.id}.${imageExtension(row.mimeType)}`]),
);
const readableSlugs = new Set(pages.map((page) => page.slug));
const archive = archiver('zip', { zlib: { level: 9 } });
res.set('Content-Type', 'application/zip');
res.set('Content-Disposition', `attachment; filename="${pond.slug}-archive.zip"`);
res.set('X-Content-Type-Options', 'nosniff');
archive.on('error', (error) => {
this.logger.error({ pondId: pond.id, err: error.message }, 'pond archive failed');
res.destroy(error);
});
archive.pipe(res);
const files: { path: string; classification: PageClassification }[] = [];
for (const page of pages) {
const level = classificationByPage.get(page.id) ?? 'unclassified';
const markdown = markClassifiedMarkdown(
markdownForZip(page.contentCache?.markdown ?? '', readableSlugs, mediaName),
level,
);
const path = `pages/${page.slug}.md`;
archive.append(markdown, { name: path });
files.push({ path, classification: level });
}
for (const row of attachments) {
// An attachment inherits its page's level (fail-closed, ADR 0022); one
// that belongs to no page inherits the pond's highest, because nothing
// narrower governs it.
const level = row.pageId
? (classificationByPage.get(row.pageId) ?? 'unclassified')
: highestClassification([...classificationByPage.values()]);
const path = `media/${mediaName.get(row.id)!}`;
files.push({ path, classification: level });
// Companion marking (issue #212): binaries cannot carry it themselves,
// and the sibling file survives unpacking where a manifest may not.
const marking = classificationMarking(level);
if (marking) {
archive.append(`${marking}\n`, { name: `${path}.classification.txt` });
files.push({ path: `${path}.classification.txt`, classification: level });
}
}
const manifest: PondArchiveManifest = {
kind: 'dorfteich-pond-archive',
formatVersion: POND_ARCHIVE_FORMAT_VERSION,
exportedAt: new Date().toISOString(),
complete,
omittedPages: totalPages - pages.length,
classification: highestClassification(files.map((file) => file.classification)),
pond: {
name: settingsRow.name,
slug: settingsRow.slug,
type: settingsRow.type,
createdAt: settingsRow.createdAt.toISOString(),
// The EFFECTIVE settings, defaults filled in — a preservation format
// must not require its reader to know Dorfteich's defaults, and the
// stored row only holds what was explicitly set.
settings: pondSettingsSchema.parse(settingsRow.settings ?? {}) as unknown as Record<
string,
unknown
>,
},
labels: labels.map((label) => ({
id: label.id,
name: label.name,
color: label.color,
parentId: label.parentId,
})),
pages: pages.map((page) => ({
id: page.id,
slug: page.slug,
title: page.title,
parentId: page.parentId,
sortKey: page.sortKey,
classification: page.classification.toLowerCase() as PageClassification,
labelIds: page.labels.map((label) => label.labelId),
createdAt: page.createdAt.toISOString(),
updatedAt: page.updatedAt.toISOString(),
file: `pages/${page.slug}.md`,
})),
// Comments of included pages only — a comment is content of its page.
comments: comments
.filter((comment) => includedPageIds.has(comment.pageId))
.map((comment) => ({
id: comment.id,
pageId: comment.pageId,
parentId: comment.parentId,
body: comment.body,
// The display name, not the account: the archive is a document, and
// it should stay readable after the account is gone.
author: comment.author?.displayName ?? null,
createdAt: comment.createdAt.toISOString(),
editedAt: comment.editedAt?.toISOString() ?? null,
resolvedAt: comment.resolvedAt?.toISOString() ?? null,
})),
attachments: attachments.map((row) => ({
id: row.id,
pageId: row.pageId,
fileName: row.fileName,
mimeType: row.mimeType,
sizeBytes: row.sizeBytes,
sha256: row.sha256,
createdAt: row.createdAt.toISOString(),
file: `media/${mediaName.get(row.id)!}`,
})),
files,
};
archive.append(README, { name: 'README.txt' });
archive.append(JSON.stringify(manifest, null, 2), { name: 'manifest.json' });
for (const row of attachments) {
const stream = this.storage.createReadStream(pond.id, row.id);
stream.on('error', (error) =>
this.logger.warn(
{ pondId: pond.id, fileId: row.id, err: error.message },
'pond archive: media read failed',
),
);
archive.append(stream, { name: `media/${mediaName.get(row.id)!}` });
}
// Audited: a whole pond leaving the instance in one file, usually right
// before it is deleted, is exactly the event an operator wants to find
// later. Recorded before finalize so the trail exists even if the
// download is aborted mid-stream.
await this.audit.record({
action: 'pond.archived',
actorId: user.id,
targetType: 'pond',
targetId: pond.id,
details: {
pages: pages.length,
attachments: attachments.length,
omittedPages: totalPages - pages.length,
complete,
},
});
await archive.finalize();
this.logger.info(
{ pondId: pond.id, pages: pages.length, attachments: attachments.length, complete },
'pond archive streamed',
);
}
}

Some files were not shown because too many files have changed in this diff Show More