Commit Graph

15 Commits

Author SHA1 Message Date
69d9072d2c #234: retention for mail_outbox
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m51s
CI / Build container images (pull_request) Successful in 3m4s
CI / Auth e2e pack (pull_request) Successful in 8m4s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
Sent mails were kept forever, and digest bodies name page titles and
actors — an unbounded copy of content-adjacent data. A new daily
mail-outbox-retention job deletes SENT rows (by sentAt) and permanently
FAILED rows (by nextAttemptAt, the last attempt's stamp) once they pass
mail.outboxRetentionDays (instance setting, default 30). PENDING rows —
including failed-but-retryable ones — stay the retry loop's alone.

Decision recorded (security.md §Privacy, residual-risk note for #231):
digest mails keep carrying page titles for now — there is no per-page
classification marking yet to key a suppression on (ADR 0022 / M32
revisits), and a VS-NfD reference configuration can leave SMTP
unconfigured entirely.

Job-count fence in system.spec: 8 -> 9.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 22:17:28 +02:00
ff505bc752 #233: prune conversion job payloads for every job kind
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m12s
CI / Build container images (pull_request) Successful in 3m28s
CI / Auth e2e pack (pull_request) Successful in 8m33s
CI / Import/export fidelity gate (pull_request) Successful in 1m2s
CD / Build and push images (push) Successful in 29s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m22s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Failing after 5m9s
CI / Auth e2e pack (push) Has been skipped
CI / Import/export fidelity gate (push) Has been skipped
CI / Build container images (push) Has been skipped
The raw input/result bytes of import/export conversion jobs were kept
forever; a deleted classified page could live on inside its last export.
A new daily conversion-payload-prune job nulls both once a finished
(succeeded or failed) job passes conversion.payloadRetentionDays
(instance setting, default 30) — the row survives for status/audit.
PENDING and RUNNING rows keep their payload, so the worker's stale-lock
recovery path is untouched; a hand-requeued pruned job fails finally
via conversionInputOf instead of crashing the worker.

The input column becomes nullable; the migration backfills by clearing
payloads of jobs already finished longer ago than the default period
(recent results stay downloadable until they age out).

Job-count fence in system.spec: 7 -> 8 (new scheduler registration).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 22:12:32 +02:00
ff842f97e2 #198: CI fence — no tracked .env or secret material, example is authoritative
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m27s
CI / Build container images (pull_request) Successful in 1m16s
CI / Auth e2e pack (pull_request) Successful in 7m49s
CI / Import/export fidelity gate (pull_request) Successful in 57s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 28s
CD / Smoke tests against Test (push) Successful in 1m32s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 5m14s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m41s
CI / Import/export fidelity gate (push) Failing after 10s
Verification result: only deploy/compose/.env.example was ever tracked
(full-history check), zero hits for obvious secret patterns across all
added lines in history — recorded on issue #231 (residual-risk list).

The new CI step in the checks job fails if any .env other than
.env.example is tracked or a tracked file matches an obvious secret
pattern (private key blocks, AWS/GitHub/GitLab/Slack token shapes).
.env.example already documents every variable the compose files
reference (verified: comm of compose ${VAR} refs vs example keys is
empty). README states the example as the authoritative reference.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 17:16:00 +02:00
3c62b7b773 #197: security response headers and an explicitly restrictive CORS policy
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m15s
CI / Build container images (pull_request) Successful in 1m9s
CI / Auth e2e pack (pull_request) Successful in 7m43s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m29s
CD / Promote to Int (push) Successful in 14s
CI / Lint, typecheck, test (push) Successful in 5m24s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m35s
CI / Import/export fidelity gate (push) Successful in 55s
Hand-rolled middleware instead of helmet: the header set is small enough
to own, every value is a deliberate decision, and the api gains no
transitive dependency. HSTS (no includeSubDomains — the api cannot speak
for sibling subdomains), nosniff, Referrer-Policy no-referrer,
X-Frame-Options SAMEORIGIN (not DENY: the plugin sandbox frame embeds
same-origin and its CSP has no frame-ancestors, so this header governs),
and a minimal deny-all Permissions-Policy.

CORS grants no foreign origin anything; only the APP_BASE_URL origin is
ever echoed (where browsers do not consult CORS anyway), with
Vary: Origin on every response. No preflight handling — same-origin
requests never preflight, and cross-origin API access is cookie-less by
design (PAT/Bearer).

Wired via the AppModule MiddlewareConsumer so createTestApp boots the
identical middleware. Fences: security-headers.e2e.test.ts (header set,
foreign origin gets no ACAO) and a frame assertion in
plugins.e2e.db.test.ts (framing stays possible). Rationale table in
security.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 17:13:24 +02:00
ed2225bb77 #196: audit-trail retention job
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m5s
CI / Build container images (pull_request) Successful in 2m48s
CI / Auth e2e pack (pull_request) Successful in 7m50s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 15s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m20s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m11s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m38s
CI / Import/export fidelity gate (push) Successful in 56s
audit.retentionDays (instance setting, default 365) bounds the audit_log:
the daily audit-retention job deletes entries past the period and records
the deletion itself (audit.pruned with count, cutoff and period) so a gap
in the trail is always explainable. Lives in its own AuditRetentionService
because the settings service audits its writes - folding retention into
AuditService would close a constructor cycle. The read-access trail
(#222-#225) is deliberately not covered; it gets its own period.

security.md gains the Logging section the schema has cited for a while;
the maintenance-job fence moves 6 -> 7 (the deliberate new row).

Refs #196

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 14:59:28 +02:00
960a806ee3 #195: trashed content leaves the search index itself
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m4s
CI / Build container images (pull_request) Successful in 2m47s
CI / Auth e2e pack (pull_request) Successful in 7m44s
CI / Import/export fidelity gate (pull_request) Successful in 55s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Successful in 1m22s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m9s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m53s
CI / Import/export fidelity gate (push) Successful in 53s
Trashing a page (promote and subtree modes) clears the affected search
vectors, restoring rebuilds them; pond trash clears every page vector of
the pond, pond restore reindexes only the live pages (pages trashed
inside stay out); the GDPR pseudonymization's personal-pond trash does
the same. reindexAll now converges to the invariant (clears trashed,
rebuilds live), and a one-off migration backfills vectors of
already-trashed content.

The query-side deleted_at guards stay untouched as the independent
second layer - the test proves both layers separately, including writing
a vector back onto a trashed page (simulating a future path that forgot
the clear) and asserting the query still hides it. New provider methods
removePond/reindexPond behind the SearchProvider seam.

Refs #195

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 14:07:34 +02:00
0bc36aa58c #194: orphan-file sweep, drop the unused Attachment.deletedAt
Some checks failed
CI / Lint, typecheck, test (pull_request) Successful in 5m1s
CI / Build container images (pull_request) Successful in 2m48s
CI / Auth e2e pack (pull_request) Failing after 3m12s
CI / Import/export fidelity gate (pull_request) Has been skipped
Nightly sweep with two directions: attachments still unclaimed (pageId
null) after a 24 h grace period - claimed by no collab persist, page
upload, or import - are reclaimed (row, file, quota released); files on
the uploads volume without a database row (drift after a crashed
upload) are removed once older than the grace period. The grace period
protects the paste-then-insert window.

Deliberate deviation from the issue's content-reference idea, documented
in schema comment and operations.md: claimed attachments whose page
content no longer embeds them are NOT auto-deleted. The page attachments
panel lists claimed files as user-managed objects (inserting into the
document is optional there), so 'not embedded' is not 'unused' - an
auto-delete would destroy panel assets. Humans clean those up in the
panel or the pond file manager, which flags orphans already.

Attachment.deletedAt is removed by migration - deletion is hard
everywhere (sweep, purge, manual), there is no soft-delete state; the
never-true deletedAt:null filters in files/export queries went with it.

Refs #194

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 13:19:58 +02:00
402b22e05f #193: pond purge — retention job and manual Site-Admin endpoint
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 4m58s
CI / Build container images (pull_request) Successful in 2m47s
CI / Auth e2e pack (pull_request) Successful in 7m46s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 28s
CD / Smoke tests against Test (push) Successful in 1m20s
CD / Promote to Int (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 5m4s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m38s
CI / Import/export fidelity gate (push) Successful in 56s
Deletion now actually deletes: a trashed pond past the trash retention
(same clock as pages, extended trash-purge job) or purged manually via
DELETE /ponds/:id/purge (Site-Admin-only, like pond restore) is removed
with everything it holds. Files go first (idempotent rm, resumable on a
crash), then one transaction ordered around the FK actions: attachments
and labels (Restrict) precede the pond; the page delete cascades
versions, comments, content cache incl. the search vector, update log,
mentions, label assignments, favorites, outgoing links and open collab
sessions; the pond delete cascades grants, usage counters (that is the
quota correction), pond-plugin opt-ins and conversion jobs; polymorphic
watches and pond quota overrides are deleted explicitly. A purge racing
a restore or another purge is a no-op; both paths record a pond.purged
audit event.

Known residues by design, documented in operations.md: target_slug in
other ponds' page links (#235) and backups within their retention.

Refs #193

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 12:44:52 +02:00
394d1c811d #192: deploy-level backup target allowlist
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 4m52s
CI / Build container images (pull_request) Successful in 3m54s
CI / Auth e2e pack (pull_request) Successful in 8m4s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Successful in 1m14s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m0s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m41s
CI / Import/export fidelity gate (push) Successful in 56s
BACKUP_ALLOWED_TARGETS (comma-separated destination hosts) constrains
where backups may go, enforced twice: the api rejects settings writes
and connection tests towards non-allowlisted hosts with admin-visible
error codes and resolves a non-allowlisted configured target to null,
and the sidecar enforces the same policy at the point of egress for the
WebDAV upload and the rsync mirror alike (shared policy helpers in
packages/shared/src/backup-target-policy.ts).

BREAKING: the empty default disables every remote target - backups stay
local only, the VS-NfD reference configuration (ADR 0026). Existing
deployments with a remote target must list its host or uploads and
mirror stop. The admin UI distinguishes unavailable-by-policy from
unconfigured (i18n de+en) and shows the permitted hosts.

Refs #192 (ADR 0026)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 12:17:14 +02:00
afef45732a #191: feeds.enabled instance switch, feed-token log masking
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 4m52s
CI / Build container images (pull_request) Successful in 3m55s
CI / Auth e2e pack (pull_request) Successful in 7m52s
CI / Import/export fidelity gate (pull_request) Successful in 55s
CD / Build and push images (push) Successful in 16s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m14s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 4m55s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m34s
CI / Import/export fidelity gate (push) Successful in 58s
Chosen path: an instance master switch following the api.enabled/
mcp.enabled pattern — while off, both feed routes AND the feed-token
management answer 404 (existence hidden). Default ON: feeds predate the
switch, existing instances and their subscribed readers keep working;
the VS-NfD reference configuration (#227) turns it off. Admin UI gets
the toggle next to the API/MCP switches (i18n de+en).

Moving the token out of the query string is documented as rejected: a
path segment lands in the same proxy and request logs, and feed readers
cannot send headers — that is why the credential is in the URL at all.
What DID leak was our own request log (pino logs req.url): the req
serializer now masks ?token= values (common/mask-token-param.ts), so no
code path logs the credential.

Refs #191

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 11:34:43 +02:00
db4c5ce9ca #190: configurable session lifetime with a server-side idle timeout
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 4m53s
CI / Build container images (pull_request) Successful in 3m55s
CI / Auth e2e pack (pull_request) Successful in 7m53s
CI / Import/export fidelity gate (pull_request) Successful in 55s
CD / Build and push images (push) Successful in 19s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 4m53s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m37s
CI / Import/export fidelity gate (push) Successful in 52s
SESSION_ABSOLUTE_HOURS (default 168 h) caps a session's total lifetime
from login: expiresAt is set once at creation and never extended — the
old sliding 30-day renewal is gone. SESSION_IDLE_HOURS (default 72 h)
ends sessions unused for that long, enforced server-side against
lastSeenAt with a write throttle scaled to the idle bound so short idle
windows still renew. Expired rows are removed on validation and the
session list applies both bounds, so idle-dead sessions never show as
active. The cookie maxAge follows the configured absolute bound.

Documented in .env.example (with the VS-NfD reference values for the
upcoming hardening guide #227), compose passes the variables through,
security.md and ADR 0007 record the amendment.

Refs #190

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 11:11:15 +02:00
d32c8c3730 #189: make the CSRF origin check fail closed
Some checks failed
CI / Lint, typecheck, test (pull_request) Failing after 1m51s
CI / Auth e2e pack (pull_request) Has been skipped
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Build container images (pull_request) Has been skipped
A cookie-carrying mutation without Origin and Referer (or with an
unparsable one) is now rejected with 403 csrf_origin_mismatch instead
of passing unchecked. The exception for non-browser clients stays
structural: PAT/bearer requests carry no session cookie and never reach
the check, and a request that does carry the cookie is always checked.

The test harness injects the matching Origin (supertest simulates a
browser page of this instance) with an explicit suppression header for
the negative cases; the Playwright fixture contexts send the header on
their manual seeding calls; release-qa.sh pins APP_BASE_URL and sends
the matching Origin. Dedicated spec covers: missing headers 403,
mismatch 403, unparsable 403, match passes, GETs untouched, PAT
mutation without headers passes, cookie+bearer still checked.

Refs #189

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 09:34:44 +02:00
3d1f4fda53 #188: purpose-bound token keys via HKDF, jose replaces the homegrown JWT
All checks were successful
CI / Build container images (pull_request) Successful in 3m51s
CI / Auth e2e pack (pull_request) Successful in 7m49s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CI / Lint, typecheck, test (pull_request) Successful in 4m43s
CD / Build and push images (push) Successful in 20s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m19s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 4m54s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m39s
CI / Import/export fidelity gate (push) Successful in 59s
COLLAB_TOKEN_SECRET becomes a root key: every purpose derives its own
HKDF-SHA-256 subkey (deriveTokenKey), and no code path signs with the
root key directly. Collaboration tokens are signed and verified by jose
with HS256 as an explicit allowlist; the sign/verify API turns async at
its three call sites. Unsubscribe tokens move from a purpose-prefix
string to the structural subkey, with a documented dual-verify window
(legacy derivation accepted until 2026-11-01, covering the 90-day TTL
of links in already-sent mail).

The cross-runtime property that justified the homegrown implementation
is now proven by a test: the built CJS and ESM dist artefacts round-trip
tokens in both directions in child processes (jose v6 reaches CJS via
Node's require(esm), pinned Node 22 images). Negative tests cover
cross-purpose subkeys, root-key-signed tokens, alg:none and RS256.

Refs #188 (ADR 0020)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 06:41:11 +02:00
3e377aaa57 #226: add the §52 VSA delimitation statement
Some checks failed
CI / Build container images (pull_request) Successful in 1m10s
CI / Auth e2e pack (pull_request) Successful in 7m46s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Deploy to Test (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (pull_request) Successful in 4m43s
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CI / Lint, typecheck, test (push) Has been cancelled
Reviewer-facing document derived from ADR 0019: per base function
(encryption, media protection, network termination, authentication,
integrity) what the application does, what it deliberately does not,
and which party provides it — every claim traceable to code via the
ist-aufnahme. Includes the operator-duty handover (with the IndexedDB
endpoint copy named explicitly, I-25) and the delta list mapping every
divergence from the target state to its closing issue.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 06:29:05 +02:00
fd07f716f6 docs: VS-NfD readiness planning (ist-aufnahme, plan, ADRs 0019-0026, issue drafts)
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 4m42s
CI / Build container images (pull_request) Successful in 1m11s
CI / Auth e2e pack (pull_request) Successful in 7m47s
CI / Import/export fidelity gate (pull_request) Successful in 55s
CD / Build and push images (push) Successful in 18s
CD / Deploy to Test (push) Successful in 14s
CD / Smoke tests against Test (push) Successful in 1m16s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 4m50s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m38s
CI / Import/export fidelity gate (push) Successful in 56s
Add docs/vs-nfd/: the analysis brief, the as-is assessment (42 findings,
all verified against the code), the prioritized action plan rev. 2 with
issue references written back to every checkbox, the two-stage issue/ADR
brief, and the full reviewed draft used to create the forge state.

Add eight proposed ADRs 0019-0026 covering the VS-NfD architecture
decisions: no security base functions (par. 52 VSA anchor), HKDF token
key separation, external authentication, page classification, read-access
audit trail (variant A), reproducible offline deployment, plugin trust
model, and backup target restriction.

Forge state created alongside this commit: 11 labels, milestones M24-M31,
issues #188-#236 (docs-only change, no code touched).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
2026-07-30 01:48:05 +02:00