0428892ef2
24 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 8ae010218e |
Obsidian vault import: endpoint, job orchestration, rollback (#117)
Some checks failed
CD / Build and push images (push) Successful in 4m5s
CD / Deploy to Test (push) Successful in 9s
CI / Lint, typecheck, test (push) Failing after 4m21s
CI / Auth e2e pack (push) Has been skipped
CI / Import/export fidelity gate (push) Has been skipped
CI / Build container images (push) Has been skipped
CD / Smoke tests against Test (push) Successful in 1m22s
CD / Promote to Int (push) Successful in 11s
POST /ponds/:pondId/import/vault (pond-admin-gated; a vault import
creates a subtree, uploads files, and creates labels — administration,
not everyday editing) takes the ZIP plus a JSON options field
{parentPageId?, labelIds?, frontmatterMode}. The archive is parsed at
enqueue for fast 400s; the job (new kind import_vault, riding the
existing isImportKind worker routing) re-parses and runs the #116
transform, then: containers top-down → notes (asset placeholders →
uploaded pond files; non-images become page attachments) → tags to
labels (nested tags build a label hierarchy via LabelsService, so
locking and cache invalidation apply) plus the dialog labels.
All-or-nothing: any failure hard-deletes the created pages (children
first) and removes the stored files (quota restored), then surfaces as
import_vault_invalid_zip / import_vault_too_large / quota_exceeded /
conversion_failed — and makes the worker's retry policy safe.
Supporting changes:
- conversion_jobs gains a nullable options jsonb column; enqueue takes
kind-specific options and a maxInputBytes override (the 25 MiB
default protects the pandoc sidecar, which a vault never touches —
vaults use the 64 MiB upload limit).
- insertPage accepts a pre-reserved slug (the batch reserves all slugs
up front against pond ∪ batch).
- NEW: pages born with content seed their outgoing page_links rows
(deriveContent now returns wikilinkSlugs) — imported pages would
otherwise stay invisible to backlinks and the graph until their
first collab save. Collab still rewrites the rows on every save, and
the existing phantom resolution heals batch creation order.
import-vault.e2e.db.test.ts (4 tests, real worker drained): gating +
input rejection, the full fixture import (tree under a mount page,
collision suffixes, link rows incl. phantom, nested tag labels, extra
label everywhere, frontmatter stripped, image embedded + PDF attached),
complete quota rollback, and a clean re-import with fresh suffixes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
|||
| eb6b0d5d02 |
Page hierarchy: parentId, create-under-parent, reparent (#106)
Some checks failed
CI / Import/export fidelity gate (push) Blocked by required conditions
CD / Build and push images (push) Successful in 3m53s
CD / Deploy to Test (push) Successful in 12s
CI / Lint, typecheck, test (push) Successful in 4m16s
CI / Build container images (push) Has been skipped
CD / Smoke tests against Test (push) Successful in 1m19s
CD / Promote to Int (push) Successful in 14s
CI / Auth e2e pack (push) Has been cancelled
Pages form a tree via a nullable parent_id self-relation (SetNull backstop; the real trash/purge semantics follow with #107). Slugs and URLs stay flat and pond-unique, so moving a page never breaks links. - Shared: generic parent-id tree helpers in tree.ts (labels re-export them; buildLabelTree keeps its name-sorted behavior), MAX_PAGE_DEPTH=6, parentId on PageView, createPageInputSchema.parentId (nullish), repositionPageInputSchema.parentId (optional; absent = keep parent). - API: create validates the parent (same pond, live, depth); PATCH /pages/:id/position reparents atomically with the placement, rejecting cycles (page_cycle) and depth violations (page_depth_exceeded); GET /ponds/:id/pages nulls parentId when the caller may not read the parent, so hidden page ids never leak. - New error codes translated de+en; hierarchy.db.test.ts covers create, 404s, depth, cycle, atomic reparent, and the permission nulling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
|||
| 0c85293830 |
Public REST API v1: personal access tokens, instance switch + per-pond opt-in (#104)
Some checks failed
CI / Lint, typecheck, test (push) Failing after 1m39s
CI / Auth e2e pack (push) Has been skipped
CI / Import/export fidelity gate (push) Has been skipped
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m51s
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 11s
Token-authenticated machine access at /api/public/v1 — the foundation for the built-in MCP endpoint (#105). Personal access tokens: - api_tokens table (SHA-256 hash, scope read|write, optional pond restriction, expiry, revocation, throttled last-used) + migration; secrets are dt_pat_<random>, shown exactly once - lifecycle endpoints under /users/me/api-tokens (session-only — a leaked token can never mint more tokens) with audit entries api.token_created/api.token_revoked - settings UI section (create with scope/expiry/pond restriction, one-time reveal with copy, list with status + revoke), de+en Activation (404 semantics per #60 on both levels): - instance setting api.enabled (default off, admin settings switch) - pond setting apiEnabled (default off, pond settings toggle; the PondsService settings-merge learned the key — the #92 lesson) Surface (/api/public/v1, excluded from the SPA's global prefix): - me, ponds, pages (list/read as Markdown+HTML, create from Markdown via the shared pipeline, PATCH title/content, DELETE to trash), search (permission-filtered + narrowed to exposed ponds, highlights as **…**), markdown ZIP export, labels (tree, create/rename/recolour/move/delete, assign/unassign), comments (threads, create, resolve/reopen) - content replacement travels the collab-owned document path: the new state lands as a MANUAL version "API update", then the established restore NOTIFY applies it — open editors converge, history stays append-only, no second lineage (VersionsService.replaceContent) - hand-maintained OpenAPI 3.1 document at /openapi.json, pinned to the controller by a route-coverage test in both directions Enforcement: - PublicApiGuard: instance switch → bearer PAT auth (request.user is the token's user) → per-token rate limit (429 + Retry-After) → scope (403 scope_required) → pond opt-in + token restriction - the shared PermissionGuard then applies the unchanged permission model; PageParamSource gained pondSlugParam for the slug+slug routes - no cookies anywhere → no CSRF surface (pinned by a hostile-Origin test) - every write audit-logged as api.write with the token attributed Tests/verification: - 12-test e2e pack: lifecycle, switches, permission matrix (reader/editor/outsider × scopes), restriction, page roundtrip incl. restore-NOTIFY assertion, labels, comments incl. policy, search narrowing, ZIP export, rate limit; full api suite 60/60 green (quota fixture via per-user override — never the instance default) - new collab-pack test proves an open editor converges onto an API content replacement (green against a local seeded stack) - UI smoke against the built SPA: token create/reveal/revoke, pond opt-in persists, admin switch persists (10/10) - docs/self-hosting/public-api.md + README link Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 5cef359b8f |
Nextcloud backup target: admin-configured, manual + scheduled uploads, in-app restore (#103)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m45s
CD / Build and push images (push) Successful in 3m49s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m35s
CI / Import/export fidelity gate (push) Successful in 47s
Off-host backups for every self-hoster, configured entirely in the admin UI — supersedes the host-specific mirror plan behind #84. shared: - webdav.ts (new package entry like token-crypto): minimal WebDAV client with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT (streamed), GET, DELETE; Nextcloud DAV path derived from the plain server URL, explicit DAV bases pass through - backup-status.ts: additive remote-upload status in status.json, the restore-status.json contract (running/succeeded/failed + staleness bound), the backup_command/backup_maintenance NOTIFY channels, and the one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz) - backup-set.ts moved here from apps/backup (api lists local sets) backup sidecar: - reads the backup.* instance settings directly from the database (admin changes apply next run; local retention row overrides the env) and the app password from the secret store - after each successful set: bundle dump + files archive + manifest into ONE self-contained tar.gz, upload via WebDAV per schedule (off/daily/weekly; manual runs always upload), prune remote bundles — never the newest — and record the outcome in status.json; upload failures alert via a new backupUploadFailed mail (de+en) - command listener on backup_command (run / restore) with a serial queue against the nightly timer - restore orchestrator: restore-status.json → maintenance NOTIFY → grace → (remote: download + manifest-verify bundle) → terminate other DB connections → shared perform-restore path (same code as restore.sh) → final status + maintenance exit api: - MaintenanceGuard (global, registered before the setup gate): 503 maintenance_mode while restore-status says running; health endpoints and the new public GET /backup/restore-status stay exempt; a stale running state (crashed sidecar) unblocks after 30 min - MaintenanceStateService watches the file and restarts the api after a successful restore (fresh caches, migrate-on-start for older dumps); main.ts refuses to touch the database while a restore runs — a container restarting mid-restore must not race pg_restore with migrate deploy - worker sweeps (conversion, mail outbox, scheduler) catch transient database failures instead of dying on an unhandled rejection — the restore's connection termination crashed the api in verification - backup admin endpoints under /admin/system/backup: settings (live connection test before save, password write-only into the secret store), nextcloud/test, sets (local via the ro backups mount + remote via WebDAV), run + restore (type-to-confirm backstop, source validation) — commands travel as NOTIFY payloads; audit actions backup.settings_changed/run_triggered/restore_requested - readyz: new warning-level backup_remote check while a target is configured (26 h daily / 170 h weekly bound) collab: - maintenance listener: on enter, persist + close every live session and refuse new connections until exit (failsafe timeout 30 min) — no in-memory document may write pre-restore content back afterwards web: - Admin → System backup section: status card with remote facts and a "Back up now" button, the Nextcloud settings form with test button, and the restore picker (local + remote sets, type-to-confirm) - global maintenance screen: any 503 maintenance_mode flips the SPA to a status page polling the exempt endpoint, reloading when the instance returns Verified end-to-end against a live stack (fresh DB, native api + sidecar, fake WebDAV server): configure → test → manual backup → bundle upload → readyz/sets/status surfaces → remote restore with maintenance gate, marker rollback and api restart; suites: shared 21, backup 9, collab 11, api 58 files green, lint + i18n:check + typecheck clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 4549d6d13f |
Add threaded page comments: data model, API, and comment policy (#91)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m19s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 4m55s
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m5s
CD / Promote to Int (push) Successful in 10s
CI / Auth e2e pack (push) Successful in 5m20s
CI / Import/export fidelity gate (push) Successful in 45s
New comments table (thread via parent_id to the root, optional document anchor on roots, resolved_at/by; page purge cascades, trash hides) with a CommentsService enforcing the permission model: reading follows page read, writing follows the new pond setting commentPolicy (readers | editors) — 404 hides unreadable pages, 403 marks a failed write policy. Endpoints: threaded list per page with an open/resolved filter (resolved threads arrive collapsed by default), create (root or reply — replies attach to roots only and carry no anchor), edit own, delete own (roots with replies are admin-only, cascade), resolve/unresolve on roots for everyone who may comment. Bodies are Markdown rendered through the shared sanitizing pipeline; smuggled markup arrives as escaped text (fixture test). The UI lands with #92. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| f0a82bad20 |
Add the first-run setup wizard API with env-backed secret store (#80)
Some checks failed
CD / Build and push images (push) Successful in 3m16s
CI / Lint, typecheck, test (push) Successful in 3m5s
CD / Deploy to Test (push) Successful in 13s
CD / Smoke tests against Test (push) Failing after 3m35s
CD / Promote to Int (push) Has been skipped
CI / Auth e2e pack (push) Successful in 5m6s
CI / Import/export fidelity gate (push) Successful in 43s
CI / Build container images (push) Has been skipped
When the api runs against a database without the setup.completedAt marker, a global SetupGuard answers every non-exempt route with 503 setup_required; only /setup/*, health probes, and the session routes stay reachable. The wizard steps (POST /setup/admin|instance|smtp| registration|complete) write straight to their production homes; the Site Admin step signs its creator in, later steps require that session. Completing sets the marker and locks every step permanently (410, also across restarts, and not reopenable via PATCH /admin/settings). SMTP entered in the wizard is verified with a live delivery test first (failure blocks the step with the transport error as detail) and then persisted to the new env-backed secret store: a mode-600 dotenv file on the new `secrets` volume (SECRETS_FILE). Explicit container env always wins over the store; empty compose-passed strings count as unset. The mail transport now resolves lazily through SmtpConfigService so wizard changes apply without a restart. SETUP_ADMIN_* env pre-seeds the whole wizard at boot for automated deploys; a backfill migration marks instances that already have a Site Admin as completed, and seed/vitest global-setup do the same for fixture databases. The setup e2e suite provisions its own fresh database (CREATE DATABASE + migrate deploy) per run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 46292c7447 |
Add plugin administration UI: instance modes and pond activation (#72)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 2m56s
CD / Build and push images (push) Successful in 3m11s
CD / Deploy to Test (push) Successful in 8s
CI / Auth e2e pack (push) Successful in 4m12s
CD / Smoke tests against Test (push) Successful in 1m6s
CD / Promote to Int (push) Successful in 13s
CI / Import/export fidelity gate (push) Successful in 42s
CI / Build container images (push) Has been skipped
- api: PATCH /admin/plugins/:id/mode (Site Admin) switches disabled/optional/required; new PluginPondController exposes GET /ponds/:id/plugins (effective list: required + optional-enabled, pond read access — the SPA loads it per pond), GET .../plugins/settings and PUT .../plugins/:pluginId (Pond Admin) to toggle optional plugins. Toggling a non-optional plugin is refused (plugin_not_optional). Install/uninstall/mode/toggle are audit-logged. - web: PluginManager in the admin area lists installed plugins with their declared permissions surfaced prominently (security.md), an upload control that shows validation errors, a mode switch with an impact hint, and a link to the sandbox preview. PondPluginSettings adds a per-pond optional-plugin toggle section to pond settings. - shared: PondPluginSetting, mode/toggle input schemas, plugin_not_optional error code + de/en messages, plugins i18n (admin/mode/pond). - tests: api db test covers mode switching, per-pond activation, the required-everywhere and disabled-nowhere propagation, and the not-optional guard; e2e plugin-admin pack drives the admin list, permission display, mode switch, and pond toggle end to end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 621aa47244 |
Add plugin storage, install API, and directory watcher (#71)
Some checks failed
CI / Auth e2e pack (push) Waiting to run
CI / Import/export fidelity gate (push) Waiting to run
CI / Build container images (push) Waiting to run
CD / Build and push images (push) Failing after 1m33s
CD / Deploy to Test (push) Has been skipped
CD / Smoke tests against Test (push) Has been skipped
CD / Promote to Int (push) Has been skipped
CI / Lint, typecheck, test (push) Has been cancelled
Backend for installing plugin ZIPs (ADR 0008, plugin-architecture.md §Lifecycle, security.md §Plugins). Consumes the #70 SDK for validation. - Schema: `plugins` (id, name, version, apiVersion, kind, mode, manifest jsonb, removedAt soft-delete) + `pond_plugins` (per-pond activation) + `PluginInstanceMode` enum; migration 20260710130000_plugins. - `PluginPackageService`: pure, stateless ZIP → validated package via fflate — structure check, manifest validation (SDK), apiVersion gate, kind/bundle/styles rules, CSS sanitation (no @import / external url() / expression()), zip-slip and unpacked-size guards. Each failure carries a stable PluginErrorCode; manifest issues travel as ApiError details. - `PluginStorageService`: on-disk layout `<PLUGINS_DIR>/<id>/<version>/`; atomic writeVersion (staging dir + rename, no 404 window mid-update), removeVersion/removePlugin, traversal-safe asset resolution, dropzone + quarantine dirs. - `PluginsService`: install/update (update only to a strictly higher version, preserving the admin's instance mode; files land before the metadata pointer flips) / uninstall (refused while required; soft-delete + files removed + pond activations dropped) / list / get. - `POST/GET/DELETE /admin/plugins` (SiteAdminGuard, multer memory upload), error→HTTP-status mapping. Public version-pinned static serving at `GET /plugins/:id/:version/*rest` with immutable cache + nosniff, only for the installed current version. - `PluginWatcherService`: watches `<PLUGINS_DIR>/_dropzone/`, runs the same validation, installs valid drops and quarantines invalid ones with the error logged; inert under NODE_ENV=test (tests drive processDropped). - SDK: `compareVersions`/`isHigherVersion`. shared: `PluginView`, `PluginInstanceMode`, `PLUGIN_ERROR_CODES`, `PLUGINS_DIR` env, plugin error i18n (de+en). Compose: `plugins` volume + `PLUGINS_DIR`. - Tests: package unit test (valid + each invalid class) and an e2e DB test (GUI install + immutable serving, non-admin 403, invalid-manifest details, dropzone install + quarantine, atomic higher-only update, required-guarded uninstall that removes files and tombstones metadata). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 8a68ef68e7 |
Add PDF export via Gotenberg (#67)
All checks were successful
CD / Build and push images (push) Successful in 4m3s
CI / Lint, typecheck, test (push) Successful in 3m5s
CI / Auth e2e pack (push) Successful in 4m7s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m17s
CD / Promote to Int (push) Successful in 12s
Server-side PDF export for reading/sharing (ADR 0009), rendered by a new internal Gotenberg (headless Chromium) sidecar. - Sidecar: `gotenberg/gotenberg:8` in the compose stack (internal, pinned, healthcheck); api `GOTENBERG_URL` env; a `renderer` readyz check at warning-level (mirrors the converter) so PDF export degrades gracefully when Gotenberg is down without failing readyz. - Export HTML: `buildPdfHtml` renders a self-contained document (no app chrome) — the page's content with images inlined as data URIs, the pond's fonts inlined as base64 `@font-face` + applied via CSS variables (ADR 0016), print CSS (A4, page-break rules, a title header), and page numbers from Gotenberg's footer. Plugin-block fallbacks are a marked TODO(#79) for M7. - Fonts in the api image: the api Dockerfile now bakes the font catalog in (`build-fonts.mjs` with FONTS_OUT) so the exporter can read a pond's chosen WOFF2 and inline them; a missing file falls back to the system stack. - Job flow: `POST /pages/:id/export {format: pdf}` builds the HTML (read permission checked by the guard) and enqueues an `export_pdf` job on the #62 queue with the HTML as input; the worker branches `to === 'pdf'` to the `GotenbergRenderer` (html → pdf) instead of pandoc, retrying an unreachable sidecar and failing a refused render (`renderer_unavailable`/`render_failed`, de+en). The client polls and downloads `GET /jobs/:id/result`. - Frontend: the page-menu PDF button is now a real export (PDF added to EXPORT_FORMATS; the disabled placeholder removed). - Tests: export.service.db PDF cases (HTML has title/font-variable/inlined image; renderer-down fails with `render_failed`); e2e PDF export self-skips without a Gotenberg sidecar (like the .docx case). Verified locally against real Gotenberg — a valid PDF with the pond font embedded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 546e8279ac |
Import .docx and .odt documents as new pages (#63)
All checks were successful
CD / Build and push images (push) Successful in 3m19s
CI / Lint, typecheck, test (push) Successful in 2m55s
CI / Auth e2e pack (push) Successful in 3m45s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
Uploading a Word/OpenOffice document to POST /ponds/:id/import enqueues a conversion job (the #62 queue) that produces a new page in the pond; the client polls GET /jobs/:id for the created resultPageId. Pipeline (ImportService, ADR 0009): pandoc-server is stateless and hands back a document's media no other way, so we convert in two passes — docx/odt → html with embed-resources inlines every image as a data: URI, then html → gfm produces clean structural Markdown with those data URIs still inline. Embedded images are stored as pond files (with quota accounting) and their references rewritten to file ids on the Markdown text before parsing (the editor parser only admits png/jpeg/gif/webp data URIs); an image whose bytes the upload pipeline rejects is dropped, not fatal. The title comes from a leading top-level heading (removed from the body) else the file name. The page is created from the resulting Yjs state. The shared conversion worker routes import-kind jobs to the pipeline via a token (breaking a module cycle), so import inherits the queue's locking, retry, and restart-survival. Media stored during a failed attempt is rolled back; a pond that runs out of storage fails the job with quota_exceeded. - schema: ConversionJob gains pond_id / source_name / result_page_id (migration 20260710041215_import_pages_conversion); ConversionJobView gains resultPageId. - PagesService.createWithState / yjs-content docToState build a page from a prepared document; FilesService.linkAttachmentsToPage links import media. - fixtures/import/: representative .docx/.odt corpus (headings, lists, nested lists, tables, images, links, bold/italic) with expected-Markdown snapshots; scripts/gen-import-fixtures.mjs regenerates them. - tests: import.service.db.test.ts drives the full pipeline with a fake converter (CI); import.fixtures.test.ts runs the real two-pass conversion over the corpus and a 50-page timing check against a reachable sidecar. - i18n: import_unsupported_format (de+en). Limits documented (25 MiB input, 60 s per pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 4755c18ef5 |
Add conversion job queue and pandoc sidecar integration (#62)
All checks were successful
CD / Build and push images (push) Successful in 4m9s
CI / Lint, typecheck, test (push) Successful in 2m50s
CI / Auth e2e pack (push) Successful in 3m44s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m15s
CD / Promote to Int (push) Successful in 11s
Import/export conversions run asynchronously against an internal pandoc-server sidecar with limits and graceful failure (ADR 0009). This is the plumbing; the import (#63) and export (#65) features enqueue jobs onto it. Sidecar & config: - pandoc/core:3.6 in HTTP server mode added to the Compose stack, internal network only, with a wget healthcheck on /version; the api depends on it healthy and reaches it via the new PANDOC_URL env (default http://pandoc:3030). - readyz gains a warning-level `converter` check: an unreachable sidecar degrades import/export but never flips the instance to unready (new `warn` status on ReadinessCheck). Conversion flow (apps/api/src/import-export/): - ConversionJob table (per-request work queue, distinct from the name-keyed maintenance Job table): owner, formats, input/result bytes, status, attempts, lockedAt. Migration + owner cascade. - PandocConverter (abstract) + PandocServerConverter: POST / with {text,from,to,standalone}; binary input formats (docx/odt/…) are base64-encoded in `text`; 60 s AbortController timeout; input/output size caps. Failures map to distinct localized codes — converter_unavailable / converter_timeout (retryable) and conversion_failed (final). - ConversionWorker: claims one job at a time with `FOR UPDATE SKIP LOCKED` (safe against overlapping sweeps and a second process), recovers a stale RUNNING lock, retries transient failures up to 3 attempts then fails. A 2 s sweep plus wake-on-enqueue means a queued job survives an API restart. - ConversionJobService.enqueue (size-limited) + owner-scoped GET /jobs/:id (poll) and GET /jobs/:id/result (stream the output); a foreign/unknown id is 404. ConversionJobView in @dorfteich/shared. Tests: - conversion-job.e2e.db.test.ts (fake converter injected via a new createTestApp override hook): enqueue→convert→poll→result; foreign/unknown job 404; a persisted PENDING job picked up by a fresh app's worker (restart survival); sidecar-down fails after 3 retries while the API stays healthy. - pandoc.converter.test.ts: success, non-200→conversion_failed, refused→ converter_unavailable, and a delay-injecting server→converter_timeout. - Verified locally against a real pandoc/core:3.6 container: markdown→html, markdown→docx (valid PK/OOXML bytes), and a docx→markdown round-trip. Local: typecheck, lint, i18n:check, build all green; api 193 tests (9 new), shared 121, web 50. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 30891f99cf |
Add non-image attachments with allowlist, SVG policy, and file managers (#61)
All checks were successful
CD / Build and push images (push) Successful in 4m2s
CI / Lint, typecheck, test (push) Successful in 2m46s
CI / Auth e2e pack (push) Successful in 3m45s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 12s
Extend uploads (#27, ADR 0011) beyond images to a configurable general attachment allowlist, plus the page attachments section and the Pond Admin file manager. Backend: - Two instance settings: `upload.allowedExtensions` (lowercase, dot-stripped, images always allowed regardless) and `upload.svgPolicy` (reject | sanitize). - FilesService.resolveUpload: raster images still decided by magic bytes; SVG is sanitized with DOMPurify (scripts, event handlers, foreignObject stripped) or rejected per policy; everything else is admitted only if its extension is on the allowlist. A sanitized SVG's stored bytes are re-accounted so pond_usage matches disk. - Downloads set `Content-Disposition: attachment` for every non-raster type (office files, PDFs, SVG) with `nosniff`, so they can never execute inline; raster images stay inline for page embeds. - New endpoints: `GET /ponds/:id/files` (pond_admin: all files + usage + orphan flag), `POST /pages/:id/files` and `GET /pages/:id/files` (page-write/read: the attachments section). New error code `upload_type_not_allowed` (de+en). Frontend: - Page attachments section (AttachmentsPanel): upload, list with type glyph, size, and uploader, insert-as-link into the document (an internal media link that downloads, never renders inline), and delete. Toggled in the editor. - Pond file manager (PondFileManager) in pond settings for Pond Admins: every file with its referencing page (or an orphan flag) and storage usage. - Admin uploads settings form (allowlist + SVG policy). New `files` i18n namespace (de+en). Tests: - files.e2e.db.test.ts: allowlisted non-image accepted and served as a download; disallowed extension rejected; renamed-.html-as-.png still fails; SVG sanitized (scripts/handlers stripped) and reject-mode rejects; page attachment listing; pond file manager usage/orphan; non-admin denied. - New e2e pack apps/web/e2e/attachments.spec.ts (+ CI step): upload → list → insert link (verified attachment disposition + nosniff), disallowed-type error, pond file manager usage/orphan. Local: typecheck, lint, i18n:check, build all green; api-db 184, shared 121, web 50; attachments pack 3/3, members 3/3, content 5/5. Adds dompurify + jsdom to the api for server-side SVG sanitization. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 42e97b9df2 |
Add Site-Admin user management (#59)
Some checks failed
CD / Build and push images (push) Successful in 3m12s
CI / Lint, typecheck, test (push) Failing after 2m29s
CI / Auth e2e pack (push) Successful in 3m32s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m11s
CD / Promote to Int (push) Successful in 11s
Instance operators get basic user administration for support, abuse handling, and GDPR groundwork (security.md §Privacy). - api `admin/`: Site-Admin-gated `/admin/users` — a searchable, paginated list (username, e-mail, status, role, pond count, last login) plus lifecycle actions: disable/enable (a disabled user is logged out everywhere and login is refused with the distinct `account_disabled`), resend verification, delete, and grant/revoke Site Admin. Guards: you cannot act on your own account (`cannot_modify_self`) and the last Site Admin cannot be dropped (`last_site_admin`). Every action is audit-logged with the actor. - `PseudonymizationService`: account deletion scrubs the PII, removes all login identities + sessions, and trashes the personal pond — the kept row is what authorship references, so shared content the user authored shows as "Deleted user" (no orphaned/cascaded content). - web: the Admin area gains a 'Users' surface — search, pagination, and the actions (destructive ones behind an inline two-step confirm; self-actions hidden). New `users` i18n namespace (de+en). - tests: `user-admin.e2e.db.test.ts` (disable → logout + login blocked; delete → pseudonymized authorship + personal pond trashed + credentials gone; last Site Admin and self protected; Site-Admin gating); a non-destructive browser `admin-users` pack proving disable-in-UI blocks login and enable restores it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 7f1c49db53 |
Add pond member management UI (#54)
All checks were successful
CD / Build and push images (push) Successful in 3m3s
CI / Lint, typecheck, test (push) Successful in 2m29s
CI / Auth e2e pack (push) Successful in 3m8s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m12s
CD / Promote to Int (push) Successful in 12s
Pond Admins manage who participates in a pond, by role, with editor/reader seat quotas — the member-facing layer over the grant model (#51/#52). - shared: `MemberView`/`PondMembersView` + add/change-role schemas (`members.ts`), a `members` i18n namespace (de+en), and member error codes. - api `members/`: a member-centric API over pond-scope user grants — `GET /ponds/:id/members` (any member, for transparency: list grouped by effective role + seat usage + `canManage`), `POST` (add by exact username or e-mail — no directory browsing), `PATCH :userId` (change role), `DELETE :userId` (remove), all Pond-Admin-gated by the guard. Editor/reader seats are enforced against `editors_per_pond`/`readers_per_pond` (#22) inside a per-pond advisory-locked transaction so counts cannot race; the owner's membership is protected, personal ponds refuse a second admin (shared grant rule), and the last Pond Admin cannot be dropped. Every change invalidates the pond permission cache and fires the access NOTIFY (#39/#53). - web `members/`: `MemberManager` in Pond Settings — list grouped by role with a search filter and seat usage, add-by-identifier form (disabled with a localized explanation when the chosen role's seats are full), per-member role change and remove; read-only for non-admins; the personal-pond rule is surfaced. There is no invitation flow (v1): adding is immediate, and the copy says so. - tests: `members.e2e.db.test.ts` (add/change/remove, seat exhaustion, personal-pond and owner rules, read-only transparency, last-admin) and a `members` browser pack (immediate second-browser access, quota disables the add action, non-admin read-only) with its own CI step. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 0c6494f209 |
Enforce permissions in API guards and retire interim access (#52)
All checks were successful
CD / Build and push images (push) Successful in 2m54s
CI / Lint, typecheck, test (push) Successful in 2m25s
CI / Auth e2e pack (push) Successful in 2m58s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m12s
CD / Promote to Int (push) Successful in 12s
Every route now declares its access rule explicitly and is enforced through the shared resolution algorithm (permissions.md): - PermissionGuard + decorators (@RequiresPondRole, @RequiresPagePermission, @RequiresAttachmentPermission, @AuthenticatedOnly) applied to every route; a route-enumeration test proves full coverage alongside @Public()/Site-Admin-guarded routes. - 404/403 policy (documented in README conventions): denied reads answer 404 (existence hiding), denied writes on readable things answer 403; trash views need write capability (ADR 0013). - PermissionService resolves page/pond questions via the shared resolver, with an in-process pond-context cache (grants + label parents) that is invalidated on every grant/label-tree change and TTL-bounded as a multi-process safety net. Grant changes also fire pond_access_changed for collab revalidation (#39/#53). - shared: pond-scope resolution (hasPondRole, canSeePond) next to the page resolver; grant wire schemas + GrantView. - Owner Pond-Admin grants: migration backfill for all existing ponds, created transactionally with every new pond (shared + personal + seed). - Grant CRUD under /ponds/:id/grants (pond_admin-gated) with structural and referential validation, last-admin protection, audit logs. - InterimAccessService deleted; page lists, search, backlinks, phantom links, and trash listings are filtered per page through the resolver; collab tokens are now truly ro for readers. - Fixture-matrix e2e (reader/editor/pond admin/foreign, label-deny, authenticated-subject, revoke-then-immediate-deny cache test). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY |
|||
| 4d48d72c40 |
Add grant model and shared permission-resolution algorithm (#51)
All checks were successful
CD / Build and push images (push) Successful in 3m3s
CI / Lint, typecheck, test (push) Successful in 2m21s
CI / Auth e2e pack (push) Successful in 2m58s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
The heart of the security model: one algorithm, implemented once, for API, collab, and UI (permissions.md — authoritative). - shared `permissions/`: pure resolution (`resolvePageCapability`) exactly per permissions.md — specificity page > label (incl. ancestor labels) > pond, deny wins within a level, default-closed, Site Admin bypass — plus the trash rule (`canAccessPage` / `canAccessTrashedPage`, ADR 0013). `grantValidationError` enforces the structural constraints. Documented, I/O-free signatures for API/collab reuse. - prisma: `RoleGrant` (+ grant enums) per data-model.md, unique on (pond, subject, role, scope); migration adds a CHECK backstop that a POND_ADMIN grant is pond-scope + user-subject. - api `grants/`: `GrantsService.createGrant` validates before insert (structural + no extra admin on a personal pond), rejects duplicates; `grantsForPond` returns the shared resolver model (what #52/#53 consume); enum mappers between the DB and the shared model. Interim "who may manage grants" stays until #52. - tests: exhaustive table-driven resolver suite — every worked example from permissions.md §Resolution, edge cases (multi-label deny-wins, ancestor inheritance, anonymous/public, most-specific-allow-beats-less-specific-deny, trash) and a property test (a less-specific grant never overrides a more-specific decision); validation unit tests; grants db test proving write-time rejection of invalid grants. - i18n: grant error codes (de + en). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY |
|||
| a3a012c41d |
Add hierarchical labels: model, CRUD API, and validation (#43)
All checks were successful
CD / Build and push images (push) Successful in 2m56s
CI / Lint, typecheck, test (push) Successful in 2m5s
CI / Auth e2e pack (push) Successful in 2m26s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 12s
Introduce pond-scoped hierarchical labels as the foundation for M4 organization and, later, M5 label-scoped permissions. - shared: `labels.ts` with the label schemas/views and the pure tree helpers (buildLabelTree, collectSubtreeIds, collectAncestorIds, labelDepth, subtreeHeight). These are the single hierarchy walk the label API and the future permission resolver both build on (permissions.md: a grant on a label applies to all its descendants). - prisma: `Label` (self-referential parent_id, unique per (pond, parent, name), cascade to subtree) and `PageLabel` assignment table; migration. - api: `LabelsService` + controller. Tree endpoint returns the hierarchy in one call; create/rename/recolor/move/delete and page assign/unassign. Validation: cycle prevention on move, depth limit 6, unique name per (pond, parent) — enforced under a per-pond advisory lock so root-label uniqueness holds despite Postgres treating NULL parents as distinct. Delete cascades the subtree and requires `?force=true` when pages are assigned. Assignment rejects labels from a different pond. Access gated through InterimAccessService on the owning pond. - i18n: label error codes and the colour validation message (de + en). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY |
|||
| a645763679 |
Add page trash: soft delete, restore, and purge job (#31)
All checks were successful
CD / Build and push images (push) Successful in 2m5s
CI / Lint, typecheck, test (push) Successful in 1m45s
CI / Auth e2e pack (push) Successful in 1m50s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m11s
CD / Promote to Int (push) Successful in 10s
Backend: a generic maintenance-job scheduler (SchedulerService, `jobs` table) that any later maintenance job registers with instead of growing its own timer loop. Due-ness and the run-mutex both live in the DB row (`lastRunAt` survives a restart; claiming a due job is one atomic `UPDATE ... WHERE status != 'RUNNING'`), and an injectable ClockService lets tests simulate retention elapsing without waiting or faking the global clock. Trash endpoints: GET /ponds/:id/trash (list), POST /pages/:id/restore, DELETE /pages/:id/purge (manual, bypasses retention) — all sharing the same purge logic as the scheduled daily job (default 30-day retention, new trash.retentionDays instance setting). Purging deletes a page's content cache, update log, and attachment files/quota; page_versions is a placeholder until M3 exists. Direct navigation to a trashed page now 404s with a distinguishable `page_trashed` code for editors (a plain 404 for everyone else) instead of the generic not-found. Attachment.pageId — added in #27 but never wired up — now gets set on every page state save to whichever page's document currently embeds the file, which is what lets purge find a page's files. Frontend: a per-pond trash view (restore/purge), a "move to trash" action with confirmation in the page menu, and a trash link in the sidebar for pond owners. Also fixes react-query retrying 4xx responses for several seconds by default, which was masking the trash-hint 404 in the UI (and would have affected any other not-found/permission error the same way). Closes #31 |
|||
| 0fae699018 |
Add file storage service and image upload API (#27)
All checks were successful
CD / Build and push images (push) Successful in 2m2s
CI / Lint, typecheck, test (push) Successful in 1m43s
CI / Auth e2e pack (push) Successful in 1m48s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 10s
Implements the FileStorage abstraction (uploads/<pondId>/<fileId> on the mounted volume), the attachments model, and POST /ponds/:id/files, GET /media/:fileId, DELETE /files/:id. Uploads are validated by sniffing magic bytes rather than trusting the client's Content-Type/filename (catches a renamed .html-as-.png), checked against the max_file_bytes and storage_bytes quotas, and served with nosniff + immutable caching. Closes #27 |
|||
| 98e159ab50 |
Add page CRUD and Yjs state persistence (#23)
All checks were successful
CD / Build and push images (push) Successful in 1m52s
CI / Lint, typecheck, test (push) Successful in 1m34s
CI / Auth e2e pack (push) Successful in 1m44s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m7s
CD / Promote to Int (push) Successful in 10s
Prisma models `pages`/`page_updates`/`page_content_cache` per data-model.md. Endpoints: POST /ponds/:id/pages (title -> empty Yjs doc state, seeded via y-prosemirror), GET /pages/:id (meta + base64 state), PUT /pages/:id/state (client-encoded Yjs state, rejected above the 5 MiB operations.md limit or if it doesn't decode into a valid document for the schema), PATCH /pages/:id (title/slug — explicit slug changes validate uniqueness per pond, title-only renames keep the slug), DELETE (soft). Access follows InterimAccessService via the page's pond, same 404-not-403 interim rule as ponds. State saves decode the Yjs update with yjs + y-prosemirror and run it through the #24 shared derivation functions (docToPlainText/ docToMarkdown/docToHtml/extractOutline) to refresh page_content_cache. The Yjs XmlFragment name ("default") and the derivation call are factored so the collab server's persistence hooks (#35) can reuse both. Raised the API's JSON body limit to 8 MiB (main.ts and the e2e test app) to fit base64-encoded page state. Closes #23 |
|||
| 64928f0ac0 |
Quota foundation: overrides, resolution, race-safe consumption (#22)
All checks were successful
CD / Build and push images (push) Successful in 1m46s
CI / Lint, typecheck, test (push) Successful in 1m17s
CI / Auth e2e pack (push) Successful in 1m39s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m6s
CD / Promote to Int (push) Successful in 10s
- quota_overrides + pond_usage models (BigInt values, unique per
subject+key); migration 20260705185146_quotas
- instance-default quota keys in the settings registry (editors 5,
readers 50, additional ponds 0, storage 1 GiB, max file 25 MiB)
- QuotaService: getEffective with pond → user → instance resolution
(zero counts as a value, not a gap); assertCanCreateSharedPond and
checkAndConsume serialize via pg_advisory_xact_lock inside the guarded
write's transaction; release never drops below zero
- pond creation enforces additional_ponds (personal ponds don't count);
quota errors carry code quota_exceeded + {quotaKey, limit}, localized
- seed grants fixtures an additional_ponds override (default is 0)
- table-driven resolution tests, parallel-consumption test, e2e for the
pond-creation limit
Closes #22
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UpQz6ypHJsLfMf4S6fyQEB
|
|||
| f0850eecd3 |
Ponds: data model, CRUD API, personal pond on verification (#21)
All checks were successful
CD / Build and push images (push) Successful in 1m46s
CI / Lint, typecheck, test (push) Successful in 1m19s
CI / Auth e2e pack (push) Successful in 1m42s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m5s
CD / Promote to Int (push) Successful in 10s
- Pond model with pond-level trash columns (ADR 0013) and settings jsonb holding only deviations from the defaults (sidebar sort, font slots per ADR 0016); migration 20260705090100_ponds - shared: pond schemas/views and slugify (German transliteration, URL-safe, length-capped); deterministic -2/-3 suffixes for collisions - InterimAccessService: single place answering pond access questions until the real role model lands in M5 - POST/GET /ponds, GET /ponds/:slug, PATCH/DELETE /ponds/:id, Site-Admin trash + restore; personal pond auto-created on e-mail verification and for active seed fixtures; personal ponds cannot be trashed - e2e pack covering verify-flow pond creation, slug suffixes, rename, foreign-pond 404s, trash/restore; slugify unit tests Closes #21 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UpQz6ypHJsLfMf4S6fyQEB |
|||
| 0bc80c9f93 |
Add auth, settings, and admin UI to the SPA
The web app grows its account surface: login (with next-redirect, unverified-hint + resend), signup (react-hook-form + shared Zod schemas, field-level api errors, closed-registration state fed by the new public GET /auth/registration), e-mail verification, forgot/reset password; a settings page with profile (locale applies immediately), password change, and active-session management; a Site-Admin page for instance name, default locale, and registration mode. AuthProvider holds /auth/me, applies the profile locale, and backs route guards (RequireAuth/RequireAnonymous/RequireSiteAdmin); the top bar gains a user menu. All strings ship in the new auth/settings namespaces (de+ en); the exception filter now preserves handler-specific error codes. Verified live: signup → Mailpit → verify → login → profile through the Vite proxy. Closes #16 Closes #17 Closes #18 Closes #19 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
|||
| e855192d23 |
Add i18n with i18next, German and English, and a key-parity check
Translation resources live in packages/shared/i18n/<lang>/<ns>.json (common, errors) and ship with de and en. The web app initializes react-i18next with bundled resources (?lng= wins, then the browser language); all shell components use useTranslation and the temporary t() stub is gone. The api localizes its uniform error bodies via a minimal i18next instance negotiated from Accept-Language. `pnpm i18n:check` fails CI when any key is missing in any language, backed by tested helpers in @dorfteich/shared. Closes #5 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |