324-325-self-hosting-guide-findings
14 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| dbbe229cb9 |
Pond knowledge graph view (#112)
Some checks failed
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CI / Auth e2e pack (push) Blocked by required conditions
CI / Import/export fidelity gate (push) Blocked by required conditions
CI / Build container images (push) Blocked by required conditions
CD / Build and push images (push) Has been cancelled
CI / Lint, typecheck, test (push) Has been cancelled
/p/:pondSlug/graph (static segment ranked above :pageSlug, same documented reserved-slug gap as trash/settings) renders the pond's readable wikilink graph from GET /ponds/:id/links: pages as nodes colored by their first label (legend included, DEFAULT_LABEL_COLOR for unlabeled), resolved links as edges, phantom targets as dashed nodes — clicking one offers to create the page, which resolves its links. Rendering is a self-contained SVG force graph: only d3-force is bundled (no d3 DOM/zoom modules, zero external requests); the layout runs synchronously to rest, zoom/pan/node-drag are plain pointer math. SVG over canvas deliberately — every node carries a data-testid the e2e packs can click. Ponds beyond 500 pages get a capped-view notice. Sidebar footer links every member to the graph (trash stays owner-only). New i18n namespace graph (de+en). Verified live: nodes/edges/legend render, node click opens the page, phantom click creates it and the node turns solid. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
|||
| 5cef359b8f |
Nextcloud backup target: admin-configured, manual + scheduled uploads, in-app restore (#103)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m45s
CD / Build and push images (push) Successful in 3m49s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m35s
CI / Import/export fidelity gate (push) Successful in 47s
Off-host backups for every self-hoster, configured entirely in the admin UI — supersedes the host-specific mirror plan behind #84. shared: - webdav.ts (new package entry like token-crypto): minimal WebDAV client with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT (streamed), GET, DELETE; Nextcloud DAV path derived from the plain server URL, explicit DAV bases pass through - backup-status.ts: additive remote-upload status in status.json, the restore-status.json contract (running/succeeded/failed + staleness bound), the backup_command/backup_maintenance NOTIFY channels, and the one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz) - backup-set.ts moved here from apps/backup (api lists local sets) backup sidecar: - reads the backup.* instance settings directly from the database (admin changes apply next run; local retention row overrides the env) and the app password from the secret store - after each successful set: bundle dump + files archive + manifest into ONE self-contained tar.gz, upload via WebDAV per schedule (off/daily/weekly; manual runs always upload), prune remote bundles — never the newest — and record the outcome in status.json; upload failures alert via a new backupUploadFailed mail (de+en) - command listener on backup_command (run / restore) with a serial queue against the nightly timer - restore orchestrator: restore-status.json → maintenance NOTIFY → grace → (remote: download + manifest-verify bundle) → terminate other DB connections → shared perform-restore path (same code as restore.sh) → final status + maintenance exit api: - MaintenanceGuard (global, registered before the setup gate): 503 maintenance_mode while restore-status says running; health endpoints and the new public GET /backup/restore-status stay exempt; a stale running state (crashed sidecar) unblocks after 30 min - MaintenanceStateService watches the file and restarts the api after a successful restore (fresh caches, migrate-on-start for older dumps); main.ts refuses to touch the database while a restore runs — a container restarting mid-restore must not race pg_restore with migrate deploy - worker sweeps (conversion, mail outbox, scheduler) catch transient database failures instead of dying on an unhandled rejection — the restore's connection termination crashed the api in verification - backup admin endpoints under /admin/system/backup: settings (live connection test before save, password write-only into the secret store), nextcloud/test, sets (local via the ro backups mount + remote via WebDAV), run + restore (type-to-confirm backstop, source validation) — commands travel as NOTIFY payloads; audit actions backup.settings_changed/run_triggered/restore_requested - readyz: new warning-level backup_remote check while a target is configured (26 h daily / 170 h weekly bound) collab: - maintenance listener: on enter, persist + close every live session and refuse new connections until exit (failsafe timeout 30 min) — no in-memory document may write pre-restore content back afterwards web: - Admin → System backup section: status card with remote facts and a "Back up now" button, the Nextcloud settings form with test button, and the restore picker (local + remote sets, type-to-confirm) - global maintenance screen: any 503 maintenance_mode flips the SPA to a status page polling the exempt endpoint, reloading when the instance returns Verified end-to-end against a live stack (fresh DB, native api + sidecar, fake WebDAV server): configure → test → manual backup → bundle upload → readyz/sets/status surfaces → remote restore with maintenance gate, marker rollback and api restart; suites: shared 21, backup 9, collab 11, api 58 files green, lint + i18n:check + typecheck clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| c8aac13dfb |
Add Site-Admin system panel with persistent audit trail (#86)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m14s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m45s
CD / Deploy to Test (push) Successful in 10s
CD / Smoke tests against Test (push) Successful in 1m11s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m20s
CI / Import/export fidelity gate (push) Successful in 45s
New /admin/system panel (operations.md §Maintenance jobs): the maintenance job list shows every registered job with truthful last-run data (new Job.lastDurationMs recorded by the scheduler) and a manual trigger that respects the run-mutex and is itself audit-logged; a backup card mirrors the sidecar's status.json including the freshness verdict; an audit-log viewer filters by actor, action, and time range with pagination; and a storage overview lists the largest ponds. Auth events and admin actions (grants, members, user/quota admin, plugins, settings, setup) now land in a new audit_log table through a central AuditService — which keeps emitting the established stdout log line — while content activity stays log-only by design. All endpoints are Site-Admin-only; covered by API DB tests and a Playwright pack in CI. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| fd2bdb3fb8 |
Add instance legal pages with public rendering and footer links (#82)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m7s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m14s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m16s
CD / Promote to Int (push) Successful in 9s
CI / Auth e2e pack (push) Successful in 5m9s
CI / Import/export fidelity gate (push) Successful in 45s
Imprint and privacy policy are two new Markdown instance settings (legal.imprint, legal.privacyPolicy), edited by Site Admins in a new "Legal pages" admin section with a toggleable rendered preview. The preview uses the same shared pipeline the server renders with (markdown → schema doc → escaped HTML), so stored markup can never smuggle script to visitors. The pages render publicly at /legal/imprint and /legal/privacy — as an SPA route plus, like #56, a self-contained server-rendered HTML document under /api/v1/legal/:kind. The endpoints are setup-exempt: legal information stays reachable even while the first-run wizard is pending. Unconfigured pages show a localized notice instead of 404ing, and Site Admins additionally get a warning banner linking to the settings. A new footer with both links appears on every SPA view (editor, auth screens, public pages) and in the server-rendered documents, whose shared shell moved to public/html-shell.ts and now renders its chrome in the instance default locale (ADR 0012). docs/self-hosting/legal-template.md ships imprint and privacy-policy templates in English and German whose sections mirror Dorfteich's actual processing activities (accounts, sessions, rate-limit IPs, proxy logs, transactional mail, content, export, deletion, no third-party requests), with a review checklist tied to security.md §Privacy. New `legal` i18n namespace (de+en); api and web e2e coverage including a new CI legal pack (footer navigation, notice vs. admin banner, and the admin form publishing a text end to end). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 28aa04d5e4 |
Add the first-run setup wizard UI (#81)
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m6s
CI / Build container images (push) Has been skipped
CD / Build and push images (push) Successful in 3m13s
CD / Deploy to Test (push) Successful in 12s
CD / Smoke tests against Test (push) Successful in 1m14s
CD / Promote to Int (push) Successful in 10s
CI / Auth e2e pack (push) Successful in 5m6s
CI / Import/export fidelity gate (push) Successful in 45s
The SPA now probes GET /setup on boot: while setup is pending it renders only the wizard at /setup, the login page (to resume a started wizard as the Site Admin), and a "setup pending" notice on every other route — without the regular chrome, whose pond/search queries would all 503. If the probe itself fails (offline reload), the app falls through to the normal routes. Once completed, /setup just goes home. The wizard walks six steps against the #80 api: welcome with language choice (drives i18n immediately and pre-fills the admin/instance locales), Site Admin account (signs in via the step-1 session cookie), instance basics, SMTP with live-test-before-save plus an explicit skip, registration mode, and a summary whose finish unlocks the app logged-in-ready. Every step validates through the shared Zod schemas before advancing; entered values live in the parent component, so Back preserves them, and re-submitting a step on a second forward pass just overwrites the same settings. A wizard someone else started shows a sign-in hand-off instead of dead admin-gated steps. New `setup` i18n namespace in de and en. Two #80 touch-ups fell out of verifying this end to end: the SMTP port's NaN case now maps to the translated required-message, and GET /setup's smtpConfigured uses `||` instead of `??` so the empty strings compose passes for unset vars fall through to the secret store. The e2e pack (setup.spec.ts) needs an instance where setup is still pending, so the CI job provisions a second api + static web against a virgin database on their own ports and runs the full wizard journey there, including the failing-relay path and both languages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 0875e2a087 |
Add the sandbox host runtime for plugin iframes (#73)
All checks were successful
CI / Build container images (push) Has been skipped
CI / Lint, typecheck, test (push) Successful in 2m57s
CI / Import/export fidelity gate (push) Successful in 46s
CD / Build and push images (push) Successful in 3m16s
CD / Deploy to Test (push) Successful in 8s
CI / Auth e2e pack (push) Successful in 4m8s
CD / Smoke tests against Test (push) Successful in 1m9s
CD / Promote to Int (push) Successful in 10s
Implements the security core of the plugin system: code-plugin surfaces run in opaque-origin iframes (sandbox="allow-scripts", never allow-same-origin) with a capability-filtered RPC bridge. - api: serve a per-plugin sandbox frame document at /plugins/:id/:version/frame with a CSP that pins every load to the plugin's own asset path (built from APP_BASE_URL, not the request Host, so a Host-rewriting proxy cannot break it) and forbids network access (connect-src 'none'). Plugin assets get Access-Control-Allow-Origin: * so the null-origin frame can load its own module bundle. - web: sandbox-host creates the frame, wires the SDK host bridge over a source-filtered postMessage transport, drives render under a 5 s deadline (hung/failed plugin -> placeholder, never a frozen page), and tears down on unmount. PluginFrame/PluginPreviewPage surface it; the built-in ui.resize handler clamps plugin-requested heights. - plugin-sdk: host bridge reports gate violations via onViolation and registers a gated handler for every v1 method, so an undeclared capability is rejected with capability_not_permitted (not unknown_method). - tests: SDK gate unit test; web sandbox unit tests (opaque origin, source filtering, timeout); and the e2e security pack with a permanent malicious fixture plugin proving no escape (DOM/cookies/storage/fetch/ undeclared capability all blocked) plus well-behaved and hung cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| f500198c5d |
Add per-pond fonts: catalog, build, application, and admin UI (#66)
All checks were successful
CD / Build and push images (push) Successful in 3m24s
CI / Lint, typecheck, test (push) Successful in 3m6s
CI / Auth e2e pack (push) Successful in 4m8s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
Self-hosted Google Fonts with per-pond selection (ADR 0016), the GDPR "zero external requests" posture (security.md, CSP `font-src 'self'`). - Catalog: a curated 15-family OFL/Apache list in shared (family, weights, category, license, google-webfonts-helper id). `deploy/fonts/build-fonts.mjs` validates every entry has license info (fails the build otherwise), downloads the WOFF2 weights into apps/web/public/fonts/ (gitignored), and generates the @font-face stylesheet — run at image build time from the web Dockerfile (with retries), never from a visitor's browser. - Application: PondFontScope sets --font-heading/body/mono (+ weights) from pond.settings.fonts on the editor + read view; the existing global CSS already reads those custom properties, so headings/body/code re-resolve to the pond's fonts. A pond with no settings arrives with the defaulted values (Roboto 400 / Roboto 200 / Fira Code), so the vision defaults always render. - Admin UI: pond-settings 'Appearance' section — three slots (family + weight) with a live preview, Pond-Admin-gated (fonts added to updatePondInputSchema and merged in PondsService.update); a font catalog attribution page (/fonts) listing families and licenses. New `font` i18n namespace (de+en). - CSP: strict Content-Security-Policy in nginx.conf (default-src 'self'; font-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self'; …) — the app's scripts are all external files, inline styles cover CSS variables. - Tests: shared catalog-integrity unit test (the invariant the build enforces); e2e fonts pack — no request leaves the origin when rendering a pond (the GDPR network assertion), a font choice applies to a page and persists, and a pond without settings renders the defaults. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| fc41c91003 |
Add public read access and server-rendered page HTML (#56)
All checks were successful
CD / Build and push images (push) Successful in 3m13s
CI / Lint, typecheck, test (push) Successful in 2m30s
CI / Auth e2e pack (push) Successful in 3m21s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
Anonymous visitors read what `public` grants allow, via the SPA and a server-rendered HTML endpoint for crawlers / PDF export (ADR 0005/0009). - api `public/`: `GET /public/:pondSlug/:pageSlug` returns a self-contained HTML document (content cache + minimal chrome + canonical link, no session-dependent content), and `…/content` returns JSON for the SPA. Both are `@Public()` and resolve the `public` subject through the shared resolver (PermissionService) — denied or missing → 404, so non-public pages never reveal their existence (security.md). Cached image nodes (`data-file-id`) are resolved to `/api/v1/media/:fileId` for the static render. - media: `GET /media/:fileId` is `@Public()` too, so embedded images on a public page stream to anonymous visitors; the attachment guard still gates on the `public` grant (non-public → 404). - web: a lightweight read-only `PublicPageView` at `/public/:pondSlug/:pageSlug` (outside the auth guard) renders the server HTML — deliberately without importing the collaborative editor, so anonymous readers load no editor bundle. New `public` i18n namespace (de+en). - tests: `public.e2e.db.test.ts` (HTML + JSON served for a public page; a non-public page never resolves; removing the grant 404s both) and a browser `public` pack (anonymous reads a public page and its image via the SPA; a non-public page shows "not found") with its own CI step. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1 |
|||
| 03e72242d3 |
Add label UI: tree management, page assignment, and sidebar filter (#44)
All checks were successful
CD / Build and push images (push) Successful in 2m53s
CI / Lint, typecheck, test (push) Successful in 2m8s
CI / Auth e2e pack (push) Successful in 2m31s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
Build the M4 label experience on top of the #43 label API. - shared: `flattenLabelTree` (tree → depth-first list) for chip lookup, filtering, and the picker; `PageListItemView` adds each page's `labelIds` to the sidebar list response. - api: `GET /ponds/:id/pages` now includes `labelIds` per page (one grouped query), so the sidebar can render chips and filter without extra calls. - web: - Pond settings page (`/p/:pondSlug/settings`) with a `LabelManager` tree: inline create, rename, recolour (`<input type=color>`), move via a parent picker that excludes the label's own subtree, and delete that confirms then force-detaches assigned pages. Every control is a native button/input/select — the tree is fully keyboard-operable. - `LabelPicker` panel on the page editor: searchable, hierarchy-indented multi-select that assigns/unassigns immediately and refreshes the page's labels and the sidebar. - Sidebar: colored label chips on page entries (readable text via a luminance-based contrast helper) and a descendant-inclusive label filter (selecting a parent matches pages tagged with its children, via the shared `collectSubtreeIds`). Owner link to pond settings. - i18n `labels` namespace (de + en). - e2e `labels.spec.ts` (new CI pack): full lifecycle from the settings UI and picker-assign + parent-filter-includes-child. Selectors are language-independent because the UI language follows the user's locale. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY |
|||
| a645763679 |
Add page trash: soft delete, restore, and purge job (#31)
All checks were successful
CD / Build and push images (push) Successful in 2m5s
CI / Lint, typecheck, test (push) Successful in 1m45s
CI / Auth e2e pack (push) Successful in 1m50s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m11s
CD / Promote to Int (push) Successful in 10s
Backend: a generic maintenance-job scheduler (SchedulerService, `jobs` table) that any later maintenance job registers with instead of growing its own timer loop. Due-ness and the run-mutex both live in the DB row (`lastRunAt` survives a restart; claiming a due job is one atomic `UPDATE ... WHERE status != 'RUNNING'`), and an injectable ClockService lets tests simulate retention elapsing without waiting or faking the global clock. Trash endpoints: GET /ponds/:id/trash (list), POST /pages/:id/restore, DELETE /pages/:id/purge (manual, bypasses retention) — all sharing the same purge logic as the scheduled daily job (default 30-day retention, new trash.retentionDays instance setting). Purging deletes a page's content cache, update log, and attachment files/quota; page_versions is a placeholder until M3 exists. Direct navigation to a trashed page now 404s with a distinguishable `page_trashed` code for editors (a plain 404 for everyone else) instead of the generic not-found. Attachment.pageId — added in #27 but never wired up — now gets set on every page state save to whichever page's document currently embeds the file, which is what lets purge find a page's files. Frontend: a per-pond trash view (restore/purge), a "move to trash" action with confirmation in the page menu, and a trash link in the sidebar for pond owners. Also fixes react-query retrying 4xx responses for several seconds by default, which was masking the trash-hint 404 in the UI (and would have affected any other not-found/permission error the same way). Closes #31 |
|||
| 49beb45b3e |
Add pond sidebar with page list, sort modes, and pond switcher (#26)
All checks were successful
CD / Build and push images (push) Successful in 1m59s
CI / Lint, typecheck, test (push) Successful in 1m38s
CI / Auth e2e pack (push) Successful in 1m48s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 9s
GET /ponds/:id/pages lists a pond's pages ordered by the pond's persisted sidebarSort setting (alpha/created; manual arrives with #45). The sidebar consumes it to show the page list with an active-page highlight, an owner-only sort switch (persists via the existing PATCH /ponds/:id), and an inline "new page" flow. The top bar gains a pond switcher; a new /p/:pondSlug route gives it somewhere to land, redirecting to the pond's first page once loaded. Sidebar collapse gains a Ctrl/Cmd+\ shortcut and a slightly refined transition. Closes #26 |
|||
| 076883a9a6 |
Add TipTap page editor with REST persistence (#25)
All checks were successful
CD / Build and push images (push) Successful in 2m0s
CI / Lint, typecheck, test (push) Successful in 1m42s
CI / Auth e2e pack (push) Successful in 1m50s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m10s
CD / Promote to Int (push) Successful in 10s
TipTap is bound to the canonical ProseMirror schema (packages/shared, #24) via a generic bridge (spec-utils.ts) that re-derives every node/mark's attrs/parseDOM/toDOM from editorSchema instead of duplicating them, so the editor's schema stays byte-for-byte identical to what the api decodes Yjs states against — guarded by a schema- fidelity + real Yjs round-trip test (@tiptap/y-tiptap client encoding against y-prosemirror server decoding). Route /p/:pondSlug/:pageSlug (RequireAuth) resolves the page via a new GET /ponds/:pondId/pages/:slug endpoint, binds a local Y.Doc via @tiptap/extension-collaboration (fragment "default"), and offers a view/edit mode toggle (sidebar auto-hides in edit mode via a small AppLayout context). Page state saves debounced to PUT /pages/:id/state with a truthful saving/saved/error(retrying) indicator; title saves separately via PATCH /pages/:id. Toolbar covers headings, marks, lists, blockquote, code block, hr, table (insert/row/column/header ops via prosemirror-tables), a minimal link mark, and an image placeholder (real upload is #27/#28). Closes #25 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
|||
| 0bc80c9f93 |
Add auth, settings, and admin UI to the SPA
The web app grows its account surface: login (with next-redirect, unverified-hint + resend), signup (react-hook-form + shared Zod schemas, field-level api errors, closed-registration state fed by the new public GET /auth/registration), e-mail verification, forgot/reset password; a settings page with profile (locale applies immediately), password change, and active-session management; a Site-Admin page for instance name, default locale, and registration mode. AuthProvider holds /auth/me, applies the profile locale, and backs route guards (RequireAuth/RequireAnonymous/RequireSiteAdmin); the top bar gains a user menu. All strings ship in the new auth/settings namespaces (de+ en); the exception filter now preserves handler-specific error codes. Verified live: signup → Mailpit → verify → login → profile through the Vite proxy. Closes #16 Closes #17 Closes #18 Closes #19 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
|||
| 300a418e85 |
Add React SPA shell with routing, layout, and API status
apps/web becomes a Vite + React application: React Router with home and 404 routes, base layout (top bar, collapsible sidebar remembered per user via localStorage, main area), CSS design tokens including the three font slots from ADR 0016, TanStack Query, and a typed fetch helper showing live API health on the home page. All UI strings go through a t() stub that issue #5 replaces with i18next. The Vite dev server proxies /api to the api dev port (3001). Closes #4 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |