Public REST API v1: personal access tokens, instance switch + per-pond opt-in #104

New Issue

fable-5 · 2026-07-12T09:37:30+02:00

fable-5 commented

2026-07-12 09:37:30 +02:00

Context

Dorfteich is only reachable through the browser session today (dt_session cookie, auth.guard.ts). Scripts, integrations and the upcoming MCP endpoint (follow-up issue) need a stable, token-authenticated public API.

Scope

Personal access tokens: every user creates tokens under Settings (name, optional expiry, scope read | write, optional restriction to selected ponds). The token value is shown once and stored hashed (auth-tokens pattern); tokens are revocable and carry a last-used timestamp. A token acts AS its user — the entire permission system (grants, label scopes, 404-vs-403 per #60) applies unchanged. Auth via Authorization: Bearer dt_pat_...: a new guard branch beside the session cookie, CSRF-exempt (no cookie involved), rate-limited per token (existing RateLimitService), audit-logged (api.token_created/api.token_revoked, plus request auditing for writes).
Activation: instance setting api.enabled (default off, admin settings UI) + per-pond opt-in apiEnabled in the pond settings (default off; pondSettingsSchema pattern, toggle in the pond settings page). Disabled instance or non-opted-in pond → the API answers 404 for that content (consistent with #60).
Endpoints (/api/public/v1/…, JSON, versioned; OpenAPI document served at /api/public/v1/openapi.json):
- GET /me — the token's user + scopes (client smoke test)
- GET /ponds — API-enabled ponds visible to the token user; GET /ponds/{slug} — metadata
- GET /ponds/{slug}/pages — page list (slug, title, labels, updatedAt)
- GET /ponds/{slug}/pages/{pageSlug} — page as Markdown + rendered HTML + metadata
- POST /ponds/{slug}/pages — create a page from title + Markdown (shared markdownToDoc pipeline)
- PATCH /ponds/{slug}/pages/{pageSlug} — update title and/or replace content from Markdown. Content replacement must go through the collab-owned document path (same pg-NOTIFY mechanism as the version restore) so open editors converge and no second document lineage appears.
- DELETE /ponds/{slug}/pages/{pageSlug} — move to trash
- GET /search?q=&pond=&label= — permission-filtered search (reuses the #49 provider incl. the substring fallback)
- GET /ponds/{slug}/export/markdown — ZIP export (reuses ExportService.streamPondMarkdownZip)
- Labels: GET/POST /ponds/{slug}/labels, PATCH/DELETE /labels/{id} (rename, recolor, move, delete), PUT/DELETE /ponds/{slug}/pages/{pageSlug}/labels/{labelId} (assign/unassign)
- Comments: GET/POST /ponds/{slug}/pages/{pageSlug}/comments + resolve/unresolve
Stage 2 (explicitly out of scope): attachment upload, version endpoints, webhooks, pond-bound service keys.

Acceptance criteria

token lifecycle in the settings UI (create with scope/expiry/pond restriction, reveal once, revoke, last-used) — de+en
every endpoint enforces the existing permission model plus the instance switch and pond opt-in; disabled → 404, insufficient scope → 403 scope_required
page create/update round-trips Markdown through the shared pipeline; concurrent editors converge (verified in the collab pack)
the OpenAPI document matches the implementation and is linked from docs/self-hosting
per-token rate limits; audit entries for token lifecycle and all writes
QA pack: token-authenticated permission matrix (reader/editor/outsider × read/write scopes × opt-in states)

Technical notes

New api_tokens table (hash, scopes, pond restriction, expiry, lastUsedAt) + guard branch in apps/api/src/auth/auth.guard.ts; token hashing like AuthTokensService (SHA-256).
Public controllers live in their own module (apps/api/src/public-api/) and call the existing services (pages/labels/comments/search/export) — no logic duplication.
Reuse RateLimitService (per-token keys) and AuditService.

Dependencies

None (prerequisite for the MCP endpoint issue).

Size: ~3–4 days

Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add de and en), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.

## Context Dorfteich is only reachable through the browser session today (`dt_session` cookie, `auth.guard.ts`). Scripts, integrations and the upcoming MCP endpoint (follow-up issue) need a stable, token-authenticated public API. ## Scope - **Personal access tokens**: every user creates tokens under Settings (name, optional expiry, scope `read` | `write`, optional restriction to selected ponds). The token value is shown once and stored hashed (auth-tokens pattern); tokens are revocable and carry a last-used timestamp. A token acts AS its user — the entire permission system (grants, label scopes, 404-vs-403 per #60) applies unchanged. Auth via `Authorization: Bearer dt_pat_...`: a new guard branch beside the session cookie, CSRF-exempt (no cookie involved), rate-limited per token (existing `RateLimitService`), audit-logged (`api.token_created`/`api.token_revoked`, plus request auditing for writes). - **Activation**: instance setting `api.enabled` (default off, admin settings UI) + per-pond opt-in `apiEnabled` in the pond settings (default off; `pondSettingsSchema` pattern, toggle in the pond settings page). Disabled instance or non-opted-in pond → the API answers 404 for that content (consistent with #60). - **Endpoints** (`/api/public/v1/…`, JSON, versioned; OpenAPI document served at `/api/public/v1/openapi.json`): - `GET /me` — the token's user + scopes (client smoke test) - `GET /ponds` — API-enabled ponds visible to the token user; `GET /ponds/{slug}` — metadata - `GET /ponds/{slug}/pages` — page list (slug, title, labels, updatedAt) - `GET /ponds/{slug}/pages/{pageSlug}` — page as Markdown + rendered HTML + metadata - `POST /ponds/{slug}/pages` — create a page from title + Markdown (shared `markdownToDoc` pipeline) - `PATCH /ponds/{slug}/pages/{pageSlug}` — update title and/or replace content from Markdown. Content replacement must go through the collab-owned document path (same pg-NOTIFY mechanism as the version restore) so open editors converge and no second document lineage appears. - `DELETE /ponds/{slug}/pages/{pageSlug}` — move to trash - `GET /search?q=&pond=&label=` — permission-filtered search (reuses the #49 provider incl. the substring fallback) - `GET /ponds/{slug}/export/markdown` — ZIP export (reuses `ExportService.streamPondMarkdownZip`) - Labels: `GET/POST /ponds/{slug}/labels`, `PATCH/DELETE /labels/{id}` (rename, recolor, move, delete), `PUT/DELETE /ponds/{slug}/pages/{pageSlug}/labels/{labelId}` (assign/unassign) - Comments: `GET/POST /ponds/{slug}/pages/{pageSlug}/comments` + resolve/unresolve - **Stage 2 (explicitly out of scope)**: attachment upload, version endpoints, webhooks, pond-bound service keys. ## Acceptance criteria - [ ] token lifecycle in the settings UI (create with scope/expiry/pond restriction, reveal once, revoke, last-used) — de+en - [ ] every endpoint enforces the existing permission model plus the instance switch and pond opt-in; disabled → 404, insufficient scope → 403 `scope_required` - [ ] page create/update round-trips Markdown through the shared pipeline; concurrent editors converge (verified in the collab pack) - [ ] the OpenAPI document matches the implementation and is linked from docs/self-hosting - [ ] per-token rate limits; audit entries for token lifecycle and all writes - [ ] QA pack: token-authenticated permission matrix (reader/editor/outsider × read/write scopes × opt-in states) ## Technical notes - New `api_tokens` table (hash, scopes, pond restriction, expiry, lastUsedAt) + guard branch in `apps/api/src/auth/auth.guard.ts`; token hashing like `AuthTokensService` (SHA-256). - Public controllers live in their own module (`apps/api/src/public-api/`) and call the existing services (pages/labels/comments/search/export) — no logic duplication. - Reuse `RateLimitService` (per-token keys) and `AuditService`. ## Dependencies None (prerequisite for the MCP endpoint issue). **Size**: ~3–4 days --- *Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add `de` **and** `en`), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.*

fable-5 added this to the M11 — Public API & MCP milestone 2026-07-12 09:37:30 +02:00

fable-5 added the

auth

frontend

backend

labels 2026-07-12 09:37:30 +02:00

fable-5 referenced this issue

2026-07-12 09:37:54 +02:00

Built-in MCP endpoint (Streamable HTTP) on top of the public API #105

fable-5 referenced this issue from a commit

2026-07-12 11:17:04 +02:00

Public REST API v1: personal access tokens, instance switch + per-pond opt-in (#104)

fable-5 referenced this issue from a commit

2026-07-12 11:23:20 +02:00

Deflake the collab restore-listener DB test (poll for the PRE_RESTORE row)

fable-5 commented

2026-07-12 11:35:17 +02:00

Implemented in 0c85293 (+ 52975ba deflaking a pre-existing race in the collab restore-listener test that the CI run surfaced). CI fully green, deployed to Test + Int.

What shipped

Personal access tokens: api_tokens table (SHA-256 hash, scope read|write, optional pond restriction, expiry, revocation, throttled last-used), migration 20260712020000_api_tokens. Secrets dt_pat_…, shown exactly once. Lifecycle endpoints under /users/me/api-tokens are session-only — a leaked token can never mint more tokens. Settings-UI section (create/reveal-once/revoke/status) de+en. Audit: api.token_created/api.token_revoked.
Activation: instance setting api.enabled (admin switch) + pond setting apiEnabled (pond-settings toggle), both default off; 404 semantics per #60 on both levels. (PondsService.update settings-merge covers the new key — the #92 lesson.)
Surface /api/public/v1 (excluded from the SPA prefix): me, ponds, pages (list, read as Markdown+HTML, create from Markdown via the shared pipeline, PATCH title/content, DELETE→trash), search (permission-filtered, narrowed to exposed ponds, **…** highlights), markdown-ZIP export, labels (tree/CRUD/assign), comments (threads/create/resolve/reopen). Hand-maintained OpenAPI 3.1 at /openapi.json, pinned to the controller by a two-way route-coverage test.
Content replacement through the collab path: the new state lands as a MANUAL version "API update", then the established restore NOTIFY applies it — open editors converge (proven by a new collab-pack e2e test), history stays append-only, no second lineage.
Enforcement: PublicApiGuard (instance switch → bearer PAT → per-token rate limit 429+Retry-After → scope 403 scope_required → pond opt-in + token restriction), then the unchanged shared PermissionGuard (PageParamSource gained pondSlugParam). No cookies → no CSRF surface (pinned by a hostile-Origin test). Every write audit-logged as api.write with the token attributed.

Deviations from the issue text (deliberate): label routes are nested under the pond (PATCH/DELETE /ponds/{slug}/labels/{id} instead of /labels/{id}) so the pond opt-in gate stays uniform; one PATCH covers rename+recolour+move.

Verification: 12-test e2e pack (lifecycle, switches, permission matrix reader/editor/outsider × scopes × opt-in, restriction, roundtrip incl. restore-NOTIFY assertion, comments policy, search narrowing, rate limit); full api suite green; UI smoke against the built SPA 10/10; docs docs/self-hosting/public-api.md.

Lesson recorded: never raise the instance-default quota in shared-DB test suites — the pack now uses a per-user quota override (the parallel quota suites caught the earlier version).

Implemented in `0c85293` (+ `52975ba` deflaking a pre-existing race in the collab restore-listener test that the CI run surfaced). CI fully green, deployed to Test + Int. **What shipped** - **Personal access tokens**: `api_tokens` table (SHA-256 hash, scope read|write, optional pond restriction, expiry, revocation, throttled last-used), migration `20260712020000_api_tokens`. Secrets `dt_pat_…`, shown exactly once. Lifecycle endpoints under `/users/me/api-tokens` are session-only — a leaked token can never mint more tokens. Settings-UI section (create/reveal-once/revoke/status) de+en. Audit: `api.token_created`/`api.token_revoked`. - **Activation**: instance setting `api.enabled` (admin switch) + pond setting `apiEnabled` (pond-settings toggle), both default off; 404 semantics per #60 on both levels. (`PondsService.update` settings-merge covers the new key — the #92 lesson.) - **Surface `/api/public/v1`** (excluded from the SPA prefix): me, ponds, pages (list, read as Markdown+HTML, create from Markdown via the shared pipeline, PATCH title/content, DELETE→trash), search (permission-filtered, narrowed to exposed ponds, `**…**` highlights), markdown-ZIP export, labels (tree/CRUD/assign), comments (threads/create/resolve/reopen). Hand-maintained OpenAPI 3.1 at `/openapi.json`, pinned to the controller by a two-way route-coverage test. - **Content replacement through the collab path**: the new state lands as a MANUAL version "API update", then the established restore NOTIFY applies it — open editors converge (proven by a new collab-pack e2e test), history stays append-only, no second lineage. - **Enforcement**: `PublicApiGuard` (instance switch → bearer PAT → per-token rate limit 429+Retry-After → scope 403 `scope_required` → pond opt-in + token restriction), then the unchanged shared `PermissionGuard` (`PageParamSource` gained `pondSlugParam`). No cookies → no CSRF surface (pinned by a hostile-Origin test). Every write audit-logged as `api.write` with the token attributed. **Deviations from the issue text** (deliberate): label routes are nested under the pond (`PATCH/DELETE /ponds/{slug}/labels/{id}` instead of `/labels/{id}`) so the pond opt-in gate stays uniform; one PATCH covers rename+recolour+move. **Verification**: 12-test e2e pack (lifecycle, switches, permission matrix reader/editor/outsider × scopes × opt-in, restriction, roundtrip incl. restore-NOTIFY assertion, comments policy, search narrowing, rate limit); full api suite green; UI smoke against the built SPA 10/10; docs `docs/self-hosting/public-api.md`. **Lesson recorded**: never raise the instance-default quota in shared-DB test suites — the pack now uses a per-user quota override (the parallel quota suites caught the earlier version).

fable-5 closed this issue

2026-07-12 11:35:17 +02:00

fable-5 referenced this issue from a commit

2026-07-12 11:36:03 +02:00

Built-in MCP endpoint (Streamable HTTP) on top of the public API (#105)

fable-5 referenced this issue

2026-07-12 11:47:59 +02:00

Built-in MCP endpoint (Streamable HTTP) on top of the public API #105

fable-5 referenced this issue

2026-07-30 11:59:31 +02:00

[VS-NfD] Define the deployment hardening-profile mode and the machine-readable VS-NfD configuration catalog #243

Sign in to join this conversation.