stwaidele/dorfteich
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 22 Wiki Activity

Document persistence hooks: load, store, and content-cache refresh #35

New Issue
Closed
opened 2026-07-04 14:52:15 +02:00 by fable-5 · 1 comment
fable-5 commented 2026-07-04 14:52:15 +02:00
Collaborator
Copy Link

Context

The collab server becomes the writer of page state: load on first open, debounced store, derived content refresh (ADR 0003).

Scope

Implement onLoadDocument (merge ydoc_state + page_updates log) and onStoreDocument (debounce 2 s / max 30 s: append update to log, periodically merge into state) against PostgreSQL, then refresh page_content_cache via the shared derivation (#23/#24) and update pages.updated_at. Remove/disable the REST PUT /pages/:id/state write path (keep read). Enforce the document size ceiling on updates.

Acceptance criteria

  • two collab clients see each other's edits; state survives collab restart
  • content cache (markdown/plain) reflects edits within seconds (poll test)
  • size-ceiling breach rejects the update and notifies the offending client
  • load of a page with a long update log is correct (fixture with 1000 updates)
  • REST state writes now return 410 with a hint

Technical notes

  • ADR 0003, data-model.md (page_updates), realtime-collaboration.md §lifecycle.
  • Wrap store operations in one transaction per flush; measure and log flush duration.

Dependencies

Depends on #24, #34.

Size: ~2 days

Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add de and en), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.

## Context The collab server becomes the writer of page state: load on first open, debounced store, derived content refresh (ADR 0003). ## Scope Implement `onLoadDocument` (merge `ydoc_state` + `page_updates` log) and `onStoreDocument` (debounce 2 s / max 30 s: append update to log, periodically merge into state) against PostgreSQL, then refresh `page_content_cache` via the shared derivation (#23/#24) and update `pages.updated_at`. Remove/disable the REST `PUT /pages/:id/state` write path (keep read). Enforce the document size ceiling on updates. ## Acceptance criteria - [ ] two collab clients see each other's edits; state survives collab restart - [ ] content cache (markdown/plain) reflects edits within seconds (poll test) - [ ] size-ceiling breach rejects the update and notifies the offending client - [ ] load of a page with a long update log is correct (fixture with 1000 updates) - [ ] REST state writes now return 410 with a hint ## Technical notes - ADR 0003, data-model.md (`page_updates`), realtime-collaboration.md §lifecycle. - Wrap store operations in one transaction per flush; measure and log flush duration. ## Dependencies Depends on #24, #34. **Size**: ~2 days --- *Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add `de` **and** `en`), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.*
fable-5 added this to the M3 — Real-time collaboration & history milestone 2026-07-04 14:52:15 +02:00
fable-5 added the
backend
collab
 labels 2026-07-04 14:52:15 +02:00
opus-4-8 commented 2026-07-08 17:36:00 +02:00
Collaborator
Copy Link

Verified on Test and Int (commit 7d8f331, pipeline green — all CI + CD jobs incl. Int promotion).

Live end-to-end on Test: logged in as fixture-user → created a page → GET /pages/:id/collab-token (mode rw) → connected a real @hocuspocus/provider client to wss://test.dorfteich.cloud/collab → edited the document → within the store debounce, GET /pages/:id/export/markdown (which serves page_content_cache.markdown) returned the edited text. So the collab server is persisting state and refreshing the derived cache live through the proxy. Both collab containers stayed healthy.

What landed (ADR 0003, realtime-collaboration.md §lifecycle):

  • onLoadDocument reconstructs a page's Y.Doc from PostgreSQL by applying pages.ydoc_state and then every page_updates row in seq order.
  • onStoreDocument persists debounced (2 s, max 30 s): appends the delta since the last flush to page_updates, periodically merges the log back into ydoc_state (inline threshold — the session-aware compaction of idle pages stays the separate job, #40), refreshes page_content_cache (plain/markdown/html/outline via the shared derivation, #24), bumps pages.updated_at, and keeps Attachment.pageId pointed at the embedding page (#31). One transaction per flush; flush duration logged.
  • Document size ceiling (MAX_PAGE_DOCUMENT_BYTES, 5 MiB) enforced on store: an oversize document is not persisted and clients are notified via a stateless error so they can revert.

Tests: persistence is an injected port. A DB-backed suite covers the Postgres implementation (store/load round-trip, cache refresh, a 1000-entry update log, size-ceiling rejection, not-found) against an isolated _collab test database; an integration suite covers the hook wiring (two-client sync, survival across a server restart, the size-ceiling stateless notification) with an in-memory fake.

Sequencing note (one acceptance box moved by design): the criterion "REST state writes now return 410" is intentionally deferred to #36. #35 keeps PUT /pages/:id/state working; #36 switches the editor to the collab provider and retires the REST write path (410) in the same change, so the deployed editor is never left unable to save between the two deploys.

Verified on Test and Int (commit 7d8f331, pipeline green — all CI + CD jobs incl. Int promotion). **Live end-to-end on Test:** logged in as fixture-user → created a page → `GET /pages/:id/collab-token` (mode `rw`) → connected a real `@hocuspocus/provider` client to `wss://test.dorfteich.cloud/collab` → edited the document → within the store debounce, `GET /pages/:id/export/markdown` (which serves `page_content_cache.markdown`) returned the edited text. So the collab server is persisting state and refreshing the derived cache live through the proxy. Both collab containers stayed healthy. **What landed (ADR 0003, realtime-collaboration.md §lifecycle):** - `onLoadDocument` reconstructs a page's `Y.Doc` from PostgreSQL by applying `pages.ydoc_state` and then every `page_updates` row in `seq` order. - `onStoreDocument` persists debounced (2 s, max 30 s): appends the delta since the last flush to `page_updates`, periodically merges the log back into `ydoc_state` (inline threshold — the session-aware compaction of idle pages stays the separate job, #40), refreshes `page_content_cache` (plain/markdown/html/outline via the shared derivation, #24), bumps `pages.updated_at`, and keeps `Attachment.pageId` pointed at the embedding page (#31). One transaction per flush; flush duration logged. - Document size ceiling (`MAX_PAGE_DOCUMENT_BYTES`, 5 MiB) enforced on store: an oversize document is not persisted and clients are notified via a stateless error so they can revert. **Tests:** persistence is an injected port. A DB-backed suite covers the Postgres implementation (store/load round-trip, cache refresh, a **1000-entry** update log, size-ceiling rejection, not-found) against an isolated `_collab` test database; an integration suite covers the hook wiring (two-client sync, survival across a server restart, the size-ceiling stateless notification) with an in-memory fake. **Sequencing note (one acceptance box moved by design):** the criterion "REST state writes now return 410" is intentionally deferred to **#36**. #35 keeps `PUT /pages/:id/state` working; #36 switches the editor to the collab provider and retires the REST write path (410) in the same change, so the deployed editor is never left unable to save between the two deploys.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
area:auth

   
area:docs

   
area:export

   
area:ops

   
area:storage

   
area:supply-chain

   
auth

Authentication, permissions, quotas

  
backend

API server (apps/api)

  
blocked

Blocked on external dependency/infrastructure

  
collab

Collaboration server (apps/collab)

  
deployment

Docker, CI/CD, hosting

  
docs

Documentation

  
effort:L

   
effort:M

   
effort:S

   
frontend

SPA (apps/web)

  
plugins

Plugin system and SDK

  
qa

Tests, fixtures, e2e

  
vs-nfd

   
vs-nfd:blocker
No Label
area:auth
area:docs
area:export
area:ops
area:storage
area:supply-chain
auth
backend
blocked
collab
deployment
docs
effort:L
effort:M
effort:S
frontend
plugins
qa
vs-nfd
vs-nfd:blocker
Milestone
Clear milestone
No items
No Milestone
M3 — Real-time collaboration & history
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: stwaidele/dorfteich#35
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?