Commit Graph

7 Commits

Author SHA1 Message Date
aaa9a253ae Add import/export fidelity gate to CI (#69)
Some checks failed
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CD / Build and push images (push) Failing after 27m51s
CI / Lint, typecheck, test (push) Successful in 3m11s
CI / Auth e2e pack (push) Successful in 4m0s
CI / Import/export fidelity gate (push) Failing after 36s
CI / Build container images (push) Has been skipped
Make the "structure-true best effort" fidelity contract (ADR 0009) an
objective, pipeline-gated suite so "best effort" cannot erode silently.

- New CI job "Import/export fidelity gate" (.gitea/workflows/ci.yml) runs
  the corpus suites against the pinned sidecar images the stages use
  (pandoc/core:3.6, gotenberg/gotenberg:8), started via docker run and
  reached over the host gateway. Small and separate so it stays well under
  five minutes; the suites self-skip in the main checks job (no sidecars).
- Export fidelity: fixtures/export corpus + gen-export-fixtures.mjs +
  export.fidelity.test.ts — exports Markdown to docx/odt through the real
  pinned pandoc and reads it back, snapshotting the round trip so a writer
  drift (ours or a version bump) fails the gate.
- PDF smoke: pdf.fidelity.test.ts renders a page through real Gotenberg and
  asserts the extracted text and a sane page count (pdf-parse, dev-only).
- Fidelity contract doc: fixtures/README.md defines "corpus green = fidelity
  acceptable" and the fixture-first bug process; per-corpus READMEs updated.

Because the snapshots are byte-exact and generated with the pinned tools,
bumping a sidecar without regenerating shifts the output and fails the suite
(AC3). The import corpus (#63) is folded into the same gate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-10 13:59:10 +02:00
546e8279ac Import .docx and .odt documents as new pages (#63)
All checks were successful
CD / Build and push images (push) Successful in 3m19s
CI / Lint, typecheck, test (push) Successful in 2m55s
CI / Auth e2e pack (push) Successful in 3m45s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m13s
CD / Promote to Int (push) Successful in 11s
Uploading a Word/OpenOffice document to POST /ponds/:id/import enqueues a
conversion job (the #62 queue) that produces a new page in the pond; the
client polls GET /jobs/:id for the created resultPageId.

Pipeline (ImportService, ADR 0009): pandoc-server is stateless and hands
back a document's media no other way, so we convert in two passes —
docx/odt → html with embed-resources inlines every image as a data: URI,
then html → gfm produces clean structural Markdown with those data URIs
still inline. Embedded images are stored as pond files (with quota
accounting) and their references rewritten to file ids on the Markdown
text before parsing (the editor parser only admits png/jpeg/gif/webp data
URIs); an image whose bytes the upload pipeline rejects is dropped, not
fatal. The title comes from a leading top-level heading (removed from the
body) else the file name. The page is created from the resulting Yjs state.

The shared conversion worker routes import-kind jobs to the pipeline via a
token (breaking a module cycle), so import inherits the queue's locking,
retry, and restart-survival. Media stored during a failed attempt is rolled
back; a pond that runs out of storage fails the job with quota_exceeded.

- schema: ConversionJob gains pond_id / source_name / result_page_id
  (migration 20260710041215_import_pages_conversion); ConversionJobView
  gains resultPageId.
- PagesService.createWithState / yjs-content docToState build a page from a
  prepared document; FilesService.linkAttachmentsToPage links import media.
- fixtures/import/: representative .docx/.odt corpus (headings, lists,
  nested lists, tables, images, links, bold/italic) with expected-Markdown
  snapshots; scripts/gen-import-fixtures.mjs regenerates them.
- tests: import.service.db.test.ts drives the full pipeline with a fake
  converter (CI); import.fixtures.test.ts runs the real two-pass conversion
  over the corpus and a 50-page timing check against a reachable sidecar.
- i18n: import_unsupported_format (de+en). Limits documented (25 MiB input,
  60 s per pass).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-10 07:34:43 +02:00
7b17e05826 Fix e2e static server crashing on an abruptly-dropped collab WebSocket (#38)
All checks were successful
CD / Build and push images (push) Successful in 56s
CI / Lint, typecheck, test (push) Successful in 1m56s
CI / Auth e2e pack (push) Successful in 2m25s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m12s
CD / Promote to Int (push) Successful in 11s
Root cause of the CI-only Auth-e2e failure (found in the runner log): the
offline pack started with `connect ECONNREFUSED :5173` — the e2e static web
server had already exited. Its `/collab` WebSocket proxy attached an 'error'
handler to the upstream socket but not to the client socket, so a WebSocket
dropped abruptly (a context closing at the end of the collab pack, or the
offline toggle) raised an unhandled 'error' → uncaught exception → the whole
test server crashed, failing the next pack. It only reproduced on the CI
runner (Linux/Node 22 emits 'error'; local macOS/Node 26 emitted 'close').

Handle 'error' on the client socket too, and add a last-resort
`uncaughtException` guard so this throwaway test server can never be taken
down mid-run by a stray socket error. Verified locally by running the collab
and offline packs back-to-back against a single server process.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY
2026-07-08 22:34:01 +02:00
af81b50fa6 Add offline editing: local persistence, PWA shell, offline resolution (#38)
Some checks failed
CD / Build and push images (push) Successful in 2m59s
CI / Lint, typecheck, test (push) Successful in 2m3s
CI / Auth e2e pack (push) Failing after 2m18s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m32s
CD / Promote to Int (push) Successful in 12s
Editing continues without a connection and merges conflict-free on
reconnect (ADR 0003, realtime-collaboration.md §Offline).

- y-indexeddb mirrors every opened page's Y.Doc to IndexedDB, sharing the
  document with the collab provider. The local copy is discarded when the
  page is left after a successful server sync (bounding IndexedDB growth)
  and kept otherwise so offline edits survive to the next visit.
- vite-plugin-pwa service worker precaches the app shell (build assets only)
  with a navigation fallback; `/api` and `/collab` are denylisted and there
  is no runtime caching, so API responses are never cached or poisoned.
- Offline page resolution WITHOUT caching API responses: the app itself
  persists the small metadata it needs to reopen a visited page (page/pond
  ids + slugs, bounded LRU in localStorage) and the last signed-in user, so
  after an offline tab reload the app stays signed in, resolves the page, and
  restores its content from IndexedDB. Both are revalidated when the network
  returns (a 401 clears the cached user).
- Local-only UI: a banner when there are edits held only on this device
  (provider `onUnsyncedChanges`), de + en.

Tests: `page-cache` unit test (remember/recall + bounded eviction); a new
`offline` e2e pack (validated locally against the full stack and wired into
CI): edit, reload while offline (shell from the SW, content from IndexedDB),
assert an API call fails offline (no SW API caching), then reconnect and a
second client converges. The e2e static server serves `.webmanifest`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY
2026-07-08 22:08:55 +02:00
7d04c0b594 Switch the editor to live collaboration (#36)
All checks were successful
CD / Build and push images (push) Successful in 2m59s
CI / Lint, typecheck, test (push) Successful in 2m0s
CI / Auth e2e pack (push) Successful in 2m10s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m14s
CD / Promote to Int (push) Successful in 11s
The editor now edits over the collaboration server instead of REST — the
moment Dorfteich becomes collaborative (ADR 0003, realtime-collaboration.md).

Web:
- New `useCollabProvider` hook binds a page's Y.Doc to a HocuspocusProvider.
  The document loads and persists through the collab server (#35); there is
  no REST autosave and no REST seed (a REST seed would fork the doc lineage
  and duplicate content). The collab token is fetched lazily on every
  (re)connect via an async token function, so an expired token is replaced
  transparently and a permission change takes effect on the next reconnect.
- Connection-state UI replaces the save indicator: connecting / connected
  ("Live") / reconnecting / offline, driven by provider status + navigator
  online state. Read-only (`ro`) tokens make the editor non-editable with a
  reason; an oversize-document stateless error (#35) surfaces a banner.
- Removed `use-page-autosave.ts` and `yjs-base64.ts` (no longer used).

API:
- `PUT /pages/:id/state` is retired and returns 410 `rest_state_write_retired`
  (the criterion deferred here from #35). Collab is the sole writer of page
  state; the read paths remain. Removed the now-dead `saveState` service.

e2e / CI:
- The e2e static server proxies the `/collab` WebSocket upgrade (mirrors
  Caddy); vite dev gains a `/collab` ws proxy. The auth-e2e CI job starts the
  collab server and runs a new collab pack.
- New `collab.spec.ts`: two browsers converge on one page (the milestone
  headline), and offline edits continue locally and sync on reconnect. The
  read-only live assertion is a `test.fixme` until real read-only grants
  exist — under interim access seeing and modifying coincide, so no `ro`
  token is issued yet (that arrives with #53). Reworked the api/trash tests
  and the content editor-basics test off the retired REST write path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PGdhRiwU1WRL4XxJfZYipY
2026-07-08 18:00:19 +02:00
d05c36701b Run the CI auth pack against the production build
All checks were successful
CD / Build and push images (push) Successful in 40s
CI / Lint, typecheck, test (push) Successful in 1m14s
CI / Auth e2e pack (push) Successful in 1m39s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m6s
CD / Promote to Int (push) Successful in 10s
The Vite dev server died mid-run on the CI runner (memory pressure),
failing every remaining test with connection refused. The auth-e2e
job now serves apps/web/dist through a dependency-free static server
with SPA fallback and /api proxy (scripts/e2e-static-server.mjs) —
matching the production nginx/Caddy layout and testing the real
build. Server logs are dumped when the job fails. Verified locally:
six of six against the static server.

Part of #20

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 06:16:27 +02:00
e855192d23 Add i18n with i18next, German and English, and a key-parity check
Translation resources live in packages/shared/i18n/<lang>/<ns>.json
(common, errors) and ship with de and en. The web app initializes
react-i18next with bundled resources (?lng= wins, then the browser
language); all shell components use useTranslation and the temporary
t() stub is gone. The api localizes its uniform error bodies via a
minimal i18next instance negotiated from Accept-Language. `pnpm
i18n:check` fails CI when any key is missing in any language, backed
by tested helpers in @dorfteich/shared.

Closes #5

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 19:23:45 +02:00