Go to file
Claude Opus 4.8 4755c18ef5
All checks were successful
CD / Build and push images (push) Successful in 4m9s
CI / Lint, typecheck, test (push) Successful in 2m50s
CI / Auth e2e pack (push) Successful in 3m44s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 8s
CD / Smoke tests against Test (push) Successful in 1m15s
CD / Promote to Int (push) Successful in 11s
Add conversion job queue and pandoc sidecar integration (#62)
Import/export conversions run asynchronously against an internal pandoc-server
sidecar with limits and graceful failure (ADR 0009). This is the plumbing;
the import (#63) and export (#65) features enqueue jobs onto it.

Sidecar & config:
- pandoc/core:3.6 in HTTP server mode added to the Compose stack, internal
  network only, with a wget healthcheck on /version; the api depends on it
  healthy and reaches it via the new PANDOC_URL env (default http://pandoc:3030).
- readyz gains a warning-level `converter` check: an unreachable sidecar
  degrades import/export but never flips the instance to unready (new `warn`
  status on ReadinessCheck).

Conversion flow (apps/api/src/import-export/):
- ConversionJob table (per-request work queue, distinct from the name-keyed
  maintenance Job table): owner, formats, input/result bytes, status, attempts,
  lockedAt. Migration + owner cascade.
- PandocConverter (abstract) + PandocServerConverter: POST / with
  {text,from,to,standalone}; binary input formats (docx/odt/…) are base64-encoded
  in `text`; 60 s AbortController timeout; input/output size caps. Failures map
  to distinct localized codes — converter_unavailable / converter_timeout
  (retryable) and conversion_failed (final).
- ConversionWorker: claims one job at a time with `FOR UPDATE SKIP LOCKED`
  (safe against overlapping sweeps and a second process), recovers a stale
  RUNNING lock, retries transient failures up to 3 attempts then fails. A 2 s
  sweep plus wake-on-enqueue means a queued job survives an API restart.
- ConversionJobService.enqueue (size-limited) + owner-scoped GET /jobs/:id
  (poll) and GET /jobs/:id/result (stream the output); a foreign/unknown id is
  404. ConversionJobView in @dorfteich/shared.

Tests:
- conversion-job.e2e.db.test.ts (fake converter injected via a new createTestApp
  override hook): enqueue→convert→poll→result; foreign/unknown job 404; a
  persisted PENDING job picked up by a fresh app's worker (restart survival);
  sidecar-down fails after 3 retries while the API stays healthy.
- pandoc.converter.test.ts: success, non-200→conversion_failed, refused→
  converter_unavailable, and a delay-injecting server→converter_timeout.
- Verified locally against a real pandoc/core:3.6 container: markdown→html,
  markdown→docx (valid PK/OOXML bytes), and a docx→markdown round-trip.

Local: typecheck, lint, i18n:check, build all green; api 193 tests
(9 new), shared 121, web 50.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-10 04:06:27 +02:00
.gitea/workflows Add non-image attachments with allowlist, SVG policy, and file managers (#61) 2026-07-10 02:52:40 +02:00
apps Add conversion job queue and pandoc sidecar integration (#62) 2026-07-10 04:06:27 +02:00
deploy Add conversion job queue and pandoc sidecar integration (#62) 2026-07-10 04:06:27 +02:00
docs/architecture Enforce permissions in API guards and retire interim access (#52) 2026-07-09 16:31:41 +02:00
packages/shared Add conversion job queue and pandoc sidecar integration (#62) 2026-07-10 04:06:27 +02:00
scripts Fix e2e static server crashing on an abruptly-dropped collab WebSocket (#38) 2026-07-08 22:34:01 +02:00
.dockerignore Add production Dockerfiles and the Compose stack with dev overlay 2026-07-04 19:30:21 +02:00
.editorconfig Scaffold pnpm monorepo with lint, format, and test tooling 2026-07-04 19:06:27 +02:00
.gitignore Add file storage service and image upload API (#27) 2026-07-08 10:35:03 +02:00
.prettierignore Add M2 fixtures and content regression pack (#32) 2026-07-08 13:14:48 +02:00
.prettierrc.json Scaffold pnpm monorepo with lint, format, and test tooling 2026-07-04 19:06:27 +02:00
eslint.config.mjs Run the CI auth pack against the production build 2026-07-05 06:16:27 +02:00
LICENSE Add architecture documentation, ADRs, and operations concept 2026-07-04 14:36:16 +02:00
package.json Scaffold pnpm monorepo with lint, format, and test tooling 2026-07-04 19:06:27 +02:00
pnpm-lock.yaml Add non-image attachments with allowlist, SVG policy, and file managers (#61) 2026-07-10 02:52:40 +02:00
pnpm-workspace.yaml Add user, identity, session, and auth-support data model 2026-07-05 00:42:22 +02:00
README.md Scaffold pnpm monorepo with lint, format, and test tooling 2026-07-04 19:06:27 +02:00
tsconfig.base.json Scaffold pnpm monorepo with lint, format, and test tooling 2026-07-04 19:06:27 +02:00

Dorfteich

Dorfteich is an open-source wiki system built around ponds (German: Teiche) — self-contained wiki spaces that people and teams organize freely with hierarchical labels, directories, and Obsidian-style page relations. Pages are edited in a collaborative WYSIWYG editor with live cursors and offline support.

Key features

  • Real-time collaboration — multiple people edit the same page simultaneously; everyone sees the other participants' cursors and input live. Offline edits merge conflict-free on reconnect (CRDT-based).
  • Ponds — isolated wiki spaces with their own members, permissions, fonts, and page organization. Every registered person gets a personal pond.
  • Flexible organization — hierarchical labels, free page ordering, [[wikilinks]] with backlinks. A classic page tree is possible but never enforced.
  • Fine-grained permissions — roles (Site Admin, Pond Admin, Editor, Reader, Public) can be granted per pond, per label, or per page; the most specific setting wins.
  • Import & export — Markdown as the primary exchange format, plus best-effort structural import from Word/OpenOffice and export to Word/OpenOffice/PDF.
  • Plugins — sandboxed extensions (custom blocks, styles, page tools) installable at runtime without redeploying the instance.
  • Self-hosting first — a single docker compose up plus a guided first-run setup wizard yields a working instance.

Repository layout

Path Contents
docs/architecture/ Architecture documentation: ADRs, data model, permission model, collaboration and plugin concepts, deployment and operations
apps/ Application packages (web frontend, API server, collaboration server) — created as implementation proceeds
packages/ Shared packages (types, permission logic, plugin SDK)
deploy/ Docker Compose stacks and deployment tooling

Development

Requirements: Node.js ≥ 22 and pnpm (npm install -g pnpm).

pnpm install        # install all workspace dependencies
pnpm lint           # ESLint + Prettier check across the repo
pnpm typecheck      # TypeScript --noEmit in every package
pnpm test           # Vitest in every package
pnpm build          # build every package (dependency order)

The workspace packages live under apps/ (web, api, collab) and packages/ (shared). Shared logic goes into packages/shared and is imported as @dorfteich/shared — never copy code between apps.

Status

The project is in the architecture and backlog phase. Implementation stories are tracked as issues in this repository. Start reading at docs/architecture/README.md.

Contributing

Code, comments, and documentation are written in English. Write clear code that humans can follow easily; when in doubt, prefer readability over cleverness. All contributions are accepted under the MIT license.