Claude Opus 4.8
aaa9a253ae
Some checks failed
CD / Deploy to Test (push) Blocked by required conditions
CD / Smoke tests against Test (push) Blocked by required conditions
CD / Promote to Int (push) Blocked by required conditions
CD / Build and push images (push) Failing after 27m51s
CI / Lint, typecheck, test (push) Successful in 3m11s
CI / Auth e2e pack (push) Successful in 4m0s
CI / Import/export fidelity gate (push) Failing after 36s
CI / Build container images (push) Has been skipped
Make the "structure-true best effort" fidelity contract (ADR 0009) an objective, pipeline-gated suite so "best effort" cannot erode silently. - New CI job "Import/export fidelity gate" (.gitea/workflows/ci.yml) runs the corpus suites against the pinned sidecar images the stages use (pandoc/core:3.6, gotenberg/gotenberg:8), started via docker run and reached over the host gateway. Small and separate so it stays well under five minutes; the suites self-skip in the main checks job (no sidecars). - Export fidelity: fixtures/export corpus + gen-export-fixtures.mjs + export.fidelity.test.ts — exports Markdown to docx/odt through the real pinned pandoc and reads it back, snapshotting the round trip so a writer drift (ours or a version bump) fails the gate. - PDF smoke: pdf.fidelity.test.ts renders a page through real Gotenberg and asserts the extracted text and a sane page count (pdf-parse, dev-only). - Fidelity contract doc: fixtures/README.md defines "corpus green = fidelity acceptable" and the fixture-first bug process; per-corpus READMEs updated. Because the snapshots are byte-exact and generated with the pinned tools, bumping a sidecar without regenerating shifts the output and fails the suite (AC3). The import corpus (#63) is folded into the same gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
71 lines
4.1 KiB
Markdown
71 lines
4.1 KiB
Markdown
# Import/export fidelity corpus (issue #69, ADR 0009)
|
|
|
|
The fidelity contract for document conversion is **structure-true best effort**:
|
|
headings, paragraphs, lists, tables, images, links, and inline emphasis are
|
|
preserved; layout (columns, text boxes, exact spacing) is explicitly out of
|
|
scope (ADR 0009). "Best effort" only stays meaningful if it is executable, so
|
|
this corpus **is** the contract:
|
|
|
|
> **The corpus is green ⇒ current fidelity is acceptable.**
|
|
|
|
The gate runs in CI (`.gitea/workflows/ci.yml`, job _Import/export fidelity
|
|
gate_) against the **pinned** sidecar images the stages run — `pandoc/core:3.6`
|
|
and `gotenberg/gotenberg:8`. It is deliberately a small, separate job that stays
|
|
well under five minutes; the same suites self-skip in the main `checks` job,
|
|
which has no sidecars.
|
|
|
|
## What the gate covers
|
|
|
|
| Suite | File | What it pins |
|
|
| ---------------- | ---------------------------------------------------- | ------------------------------------------------------------------------------------ |
|
|
| Import snapshots | `apps/api/src/import-export/import.fixtures.test.ts` | `.docx`/`.odt` → Markdown for the [`import/`](./import) corpus |
|
|
| Export snapshots | `apps/api/src/import-export/export.fidelity.test.ts` | Markdown → `.docx`/`.odt` → Markdown round trip for the [`export/`](./export) corpus |
|
|
| PDF smoke | `apps/api/src/import-export/pdf.fidelity.test.ts` | a rendered PDF contains the expected text, with a sane page count |
|
|
|
|
Each corpus directory has its own README describing its files.
|
|
|
|
## The fixture-first bug process (the contract in practice)
|
|
|
|
A fidelity problem is not fixed by hand-patching output until one document looks
|
|
right — that lets "best effort" quietly erode. Instead, **every fidelity bug
|
|
enters the corpus first**:
|
|
|
|
1. **Reproduce it as a fixture.** Add the smallest document that exhibits the
|
|
problem to the relevant corpus (`import/` a source document, `export/` a
|
|
source `.md`). If the shape is new, extend an existing source rather than
|
|
adding a near-duplicate.
|
|
2. **Regenerate the snapshot** with the pinned sidecar so it records _today's_
|
|
real output:
|
|
- import: `PANDOC_URL=… node scripts/gen-import-fixtures.mjs`
|
|
- export: `PANDOC_URL=… node scripts/gen-export-fixtures.mjs`
|
|
(start the sidecar with `docker run --rm -p 3030:3030 pandoc/core:3.6 server`).
|
|
3. **Commit the fixture and its snapshot together**, and review the snapshot
|
|
diff — it is the human-readable statement of what the conversion does. If the
|
|
snapshot encodes the _bug_, fix the pipeline and regenerate until the diff
|
|
shows the intended structure; if it encodes a genuine pinned-tool limitation
|
|
(see below), the snapshot _is_ the accepted behaviour and the fix is
|
|
documentation, not code.
|
|
4. **The gate now defends it.** Any later regression — ours or a tool's —
|
|
changes that document's output and fails the suite.
|
|
|
|
## Why pinning matters (and how a version bump fails the gate)
|
|
|
|
The snapshots are byte-exact and generated with the pinned sidecar versions.
|
|
pandoc changes its writers between releases (3.10 wraps lists and pads tables
|
|
differently from 3.6), so **bumping `pandoc/core` — or `gotenberg/gotenberg` —
|
|
without regenerating shifts the output and fails the suite.** That is the point:
|
|
a version change becomes a reviewed snapshot diff, never a silent drift. To bump
|
|
a sidecar: change the pin in `deploy/compose/docker-compose.yml`, the CI job, and
|
|
these tests' defaults; regenerate the corpus; review the diff; commit together.
|
|
|
|
## Known pinned-tool limitations (accepted, not bugs)
|
|
|
|
- **ODT loses image alt text and table header rows**: pandoc's HTML→ODT _writer_
|
|
does not encode either, so the ODT sources genuinely lack them (the DOCX
|
|
variants keep both).
|
|
- **ODT loosens tight lists and flattens fenced code blocks** to plain
|
|
paragraphs on the export round trip; DOCX preserves both.
|
|
|
|
These are recorded in the corpus snapshots on purpose — extend the corpus rather
|
|
than chasing layout parity.
|