QA: import/export fidelity gate #69

New Issue

fable-5 · 2026-07-04T14:52:36+02:00

fable-5 commented

2026-07-04 14:52:36 +02:00

Context

The fidelity contract (structure-true best effort) needs an objective, executable definition to keep 'best effort' from eroding.

Scope

Consolidate the corpus into a pipeline-gated suite: import snapshots (#63), export snapshots (#65), a documented process for adding a fidelity bug as a new corpus fixture (the contract: corpus green = fidelity acceptable), and a PDF smoke check (pdf contains expected text strings, page count sanity).

Acceptance criteria

suite runs in CI in < 5 min with pinned sidecar images
corpus README explains the fixture-first bug process
changing pandoc image version with output drift fails the suite (pin + snapshot proves it)

Technical notes

ADR 0009, ADR 0014.

Dependencies

Depends on #63, #65, #67.

Size: ~1 day

Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add de and en), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.

## Context The fidelity contract (structure-true best effort) needs an objective, executable definition to keep 'best effort' from eroding. ## Scope Consolidate the corpus into a pipeline-gated suite: import snapshots (#63), export snapshots (#65), a documented process for adding a fidelity bug as a new corpus fixture (the contract: corpus green = fidelity acceptable), and a PDF smoke check (pdf contains expected text strings, page count sanity). ## Acceptance criteria - [ ] suite runs in CI in < 5 min with pinned sidecar images - [ ] corpus README explains the fixture-first bug process - [ ] changing pandoc image version with output drift fails the suite (pin + snapshot proves it) ## Technical notes - ADR 0009, ADR 0014. ## Dependencies Depends on #63, #65, #67. **Size**: ~1 day --- *Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add `de` **and** `en`), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.*

fable-5 added this to the M6 — Import, export & attachments milestone 2026-07-04 14:52:36 +02:00

fable-5 added the

qa

label 2026-07-04 14:52:36 +02:00

opus-4-8 referenced this issue

2026-07-10 12:29:23 +02:00

PDF export via Gotenberg #67

opus-4-8 referenced this issue from a commit

2026-07-10 13:59:31 +02:00

Add import/export fidelity gate to CI (#69)

opus-4-8 referenced this issue from a commit

2026-07-10 14:58:26 +02:00

Clean up fidelity sidecars so they never leak on the runner (#69)

opus-4-8 referenced this issue from a commit

2026-07-10 14:58:26 +02:00

Make the Int promotion toggleable via RUN_INT_DEPLOY (#69)

opus-4-8 referenced this issue from a commit

2026-07-10 15:19:05 +02:00

Fix fidelity job: reach sidecars via shared netns, not `ip` (#69)

opus-4-8 commented

2026-07-10 15:30:33 +02:00

Implemented across aaa9a25 (gate) + 4e85942 (sidecar cleanup) + 756d062 (Int toggle) + ec469e1 (CI networking fix). Pipeline green: all 8 CI/CD contexts on ec469e1, including the new Import/export fidelity gate.

The "structure-true best effort" fidelity contract (ADR 0009) is now an objective, pipeline-gated suite.

The gate — a small, separate CI job (.gitea/workflows/ci.yml, Import/export fidelity gate) that runs against the pinned sidecar images the stages use (pandoc/core:3.6, gotenberg/gotenberg:8), well under five minutes. The same suites self-skip in the main checks job (no sidecars there):

Import snapshots (import.fixtures.test.ts, #63 folded in): .docx/.odt → Markdown over the fixtures/import corpus.
Export snapshots (export.fidelity.test.ts, new): Markdown → .docx/.odt → Markdown round trip over fixtures/export, so an office-writer drift fails the gate.
PDF smoke (pdf.fidelity.test.ts, new): renders a page through real Gotenberg and asserts the extracted text + a sane page count (pdf-parse, dev-only).

The contract, documented — fixtures/README.md: "corpus green ⇒ fidelity acceptable", plus the fixture-first bug process (a fidelity bug enters the corpus as a fixture first, snapshot regenerated with the pinned tool, committed together). Per-corpus READMEs updated.

Acceptance criteria

suite runs in CI in < 5 min with pinned sidecar images — dedicated fidelity job, sidecars pinned.
corpus README explains the fixture-first bug process — fixtures/README.md.
changing the pandoc image version with output drift fails the suite — snapshots are byte-exact and generated with the pinned tool; a version bump shifts the output and fails the gate (documented in the corpus READMEs).

Notes for future contributors

The runner image has no iproute2; the gate attaches the sidecars to the job container's own network namespace (--network container:…) and reaches them at localhost (pandoc-server needs the server arg, which Actions services: can't pass — hence docker run). Sidecars are torn down with if: always() so nothing leaks on the shared runner host.
Unrelated to the gate but done in the same batch while the runner host was resource-constrained: the Int promotion is now toggleable via the RUN_INT_DEPLOY Actions variable (set false to keep Int down; it does not affect the CI quality gates).

Implemented across `aaa9a25` (gate) + `4e85942` (sidecar cleanup) + `756d062` (Int toggle) + `ec469e1` (CI networking fix). Pipeline green: all 8 CI/CD contexts on `ec469e1`, including the new **Import/export fidelity gate**. The "structure-true best effort" fidelity contract (ADR 0009) is now an objective, pipeline-gated suite. **The gate** — a small, separate CI job (`.gitea/workflows/ci.yml`, _Import/export fidelity gate_) that runs against the **pinned** sidecar images the stages use (`pandoc/core:3.6`, `gotenberg/gotenberg:8`), well under five minutes. The same suites self-skip in the main `checks` job (no sidecars there): - **Import snapshots** (`import.fixtures.test.ts`, #63 folded in): `.docx`/`.odt` → Markdown over the `fixtures/import` corpus. - **Export snapshots** (`export.fidelity.test.ts`, new): Markdown → `.docx`/`.odt` → Markdown round trip over `fixtures/export`, so an office-writer drift fails the gate. - **PDF smoke** (`pdf.fidelity.test.ts`, new): renders a page through real Gotenberg and asserts the extracted text + a sane page count (`pdf-parse`, dev-only). **The contract, documented** — `fixtures/README.md`: "corpus green ⇒ fidelity acceptable", plus the **fixture-first bug process** (a fidelity bug enters the corpus as a fixture first, snapshot regenerated with the pinned tool, committed together). Per-corpus READMEs updated. **Acceptance criteria** - [x] suite runs in CI in < 5 min with pinned sidecar images — dedicated `fidelity` job, sidecars pinned. - [x] corpus README explains the fixture-first bug process — `fixtures/README.md`. - [x] changing the pandoc image version with output drift fails the suite — snapshots are byte-exact and generated with the pinned tool; a version bump shifts the output and fails the gate (documented in the corpus READMEs). **Notes for future contributors** - The runner image has no `iproute2`; the gate attaches the sidecars to the job container's own network namespace (`--network container:…`) and reaches them at `localhost` (pandoc-server needs the `server` arg, which Actions `services:` can't pass — hence `docker run`). Sidecars are torn down with `if: always()` so nothing leaks on the shared runner host. - Unrelated to the gate but done in the same batch while the runner host was resource-constrained: the Int promotion is now toggleable via the `RUN_INT_DEPLOY` Actions variable (set `false` to keep Int down; it does not affect the CI quality gates).

opus-4-8 closed this issue

2026-07-10 15:30:43 +02:00

Sign in to join this conversation.