#209: pandoc reference documents carry the VS-NfD marking for DOCX/ODT
Some checks failed
CI / Lint, typecheck, test (pull_request) Failing after 6m34s
CI / Import/export fidelity gate (pull_request) Has been skipped
CI / Build container images (pull_request) Has been skipped
CI / Auth e2e pack (pull_request) Has been skipped

reference-vs-nfd.docx/.odt ship as derived binaries: the pinned pandoc's
default reference documents plus a header and footer with the marking —
part of the document's page setup, so it repeats on every page in Word
and LibreOffice and is not deletable body text. Source of truth is
scripts/gen-classified-reference-docs.mjs (wording from shared
classificationMarking(); maintenance documented in assets/README.md).
The converter passes reference docs to pandoc-server via in-request
files + reference-doc; the worker attaches them for marked docx/odt jobs
(job option {marking}, as in #208). Unclassified exports pass nothing
and are unchanged (pinned by fake-converter test). Fidelity suite
asserts against real pandoc 3.6 that marked outputs carry the
header/footer parts and unmarked ones do not; per-page repetition
verified via LibreOffice 25.8 headless PDF (5/5 pages, 2 markings each,
both formats). Word: quick manual look pending (sample files in the
workspace), procedure documented in assets/README.md.

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Fable 5 2026-07-31 07:09:47 +02:00
parent c2df7c0c23
commit 74a9e495e4
13 changed files with 312 additions and 4 deletions

View File

@ -21,6 +21,7 @@ RUN pnpm install --frozen-lockfile --filter @dorfteich/api... \
# needed for migrate-on-start) at /out.
&& pnpm --filter @dorfteich/api deploy --prod --legacy /out \
&& cp -r apps/api/dist /out/dist \
&& cp -r apps/api/assets /out/assets \
&& cp -r /repo/fonts /out/fonts
FROM node:22.15.1-alpine

45
apps/api/assets/README.md Normal file
View File

@ -0,0 +1,45 @@
# Runtime assets
## `reference-vs-nfd.docx` / `reference-vs-nfd.odt` (issue #209, ADR 0022)
Pandoc reference documents for the DOCX/ODT export of a **classified**
page: their page setup defines a header and footer carrying the VS-NfD
marking, which pandoc copies into its output — so the marking repeats on
every page in Word and LibreOffice and is not deletable body text.
Unclassified exports pass no reference document and are unchanged.
These are **derived binaries — never edit them by hand.** Source of truth
is `../scripts/gen-classified-reference-docs.mjs`: it takes the default
reference documents of the pinned sidecar (`pandoc/core:3.6`, the exact
image the stages run) and injects the header/footer, with the wording from
`classificationMarking()` in `@dorfteich/shared` (single source, ADR
0022). Regenerate — after a pandoc pin bump, a wording change, or a layout
tweak in the script — with Docker running:
```sh
pnpm --filter @dorfteich/shared build # the script imports the wording
node apps/api/scripts/gen-classified-reference-docs.mjs
```
Commit script and binaries together. The fidelity suite
(`export.fidelity.test.ts`) asserts against the real pinned pandoc that a
marked export carries the header/footer parts and an unmarked one does
not.
### Per-page verification in the office suites
After regenerating, confirm the marking repeats on **every** page of a
multi-page export (not just structurally in the XML):
1. Produce a marked multi-page export (any classified page with a few
screens of text, exported to `.docx` and `.odt`).
2. **LibreOffice** (scriptable):
`soffice --headless --convert-to pdf <file>` and check every PDF page
shows the marking twice (header + footer) — e.g. with `pypdf`.
3. **Word**: open the `.docx`, check header and footer on every page
(print preview). Word's AppleScript/sandbox makes this hard to script —
this step is a quick manual look.
Last verified 2026-07-31 (pandoc 3.6 output): LibreOffice 25.8, both
formats, 5/5 pages with 2 markings each. Word: manual check pending —
sample files in the workspace under `doku/209-marked-sample.docx/.odt`.

Binary file not shown.

Binary file not shown.

View File

@ -786,8 +786,8 @@ model ConversionJob {
/// other job kind, whose payload the general retention (#233) prunes.
expiresAt DateTime? @map("expires_at")
/// Kind-specific job options (issue #117): a vault import carries
/// `{parentPageId, labelIds, frontmatterMode}`; a PDF export of a
/// classified page carries `{marking}` (issue #208). Null otherwise.
/// `{parentPageId, labelIds, frontmatterMode}`; a PDF/DOCX/ODT export of a
/// classified page carries `{marking}` (issues #208/#209). Null otherwise.
options Json?
createdAt DateTime @default(now()) @map("created_at")
updatedAt DateTime @updatedAt @map("updated_at")

View File

@ -0,0 +1,118 @@
/**
* Regenerate the classified reference documents (issue #209, ADR 0022):
* `apps/api/assets/reference-vs-nfd.docx` / `.odt`.
*
* The DOCX/ODT export of a classified page passes these to pandoc via
* `--reference-doc`; pandoc copies the reference's page setup including
* headers and footers into its output, which is how the VS-NfD marking
* repeats on every page in Word and LibreOffice without being deletable
* body text.
*
* The binaries are DERIVED files: base = the default reference documents of
* the PINNED pandoc (`pandoc/core:3.6`, the exact sidecar the stages run),
* plus a header and footer carrying the marking. Never edit the binaries by
* hand edit this script and re-run it (Docker required):
*
* node apps/api/scripts/gen-classified-reference-docs.mjs
*
* The marking wording comes from @dorfteich/shared (single source, ADR
* 0022); the shared package must be built (`pnpm --filter @dorfteich/shared
* build`).
*/
import { execFileSync } from 'node:child_process';
import { mkdirSync, writeFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { classificationMarking } from '@dorfteich/shared';
import { strToU8, strFromU8, unzipSync, zipSync } from 'fflate';
const PANDOC_IMAGE = 'pandoc/core:3.6';
const MARKING = classificationMarking('vs_nfd');
const outDir = join(dirname(fileURLToPath(import.meta.url)), '../assets');
function defaultReference(name) {
return execFileSync('docker', ['run', '--rm', PANDOC_IMAGE, '--print-default-data-file', name], {
maxBuffer: 64 * 1024 * 1024,
});
}
function escapeXml(value) {
return value.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
}
/** DOCX: add word/header1.xml + word/footer1.xml, register them in the
* content types and document relationships, and reference them from the
* document's sectPr Word repeats them on every page. */
function patchDocx(bytes) {
const zip = unzipSync(new Uint8Array(bytes));
const marking = escapeXml(MARKING);
const partXml = (root) =>
`<?xml version="1.0" encoding="UTF-8" standalone="yes"?>\n` +
`<w:${root} xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main">` +
`<w:p><w:pPr><w:jc w:val="center"/></w:pPr>` +
`<w:r><w:rPr><w:b/></w:rPr><w:t xml:space="preserve">${marking}</w:t></w:r>` +
`</w:p></w:${root}>`;
zip['word/header1.xml'] = strToU8(partXml('hdr'));
zip['word/footer1.xml'] = strToU8(partXml('ftr'));
const types = strFromU8(zip['[Content_Types].xml']);
zip['[Content_Types].xml'] = strToU8(
types.replace(
'</Types>',
'<Override PartName="/word/header1.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.header+xml" />' +
'<Override PartName="/word/footer1.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.footer+xml" />' +
'</Types>',
),
);
const rels = strFromU8(zip['word/_rels/document.xml.rels']);
zip['word/_rels/document.xml.rels'] = strToU8(
rels.replace(
'</Relationships>',
'<Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/header" Id="rIdVsNfdHeader" Target="header1.xml" />' +
'<Relationship Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/footer" Id="rIdVsNfdFooter" Target="footer1.xml" />' +
'</Relationships>',
),
);
const doc = strFromU8(zip['word/document.xml']);
if (!doc.includes('<w:sectPr>')) throw new Error('reference.docx has no sectPr');
zip['word/document.xml'] = strToU8(
doc.replace(
'<w:sectPr>',
'<w:sectPr>' +
'<w:headerReference xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" w:type="default" r:id="rIdVsNfdHeader" />' +
'<w:footerReference xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships" w:type="default" r:id="rIdVsNfdFooter" />',
),
);
return zipSync(zip);
}
/** ODT: give the Standard master page a header with the marking and put the
* marking next to the existing page number in its footer LibreOffice
* repeats master-page headers/footers on every page. */
function patchOdt(bytes) {
const zip = unzipSync(new Uint8Array(bytes));
const marking = escapeXml(MARKING);
const styles = strFromU8(zip['styles.xml']);
if (!styles.includes('<style:footer>')) throw new Error('reference.odt has no footer');
const patched = styles
.replace(
'<style:footer>',
`<style:header><text:p text:style-name="MP1">${marking}</text:p></style:header><style:footer>`,
)
.replace(
'<style:footer>\n <text:p text:style-name="MP1">',
`<style:footer>\n <text:p text:style-name="MP1">${marking} · `,
);
zip['styles.xml'] = strToU8(patched);
return zipSync(zip);
}
mkdirSync(outDir, { recursive: true });
writeFileSync(join(outDir, 'reference-vs-nfd.docx'), patchDocx(defaultReference('reference.docx')));
writeFileSync(join(outDir, 'reference-vs-nfd.odt'), patchOdt(defaultReference('reference.odt')));
console.log(`generated reference-vs-nfd.docx/.odt in ${outDir} (marking: ${MARKING})`);

View File

@ -1,3 +1,6 @@
import { readFile } from 'node:fs/promises';
import { join } from 'node:path';
import { Injectable, OnModuleDestroy, OnModuleInit } from '@nestjs/common';
import { ModuleRef } from '@nestjs/core';
import { ConversionJob } from '@prisma/client';
@ -66,10 +69,34 @@ export class ConversionWorker implements OnModuleInit, OnModuleDestroy {
to: job.targetFormat,
input: Buffer.from(conversionInputOf(job)),
standalone: job.standalone,
referenceDoc: await this.classifiedReferenceDoc(job),
});
return { bytes: result.output, mimeType: result.mimeType };
}
/**
* The classified reference document for a marked docx/odt export (issue
* #209, ADR 0022): pandoc copies its header/footer which carry the
* VS-NfD marking into the output, so the marking repeats on every page
* in Word/LibreOffice and is not deletable body text. Only present when
* the enqueue put a `marking` into the job options; the binaries ship in
* `apps/api/assets/` (see `scripts/gen-classified-reference-docs.mjs`).
*/
private async classifiedReferenceDoc(
job: ConversionJob,
): Promise<{ name: string; bytes: Buffer } | undefined> {
const marked = Boolean((job.options as { marking?: string } | null)?.marking);
if (!marked || (job.targetFormat !== 'docx' && job.targetFormat !== 'odt')) return undefined;
const name = `reference-vs-nfd.${job.targetFormat}`;
const cached = this.referenceDocs.get(name);
if (cached) return { name, bytes: cached };
const bytes = await readFile(join(__dirname, '../../assets', name));
this.referenceDocs.set(name, bytes);
return { name, bytes };
}
private readonly referenceDocs = new Map<string, Buffer>();
onModuleInit(): void {
if (this.config.env.NODE_ENV === 'test') return; // tests drive drain() directly
this.timer = setInterval(() => this.drainSafely(), SWEEP_MS);

View File

@ -1,6 +1,8 @@
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import { classificationMarking } from '@dorfteich/shared';
import { strFromU8, unzipSync } from 'fflate';
import { beforeAll, describe, expect, it, TestContext } from 'vitest';
import { AppConfig } from '../config/app-config.service';
@ -8,6 +10,8 @@ import { AppConfig } from '../config/app-config.service';
import { markdownForDocument } from './export-markdown';
import { PandocServerConverter } from './pandoc.converter';
const MARKING = classificationMarking('vs_nfd')!;
/**
* Export fidelity regression (issue #69, ADR 0009): exports the committed
* Markdown corpus to `.docx`/`.odt` through the real pinned pandoc and reads
@ -70,4 +74,68 @@ describe('export fidelity corpus (real pandoc, issue #69)', () => {
});
}
}
// The classified reference documents (issue #209, ADR 0022): a marked
// export must carry the VS-NfD marking in the document's own header/footer
// definition (repeats per page in Word/LibreOffice, not deletable body
// text); an unmarked export must not. Asserted structurally against the
// real pinned pandoc; the body round-trip above stays untouched by the
// reference doc (headers are outside the content pandoc reads back).
it('a marked docx export carries the marking in header1.xml/footer1.xml; unmarked does not', async (ctx: TestContext) => {
if (!reachable) ctx.skip();
const referenceDoc = {
name: 'reference-vs-nfd.docx',
bytes: readFileSync(join(process.cwd(), 'assets/reference-vs-nfd.docx')),
};
const marked = await converter.convert({
from: 'gfm',
to: 'docx',
input: Buffer.from('# Marked\n\nbody', 'utf8'),
standalone: true,
referenceDoc,
});
const parts = unzipSync(new Uint8Array(marked.output));
const header = strFromU8(parts['word/header1.xml']!);
const footer = strFromU8(parts['word/footer1.xml']!);
expect(header).toContain(MARKING);
expect(footer).toContain(MARKING);
expect(strFromU8(parts['word/document.xml']!)).toContain('headerReference');
const unmarked = await converter.convert({
from: 'gfm',
to: 'docx',
input: Buffer.from('# Open\n\nbody', 'utf8'),
standalone: true,
});
const openParts = unzipSync(new Uint8Array(unmarked.output));
expect(openParts['word/header1.xml']).toBeUndefined();
});
it('a marked odt export carries the marking in its master-page header/footer; unmarked does not', async (ctx: TestContext) => {
if (!reachable) ctx.skip();
const referenceDoc = {
name: 'reference-vs-nfd.odt',
bytes: readFileSync(join(process.cwd(), 'assets/reference-vs-nfd.odt')),
};
const marked = await converter.convert({
from: 'gfm',
to: 'odt',
input: Buffer.from('# Marked\n\nbody', 'utf8'),
standalone: true,
referenceDoc,
});
const styles = strFromU8(unzipSync(new Uint8Array(marked.output))['styles.xml']!);
expect(styles).toContain('<style:header>');
const occurrences = styles.split(MARKING).length - 1;
expect(occurrences).toBeGreaterThanOrEqual(2); // header + footer
const unmarked = await converter.convert({
from: 'gfm',
to: 'odt',
input: Buffer.from('# Open\n\nbody', 'utf8'),
standalone: true,
});
const openStyles = strFromU8(unzipSync(new Uint8Array(unmarked.output))['styles.xml']!);
expect(openStyles).not.toContain(MARKING);
});
});

View File

@ -31,8 +31,10 @@ const PNG_BASE64 =
class RecordingConverter extends PandocConverter {
lastInput = '';
lastReferenceDoc: string | null = null;
convert(request: ConversionRequest): Promise<ConversionResult> {
this.lastInput = request.input.toString('utf8');
this.lastReferenceDoc = request.referenceDoc?.name ?? null;
return Promise.resolve({ output: Buffer.from('OFFICE-BYTES'), mimeType: 'application/x-test' });
}
reachable(): Promise<boolean> {
@ -329,6 +331,30 @@ describe.skipIf(!hasTestDb)('export (e2e, issue #65)', () => {
.set('Cookie', ownerCookie)
.expect(200);
expect(result.text).toBe('OFFICE-BYTES');
// An unclassified page converts without a reference doc (issue #209).
expect(fake.lastReferenceDoc).toBeNull();
});
it('hands pandoc the classified reference doc for a marked page (issue #209)', async () => {
const slug = await seedPage(personalPondId, 'Classified Docx', '# Classified Docx\n\nbody');
const page = await prisma.page.findFirstOrThrow({
where: { pondId: personalPondId, slug },
});
await prisma.page.update({ where: { id: page.id }, data: { classification: 'VS_NFD' } });
const enqueued = await api()
.post(`/api/v1/pages/${page.id}/export`)
.set('Cookie', ownerCookie)
.send({ format: 'docx' })
.expect(201);
await worker.drain();
expect(fake.lastReferenceDoc).toBe('reference-vs-nfd.docx');
const done = await api()
.get(`/api/v1/jobs/${enqueued.body.id}`)
.set('Cookie', ownerCookie)
.expect(200);
expect(done.body.status).toBe('succeeded');
});
it('exports a page to PDF: content + image inlined, font CSS, via Gotenberg', async () => {

View File

@ -186,6 +186,10 @@ export class ExportService {
const dataUriById = await this.inlineImages(page.pondId, imageFileIds(markdown));
const document = markdownForDocument(markdown, dataUriById);
// A classified page's export records its marking as a job option (#209):
// the worker then hands pandoc the classified reference document whose
// header/footer carry the marking on every page in Word/LibreOffice.
const marking = classificationMarking(page.classification.toLowerCase() as PageClassification);
const job = await this.jobs.enqueue({
ownerId: user.id,
kind: `export_${format}`,
@ -193,6 +197,7 @@ export class ExportService {
to: format,
input: Buffer.from(document, 'utf8'),
standalone: true,
...(marking ? { options: { marking } } : {}),
});
this.logger.info(
{ jobId: job.id, pageId, format, userId: user.id },

View File

@ -17,6 +17,11 @@ export interface ConversionRequest {
/** Line-wrapping of the writer's output. Import uses `none` so a paragraph
* stays on one line (no soft breaks inside image alt text or links). */
wrap?: 'none' | 'auto' | 'preserve';
/** Reference document for the docx/odt writers (issue #209, ADR 0022):
* pandoc copies its page setup including the header/footer that carry
* the VS-NfD marking into the output. Sent to pandoc-server as an
* in-request file plus the `reference-doc` option. */
referenceDoc?: { name: string; bytes: Buffer };
}
export interface ConversionResult {
@ -141,6 +146,14 @@ export class PandocServerConverter extends PandocConverter {
// so these are only present when the import pipeline sets them.
...(request.embedResources ? { 'embed-resources': true } : {}),
...(request.wrap ? { wrap: request.wrap } : {}),
...(request.referenceDoc
? {
'reference-doc': request.referenceDoc.name,
files: {
[request.referenceDoc.name]: request.referenceDoc.bytes.toString('base64'),
},
}
: {}),
}),
signal: controller.signal,
});

View File

@ -55,7 +55,7 @@ _Meilenstein: `M26 — VS-NfD: classification metadata`_
- [x] Web-Ansicht (Kopf/Fuß) · 1 AT · #206
- [x] **Print-CSS** (`@media print`, Kopf/Fuß je Seite) — fehlt komplett · 1 AT · #207
- [x] PDF via gotenberg (`pdf-html.ts` Header/Footer-Template) · 1 AT · #208
- [ ] DOCX/ODT via pandoc (Reference-Doc mit Kopf-/Fußzeile) · 23 AT · #209
- [x] DOCX/ODT via pandoc (Reference-Doc mit Kopf-/Fußzeile) · 23 AT · #209
- [ ] Markdown-ZIP (Frontmatter + Aufdruck) · 1 AT · #210
- [ ] Atom-Feeds, Public-API, Suchergebnisse, No-JS-Shell · 23 AT · #211
- [ ] Attachment-Download (Dateiname-Präfix + Begleitdatei) · 12 AT · #212

View File

@ -24,7 +24,12 @@ export default tseslint.config(
prettier,
{
// Plain-Node maintenance/build scripts (no TypeScript, no bundler).
files: ['scripts/**/*.mjs', 'deploy/**/*.mjs', 'packages/plugins/*/build.mjs'],
files: [
'scripts/**/*.mjs',
'deploy/**/*.mjs',
'packages/plugins/*/build.mjs',
'apps/api/scripts/**/*.mjs',
],
languageOptions: {
globals: {
console: 'readonly',