Adds docs/operations/update-runbook.md (obtain, verify by digest, back up, apply, verify, roll back) with the migration behaviour stated explicitly: a failed migration rolls back its own transaction but is recorded in _prisma_migrations and blocks every further migrate deploy (P3009) — including a re-deployed old image — until migrate resolve --rolled-back; semantically irreversible migrations have exactly one way back, the pre-update backup set. No rolling updates on a compose stage. Rehearsed in the isolated environment of #220: regular update to a v2 image set, then a deliberate failed-update (P3018 division by zero, schema change proven rolled back) with image-rollback-alone shown insufficient and the documented recovery executed. Protocol: docs/vs-nfd/98-update-rollback-protokoll.md. ADR 0024 decisions 5+6 recorded as executed; operations handbook and restore runbook updated; plan checkbox P1-3 ticked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AUtYMxwTCMHG9mVHnwbFg8
5.4 KiB
ADR 0024: Reproducible offline deployment
- Status: proposed
- Date: 2026-07-29
Context
A VS zone has no internet egress. "Should work offline" is the answer that loses a first meeting; "tested, here is the procedure" is the one that wins it — which is why the plan pulled this out of the roadmap into Phase 1.
The current state is favourable but unverified. There is no telemetry, no
update check, no CDN; fonts are self-hosted (ADR 0016); CSP is
default-src 'self'; search is Postgres rather than an external engine;
the drawio plugin is vendored rather than loaded from a remote editor. What
is missing is evidence, plus two real gaps: images are referenced by tag
(including the floating gotenberg/gotenberg:8), and there is no
documented mirror or update path.
Decision
- All third-party images are pinned by digest (
name:tag@sha256:…). The tag stays for human readability; the digest decides what runs. A CI check rejects any un-digested third-party reference. - The image list is generated, not hand-maintained, so a mirror procedure cannot silently miss a service.
- An internal registry is the supported source. Compose takes the registry prefix from configuration; no site edits image references.
- Reproducibility without network is a stated choice between two
paths: an offline pnpm store enabling
install+buildwith networking disabled, or prebuilt images only with no customer-side build. Either is acceptable; leaving it unstated is not, because it determines whether the customer can patch locally. - The airgap claim is proven by a documented run in a network-isolated environment, covering every function including the export sidecars, and listing every outbound connection attempt observed. This run is the artefact, and it also answers the plan's open question about what breaks offline.
- The offline update path is part of the decision, not an afterthought: bundle, verify by digest, back up, apply, verify, roll back — with the irreversibility of migrations stated explicitly.
Decisions taken in #219
- Both paths, with stated roles: customers OPERATE prebuilt, digest-pinned images (the CI release build; mirrored per #218) — no customer-side build for operation. ADDITIONALLY the workspace build is verified to work with networking disabled, so site-local patching remains possible (open source as an audit advantage) — the "no local patching" consequence therefore does not apply.
- The offline kit is the pnpm store plus the build user's
~/.cache(the prisma postinstall pulls its engines from there instead of the network) — both filled by one onlinepnpm install --frozen-lockfile. - Evidence:
pnpm install --offline --frozen-lockfileandpnpm buildunderdocker run --network none(node:22.15.1-alpine, pnpm 11.9.0 — the pinned toolchain), reproduced twice from clean checkouts (docs/vs-nfd/96-offline-build-protokoll.md). - The one network dependency found: the drawio plugin's installable
ZIP fetches its pinned vendor tarball on first build. Deploy images
contain no plugin ZIPs, so the delivery-relevant build is fully
offline (
CI=1skips the fetch, as in CI); for an offline ZIP build the tarball is pre-seeded intopackages/plugins/drawio/vendor/.
Decisions taken in #220 and #221
- The airgap claim is proven (decision 5 executed): a full deployment
ran in an environment with no egress path at all; the complete capture
shows the application makes exactly one outbound attempt — SMTP — which
fails contained in the outbox while the instance stays functional.
Evidence:
docs/vs-nfd/97-isolationslauf-protokoll.md. The run surfaced and fixed one real defect (#288, restore on partitioned tables). - The update path is defined and rehearsed (decision 6 executed):
procedure in
docs/operations/update-runbook.md; rehearsed in the same isolated environment including one deliberate failed-update rollback. Migration irreversibility is stated explicitly: failed migrations roll back their own transaction but block every furthermigrate deploy(P3009) untilmigrate resolve --rolled-back; successfully applied, semantically irreversible migrations have exactly one way back — the pre-update backup set. No rolling updates on a compose stage; updates happen in a short maintenance window. Evidence:docs/vs-nfd/98-update-rollback-protokoll.md.
Consequences
- Digest pinning creates recurring maintenance: security updates now require an explicit, reviewable change. That visibility is the point.
- Digest pinning must precede the mirror and update work, so it sits in the
hardening & supply chainmilestone rather than this one. - The isolated test run will surface findings; each becomes its own issue referenced from #220 rather than expanding that issue's scope.
- Outbound SMTP is the one connection an authority may or may not permit;
the deployment must be functional without it, and the consequences of
disabling it (no notifications, no verification mail — which interacts
with
auth.local.enabled = false) are documented. - CD does not sync stage composes, so digest changes need an explicit rollout step on the stage hosts.
Implementing issues
#203 (digest pinning), #218 (registry mirror), #219 (network-free build), #220 (isolated test run), #221 (offline update path), #236 (pinned Node version — added from the Ist-Aufnahme, I-26).