#233: prune conversion job payloads for every job kind #254

fable-5 · 2026-07-30T22:12:45+02:00

fable-5 commented

2026-07-30 22:12:45 +02:00

Closes-when-merged: #233 (issue is closed manually after green CI/CD per process).

New daily scheduled job conversion-payload-prune: nulls input/result/result_mime_type of finished (SUCCEEDED/FAILED) conversion jobs older than conversion.payloadRetentionDays (new instance setting, default 30). Rows survive for status/audit.
PENDING/RUNNING rows are never touched, so the worker's stale-lock recovery keeps working (asserted by test).
conversion_jobs.input becomes nullable; migration backfills already-finished jobs past the default period.
conversionInputOf() guards the worker/import paths: a claimed job without input fails finally instead of crashing the worker.
Tests: conversion-payload-prune.e2e.db.test.ts (finished old pruned, fresh/pending/stale-RUNNING untouched, no-op run). Full api suite green (471) against a fresh DB, migrate deploy verified on a second fresh DB.
Docs: operations.md job table + paragraph; plan checkbox I-22. system.spec job-count fence 7 -> 8.

Closes-when-merged: #233 (issue is closed manually after green CI/CD per process). - New daily scheduled job `conversion-payload-prune`: nulls `input`/`result`/`result_mime_type` of finished (SUCCEEDED/FAILED) conversion jobs older than `conversion.payloadRetentionDays` (new instance setting, default 30). Rows survive for status/audit. - PENDING/RUNNING rows are never touched, so the worker's stale-lock recovery keeps working (asserted by test). - `conversion_jobs.input` becomes nullable; migration backfills already-finished jobs past the default period. - `conversionInputOf()` guards the worker/import paths: a claimed job without input fails finally instead of crashing the worker. - Tests: `conversion-payload-prune.e2e.db.test.ts` (finished old pruned, fresh/pending/stale-RUNNING untouched, no-op run). Full api suite green (471) against a fresh DB, `migrate deploy` verified on a second fresh DB. - Docs: operations.md job table + paragraph; plan checkbox I-22. system.spec job-count fence 7 -> 8.

fable-5 added 1 commit 2026-07-30 22:12:46 +02:00

#233 : prune conversion job payloads for every job kind

CI / Lint, typecheck, test (pull_request) Successful in 5m12s

Details

CI / Build container images (pull_request) Successful in 3m28s

Details

CI / Auth e2e pack (pull_request) Successful in 8m33s

Details

CI / Import/export fidelity gate (pull_request) Successful in 1m2s

Details

CD / Build and push images (push) Successful in 29s

Details

CD / Deploy to Test (push) Successful in 12s

Details

CD / Smoke tests against Test (push) Successful in 1m22s

Details

CD / Promote to Int (push) Successful in 11s

Details

CI / Lint, typecheck, test (push) Failing after 5m9s

Details

CI / Auth e2e pack (push) Has been skipped

Details

CI / Import/export fidelity gate (push) Has been skipped

Details

CI / Build container images (push) Has been skipped

Details

ff505bc752

The raw input/result bytes of import/export conversion jobs were kept
forever; a deleted classified page could live on inside its last export.
A new daily conversion-payload-prune job nulls both once a finished
(succeeded or failed) job passes conversion.payloadRetentionDays
(instance setting, default 30) — the row survives for status/audit.
PENDING and RUNNING rows keep their payload, so the worker's stale-lock
recovery path is untouched; a hand-requeued pruned job fails finally
via conversionInputOf instead of crashing the worker.

The input column becomes nullable; the migration backfills by clearing
payloads of jobs already finished longer ago than the default period
(recent results stay downloadable until they age out).

Job-count fence in system.spec: 7 -> 8 (new scheduler registration).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ

fable-5 referenced this pull request

2026-07-30 22:17:45 +02:00

#234: retention for mail_outbox #255

fable-5 referenced this pull request

2026-07-30 22:18:49 +02:00

#235: keep page_links rows pointing at purged pages (recorded decision) #256

fable-5 merged commit ff505bc752 into main

2026-07-30 22:34:32 +02:00

fable-5 referenced this pull request

2026-07-30 22:35:25 +02:00

[VS-NfD] Prune conversion job payloads for every job kind #233

Sign in to join this conversation.