#196: audit-trail retention job
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m5s
CI / Build container images (pull_request) Successful in 2m48s
CI / Auth e2e pack (pull_request) Successful in 7m50s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 15s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m20s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m11s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m38s
CI / Import/export fidelity gate (push) Successful in 56s
All checks were successful
CI / Lint, typecheck, test (pull_request) Successful in 5m5s
CI / Build container images (pull_request) Successful in 2m48s
CI / Auth e2e pack (pull_request) Successful in 7m50s
CI / Import/export fidelity gate (pull_request) Successful in 56s
CD / Build and push images (push) Successful in 15s
CD / Deploy to Test (push) Successful in 16s
CD / Smoke tests against Test (push) Successful in 1m20s
CD / Promote to Int (push) Successful in 11s
CI / Lint, typecheck, test (push) Successful in 5m11s
CI / Build container images (push) Has been skipped
CI / Auth e2e pack (push) Successful in 7m38s
CI / Import/export fidelity gate (push) Successful in 56s
audit.retentionDays (instance setting, default 365) bounds the audit_log: the daily audit-retention job deletes entries past the period and records the deletion itself (audit.pruned with count, cutoff and period) so a gap in the trail is always explainable. Lives in its own AuditRetentionService because the settings service audits its writes - folding retention into AuditService would close a constructor cycle. The read-access trail (#222-#225) is deliberately not covered; it gets its own period. security.md gains the Logging section the schema has cited for a while; the maintenance-job fence moves 6 -> 7 (the deliberate new row). Refs #196 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168Ph5uBmHm8X28CSVpbpnJ
This commit is contained in:
parent
960a806ee3
commit
ed2225bb77
90
apps/api/src/audit/audit-retention.e2e.db.test.ts
Normal file
90
apps/api/src/audit/audit-retention.e2e.db.test.ts
Normal file
@ -0,0 +1,90 @@
|
|||||||
|
import { INestApplication } from '@nestjs/common';
|
||||||
|
import { PrismaClient } from '@prisma/client';
|
||||||
|
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
|
||||||
|
|
||||||
|
import { createTestApp } from '../testing/test-app';
|
||||||
|
import { createTestPrisma, hasTestDb, uniqueSuffix } from '../testing/test-db';
|
||||||
|
import { AuditRetentionService } from './audit-retention.service';
|
||||||
|
|
||||||
|
const DAY = 24 * 60 * 60 * 1000;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Audit-trail retention (issue #196): entries past `audit.retentionDays`
|
||||||
|
* are pruned, newer ones stay, and the pruning itself lands in the trail
|
||||||
|
* (`audit.pruned` with count and cutoff) so the gap is explainable.
|
||||||
|
*/
|
||||||
|
describe.skipIf(!hasTestDb)('audit retention (e2e, issue #196)', () => {
|
||||||
|
let app: INestApplication;
|
||||||
|
let prisma: PrismaClient;
|
||||||
|
const suffix = uniqueSuffix();
|
||||||
|
const marker = `retention-${suffix}`;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
prisma = createTestPrisma();
|
||||||
|
// A short period so ages are unambiguous; written straight to the row
|
||||||
|
// BEFORE the app boots (the settings cache is in-process and fills on
|
||||||
|
// first read). The key is cleaned afterAll.
|
||||||
|
await prisma.instanceSetting.upsert({
|
||||||
|
where: { key: 'audit.retentionDays' },
|
||||||
|
create: { key: 'audit.retentionDays', value: 30 },
|
||||||
|
update: { value: 30 },
|
||||||
|
});
|
||||||
|
app = await createTestApp();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterAll(async () => {
|
||||||
|
await prisma.instanceSetting.deleteMany({ where: { key: 'audit.retentionDays' } });
|
||||||
|
await prisma.auditEntry.deleteMany({
|
||||||
|
where: { OR: [{ targetId: { contains: suffix } }, { action: 'audit.pruned' }] },
|
||||||
|
});
|
||||||
|
await prisma.$disconnect();
|
||||||
|
await app.close();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('prunes entries past the period, keeps newer ones, and records the pruning', async () => {
|
||||||
|
await prisma.auditEntry.createMany({
|
||||||
|
data: [
|
||||||
|
{
|
||||||
|
action: 'test.old',
|
||||||
|
targetType: 'test',
|
||||||
|
targetId: marker,
|
||||||
|
at: new Date(Date.now() - 40 * DAY),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
action: 'test.older',
|
||||||
|
targetType: 'test',
|
||||||
|
targetId: marker,
|
||||||
|
at: new Date(Date.now() - 400 * DAY),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
action: 'test.fresh',
|
||||||
|
targetType: 'test',
|
||||||
|
targetId: marker,
|
||||||
|
at: new Date(Date.now() - 5 * DAY),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
});
|
||||||
|
|
||||||
|
const pruned = await app.get(AuditRetentionService).pruneExpired();
|
||||||
|
expect(pruned).toBeGreaterThanOrEqual(2);
|
||||||
|
|
||||||
|
const remaining = await prisma.auditEntry.findMany({ where: { targetId: marker } });
|
||||||
|
expect(remaining.map((entry) => entry.action)).toEqual(['test.fresh']);
|
||||||
|
|
||||||
|
// The gap is explainable: the pruning run is itself on the trail.
|
||||||
|
const prunedEvent = await prisma.auditEntry.findFirst({
|
||||||
|
where: { action: 'audit.pruned' },
|
||||||
|
orderBy: { at: 'desc' },
|
||||||
|
});
|
||||||
|
expect(prunedEvent).not.toBeNull();
|
||||||
|
expect(prunedEvent!.details).toMatchObject({ retentionDays: 30 });
|
||||||
|
expect((prunedEvent!.details as { count: number }).count).toBeGreaterThanOrEqual(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('is a no-op when nothing is due', async () => {
|
||||||
|
const pruned = await app.get(AuditRetentionService).pruneExpired();
|
||||||
|
expect(pruned).toBe(0);
|
||||||
|
// The fresh marker entry from the first test is untouched.
|
||||||
|
expect(await prisma.auditEntry.count({ where: { targetId: marker } })).toBe(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
47
apps/api/src/audit/audit-retention.service.ts
Normal file
47
apps/api/src/audit/audit-retention.service.ts
Normal file
@ -0,0 +1,47 @@
|
|||||||
|
import { Injectable } from '@nestjs/common';
|
||||||
|
import { PinoLogger } from 'nestjs-pino';
|
||||||
|
|
||||||
|
import { ClockService } from '../common/clock.service';
|
||||||
|
import { PrismaService } from '../prisma/prisma.service';
|
||||||
|
import { InstanceSettingsService } from '../settings/instance-settings.service';
|
||||||
|
|
||||||
|
import { AuditService } from './audit.service';
|
||||||
|
|
||||||
|
const MS_PER_DAY = 24 * 60 * 60 * 1000;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Audit-trail retention (issue #196): the daily job deletes `audit_log`
|
||||||
|
* entries older than the configurable `audit.retentionDays` (default one
|
||||||
|
* year) and records the deletion itself (`audit.pruned` with count and
|
||||||
|
* cutoff) so a gap in the trail is always explainable. Separate from
|
||||||
|
* {@link AuditService} because the settings service audits its own writes
|
||||||
|
* — folding retention into AuditService would close a constructor cycle.
|
||||||
|
* The read-access trail (#224) is deliberately not covered here.
|
||||||
|
*/
|
||||||
|
@Injectable()
|
||||||
|
export class AuditRetentionService {
|
||||||
|
constructor(
|
||||||
|
private readonly prisma: PrismaService,
|
||||||
|
private readonly settings: InstanceSettingsService,
|
||||||
|
private readonly audit: AuditService,
|
||||||
|
private readonly clock: ClockService,
|
||||||
|
private readonly logger: PinoLogger,
|
||||||
|
) {
|
||||||
|
this.logger.setContext(AuditRetentionService.name);
|
||||||
|
}
|
||||||
|
|
||||||
|
async pruneExpired(): Promise<number> {
|
||||||
|
const retentionDays = await this.settings.get('audit.retentionDays');
|
||||||
|
const cutoff = new Date(this.clock.now().getTime() - retentionDays * MS_PER_DAY);
|
||||||
|
const { count } = await this.prisma.auditEntry.deleteMany({
|
||||||
|
where: { at: { lt: cutoff } },
|
||||||
|
});
|
||||||
|
if (count > 0) {
|
||||||
|
await this.audit.record({
|
||||||
|
action: 'audit.pruned',
|
||||||
|
details: { count, cutoff: cutoff.toISOString(), retentionDays },
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -1,7 +1,16 @@
|
|||||||
import { Global, Module } from '@nestjs/common';
|
import { Global, Module, OnModuleInit } from '@nestjs/common';
|
||||||
|
|
||||||
|
import { CommonModule } from '../common/common.module';
|
||||||
|
import { SchedulerModule } from '../scheduler/scheduler.module';
|
||||||
|
import { SchedulerService } from '../scheduler/scheduler.service';
|
||||||
|
import { SettingsModule } from '../settings/settings.module';
|
||||||
|
|
||||||
|
import { AuditRetentionService } from './audit-retention.service';
|
||||||
import { AuditService } from './audit.service';
|
import { AuditService } from './audit.service';
|
||||||
|
|
||||||
|
/** Daily, per operations.md's maintenance-jobs table (issue #196). */
|
||||||
|
const AUDIT_RETENTION_CADENCE_SECONDS = 24 * 60 * 60;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Global because the audit trail cuts across nearly every feature module
|
* Global because the audit trail cuts across nearly every feature module
|
||||||
* (auth, grants, members, admin, plugins, setup) — like PrismaModule, one
|
* (auth, grants, members, admin, plugins, setup) — like PrismaModule, one
|
||||||
@ -9,7 +18,23 @@ import { AuditService } from './audit.service';
|
|||||||
*/
|
*/
|
||||||
@Global()
|
@Global()
|
||||||
@Module({
|
@Module({
|
||||||
providers: [AuditService],
|
imports: [CommonModule, SchedulerModule, SettingsModule],
|
||||||
|
providers: [AuditService, AuditRetentionService],
|
||||||
exports: [AuditService],
|
exports: [AuditService],
|
||||||
})
|
})
|
||||||
export class AuditModule {}
|
export class AuditModule implements OnModuleInit {
|
||||||
|
constructor(
|
||||||
|
private readonly scheduler: SchedulerService,
|
||||||
|
private readonly retention: AuditRetentionService,
|
||||||
|
) {}
|
||||||
|
|
||||||
|
onModuleInit(): void {
|
||||||
|
this.scheduler.register({
|
||||||
|
name: 'audit-retention',
|
||||||
|
cadenceSeconds: AUDIT_RETENTION_CADENCE_SECONDS,
|
||||||
|
run: async () => {
|
||||||
|
await this.retention.pruneExpired();
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@ -35,6 +35,11 @@ export const INSTANCE_SETTINGS = {
|
|||||||
// Trash retention (ADR 0013, issue #31): days a soft-deleted page stays
|
// Trash retention (ADR 0013, issue #31): days a soft-deleted page stays
|
||||||
// restorable before the daily purge job removes it for good.
|
// restorable before the daily purge job removes it for good.
|
||||||
'trash.retentionDays': z.number().int().min(1).default(30),
|
'trash.retentionDays': z.number().int().min(1).default(30),
|
||||||
|
// Audit-trail retention (issue #196): days an `audit_log` entry is kept
|
||||||
|
// before the daily retention job removes it; the deletion itself is
|
||||||
|
// recorded (`audit.pruned`) so the gap is explainable. The read-access
|
||||||
|
// trail (#224) is deliberately NOT covered — it gets its own period.
|
||||||
|
'audit.retentionDays': z.number().int().min(1).default(365),
|
||||||
// Non-image upload allowlist (ADR 0011, issue #61): lowercase extensions
|
// Non-image upload allowlist (ADR 0011, issue #61): lowercase extensions
|
||||||
// without the dot. Images are always allowed regardless; SVG is governed
|
// without the dot. Images are always allowed regardless; SVG is governed
|
||||||
// by `upload.svgPolicy`. Normalized (lowercased, dot-stripped, deduped) so
|
// by `upload.svgPolicy`. Normalized (lowercased, dot-stripped, deduped) so
|
||||||
|
|||||||
@ -17,10 +17,10 @@ test('lists maintenance jobs and triggers one manually', async ({ browser }) =>
|
|||||||
const jobsTable = page.locator('.system-jobs__table');
|
const jobsTable = page.locator('.system-jobs__table');
|
||||||
await expect(jobsTable).toBeVisible();
|
await expect(jobsTable).toBeVisible();
|
||||||
// All registered jobs appear (language-neutral: row count + button).
|
// All registered jobs appear (language-neutral: row count + button).
|
||||||
// 5 → 6 with issue #194: the orphan-file sweep joined the job table.
|
|
||||||
// Keep in sync with the scheduler registrations: trash-purge,
|
// Keep in sync with the scheduler registrations: trash-purge,
|
||||||
// version-thinning, page-compaction, data-export-purge, notification-digest.
|
// version-thinning, page-compaction, data-export-purge,
|
||||||
await expect(jobsTable.locator('tbody tr')).toHaveCount(6);
|
// notification-digest, orphan-file-sweep (#194), audit-retention (#196).
|
||||||
|
await expect(jobsTable.locator('tbody tr')).toHaveCount(7);
|
||||||
|
|
||||||
const firstRow = jobsTable.locator('tbody tr').first();
|
const firstRow = jobsTable.locator('tbody tr').first();
|
||||||
await firstRow.getByRole('button').click();
|
await firstRow.getByRole('button').click();
|
||||||
|
|||||||
@ -99,12 +99,13 @@ monitoring, structured logs, backup alerting — no dedicated metrics stack.
|
|||||||
## Maintenance jobs (in-app scheduler, `jobs` table)
|
## Maintenance jobs (in-app scheduler, `jobs` table)
|
||||||
|
|
||||||
| Job | Cadence | Purpose |
|
| Job | Cadence | Purpose |
|
||||||
| --------------------- | ----------------------- | -------------------------------------------------- |
|
| --------------------- | ----------------------- | --------------------------------------------------- |
|
||||||
| trash purge | daily | delete pages/ponds past trash retention (ADR 0013) |
|
| trash purge | daily | delete pages/ponds past trash retention (ADR 0013) |
|
||||||
| version thinning | daily | auto-version retention policy (ADR 0013) |
|
| version thinning | daily | auto-version retention policy (ADR 0013) |
|
||||||
| update-log compaction | hourly, idle pages only | bound Yjs log growth |
|
| update-log compaction | hourly, idle pages only | bound Yjs log growth |
|
||||||
| quota reconciliation | nightly | recompute `pond_usage`, report drift |
|
| quota reconciliation | nightly | recompute `pond_usage`, report drift |
|
||||||
| orphan file sweep | nightly | volume ↔ DB consistency (ADR 0011) |
|
| orphan file sweep | nightly | volume ↔ DB consistency (ADR 0011) |
|
||||||
|
| audit retention | daily | prune `audit_log` past `audit.retentionDays` (#196) |
|
||||||
| mail outbox retry | every minute | e-mail delivery with backoff |
|
| mail outbox retry | every minute | e-mail delivery with backoff |
|
||||||
|
|
||||||
Job outcomes are visible in the Site Admin UI (last run, status) — that
|
Job outcomes are visible in the Site Admin UI (last run, status) — that
|
||||||
|
|||||||
@ -119,6 +119,23 @@ or sloppy plugin authors, compromised dependencies.
|
|||||||
- Dependencies: lockfile-pinned; monthly update batch; images pinned to
|
- Dependencies: lockfile-pinned; monthly update batch; images pinned to
|
||||||
digests in Prod.
|
digests in Prod.
|
||||||
|
|
||||||
|
## Logging
|
||||||
|
|
||||||
|
- Application logs are pino JSON on stdout; `authorization` and `cookie`
|
||||||
|
headers are redacted, request bodies are never logged, and feed-token
|
||||||
|
query values are masked (issue #191). Log forwarding and retention are
|
||||||
|
the container runtime's job (SIEM division of labour — the application
|
||||||
|
side of that contract is the stable event catalogue, issue #201).
|
||||||
|
- The persistent audit trail (`audit_log`, issue #86) records auth and
|
||||||
|
admin events — who changed access or configuration, not who edited
|
||||||
|
what; content activity stays log-only by design.
|
||||||
|
- Audit retention (issue #196): entries are kept for
|
||||||
|
`audit.retentionDays` (instance setting, default 365) and pruned by the
|
||||||
|
daily `audit-retention` job; each pruning run is itself recorded as
|
||||||
|
`audit.pruned` with count and cutoff, so a gap in the trail is always
|
||||||
|
explainable. The read-access trail (#222–#225) is deliberately not
|
||||||
|
covered by this period — it gets its own.
|
||||||
|
|
||||||
## Privacy (GDPR)
|
## Privacy (GDPR)
|
||||||
|
|
||||||
- No external requests from the browser (fonts self-hosted, no CDNs, no
|
- No external requests from the browser (fonts self-hosted, no CDNs, no
|
||||||
|
|||||||
@ -103,7 +103,7 @@ chain`_
|
|||||||
- [x] **Orphan-File-Sweep** implementieren, `Attachment.deletedAt` nutzen
|
- [x] **Orphan-File-Sweep** implementieren, `Attachment.deletedAt` nutzen
|
||||||
oder entfernen · 2 AT · #194
|
oder entfernen · 2 AT · #194
|
||||||
- [x] **Papierkorb aus dem Suchindex** entfernen statt query-seitig filtern · 2 AT · #195
|
- [x] **Papierkorb aus dem Suchindex** entfernen statt query-seitig filtern · 2 AT · #195
|
||||||
- [ ] **Retention-Job für `audit_log`** · 1 AT · #196
|
- [x] **Retention-Job für `audit_log`** · 1 AT · #196
|
||||||
- [ ] **Security-Header** (helmet), CORS explizit restriktiv · 1 AT · #197
|
- [ ] **Security-Header** (helmet), CORS explizit restriktiv · 1 AT · #197
|
||||||
- [ ] **SBOM in CI** (CycloneDX/syft) + Lizenzreport als Artefakt · 1–2 AT · #202
|
- [ ] **SBOM in CI** (CycloneDX/syft) + Lizenzreport als Artefakt · 1–2 AT · #202
|
||||||
- [ ] `deploy/compose/.env` prüfen, Beispieldatei statt Realdatei · 0,5 AT · #198
|
- [ ] `deploy/compose/.env` prüfen, Beispieldatei statt Realdatei · 0,5 AT · #198
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user