Nextcloud backup target: admin-configured, manual + scheduled runs, in-app restore #103

New Issue

fable-5 · 2026-07-12T09:36:43+02:00

fable-5 commented

2026-07-12 09:36:43 +02:00

Context

Backups today are local-only (nightly sidecar sets on the backups volume, #83) plus a planned host-specific rsync mirror to a private server (#84) that is blocked on network infrastructure and useless to other self-hosters. Dorfteich should instead offer a built-in, admin-configurable off-host backup target that any self-hoster has access to: a Nextcloud.

Scope

Connection settings in the site-admin UI: WebDAV base URL, username, app password, target folder. The app password goes into the existing secret store (SmtpConfigService pattern from #80), never into the DB. A "test connection" button verifies credentials + folder before saving (like the setup wizard's SMTP test).
Upload: every completed local backup set (db dump + files archive, #83) is compressed into ONE self-contained archive per set (single .tar.gz containing db-<id>.dump, files-<id>.tar.gz, and a manifest) and uploaded via WebDAV — everything needed to rebuild an instance after total loss.
Triggers: a manual button in the admin system panel ("back up now", runs dump + upload) and a configurable interval (off / daily / weekly / custom hour, admin setting). The nightly local run keeps its existing behaviour; the upload happens after each successful set.
Local rotation setting: the local retention (today env BACKUP_RETENTION_DAYS) becomes an admin setting (number of sets/days to keep locally); a separate retention setting governs the Nextcloud side (prune old remote sets, never the newest complete one — same guarantee as #83).
Restore: the admin UI lists remote backup sets (name, timestamp, size). Restoring one requires a type-to-confirm safety prompt, then the instance orchestrates itself: maintenance mode (503 for everything but a status page), download + verify the set, restore db + volumes via the existing sidecar restore path, restart, readyz-gated recovery. The documented operator path (deploy/backup/restore.sh with a manually downloaded set) stays as the disaster-recovery fallback for when the app itself is gone — document fetching a set from Nextcloud in the restore runbook.
Status surface: upload outcome + last successful off-host backup become part of status.json (contract in packages/shared/src/backup-status.ts), shown on the #86 admin backup card and in the #85 readyz freshness check (warn when the remote copy is stale while a target is configured).

Acceptance criteria

a fresh instance can configure a Nextcloud target entirely in the admin UI (no env edits), test the connection, and trigger a manual backup that appears in the Nextcloud folder as one compressed archive
scheduled uploads run at the configured interval; local and remote retention are admin-configurable and never delete the newest complete set on either side
a restore from the admin UI (with safety confirmation) brings back content, files and settings; a wiped instance can be rebuilt from the Nextcloud archive alone using the documented operator path
readyz/admin card reflect off-host staleness; failures alert via the existing backup failure mail
credentials live in the secret store, never in instance_settings, exports or logs

Technical notes

The api and the backup sidecar share no channel today; the established primitive is pg NOTIFY/LISTEN (PAGE_RESTORE_CHANNEL pattern, listener example in apps/collab/src/restore-listener.ts) — the sidecar gains a listener for "run backup now" / "restore set X" commands, the api publishes them and tracks progress via status.json (extend the shared contract).
WebDAV needs no heavy dependency (plain HTTP PUT/GET/PROPFIND with basic auth); chunked upload for very large sets is a stretch goal.
Maintenance mode: SetupGuard-style global 503 gate (#80) flipped by the restore orchestrator.
New admin settings go into the INSTANCE_SETTINGS registry (apps/api/src/settings/instance-settings.service.ts) — non-secret keys only.

Dependencies

Builds on #83 (sidecar), #85 (readyz), #86 (admin panel). Supersedes the generic need behind #84, which stays open as a personal, tunnel-blocked extra.

Size: ~3 days

Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add de and en), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.

## Context Backups today are local-only (nightly sidecar sets on the backups volume, #83) plus a planned host-specific rsync mirror to a private server (#84) that is blocked on network infrastructure and useless to other self-hosters. Dorfteich should instead offer a built-in, admin-configurable off-host backup target that any self-hoster has access to: a Nextcloud. ## Scope - **Connection settings** in the site-admin UI: WebDAV base URL, username, app password, target folder. The app password goes into the existing secret store (`SmtpConfigService` pattern from #80), never into the DB. A "test connection" button verifies credentials + folder before saving (like the setup wizard's SMTP test). - **Upload**: every completed local backup set (db dump + files archive, #83) is compressed into ONE self-contained archive per set (single `.tar.gz` containing `db-<id>.dump`, `files-<id>.tar.gz`, and a manifest) and uploaded via WebDAV — everything needed to rebuild an instance after total loss. - **Triggers**: a manual button in the admin system panel ("back up now", runs dump + upload) and a configurable interval (off / daily / weekly / custom hour, admin setting). The nightly local run keeps its existing behaviour; the upload happens after each successful set. - **Local rotation setting**: the local retention (today env `BACKUP_RETENTION_DAYS`) becomes an admin setting (number of sets/days to keep locally); a separate retention setting governs the Nextcloud side (prune old remote sets, never the newest complete one — same guarantee as #83). - **Restore**: the admin UI lists remote backup sets (name, timestamp, size). Restoring one requires a type-to-confirm safety prompt, then the instance orchestrates itself: maintenance mode (503 for everything but a status page), download + verify the set, restore db + volumes via the existing sidecar restore path, restart, readyz-gated recovery. The documented operator path (`deploy/backup/restore.sh` with a manually downloaded set) stays as the disaster-recovery fallback for when the app itself is gone — document fetching a set from Nextcloud in the restore runbook. - **Status surface**: upload outcome + last successful off-host backup become part of `status.json` (contract in `packages/shared/src/backup-status.ts`), shown on the #86 admin backup card and in the #85 readyz freshness check (warn when the remote copy is stale while a target is configured). ## Acceptance criteria - [ ] a fresh instance can configure a Nextcloud target entirely in the admin UI (no env edits), test the connection, and trigger a manual backup that appears in the Nextcloud folder as one compressed archive - [ ] scheduled uploads run at the configured interval; local and remote retention are admin-configurable and never delete the newest complete set on either side - [ ] a restore from the admin UI (with safety confirmation) brings back content, files and settings; a wiped instance can be rebuilt from the Nextcloud archive alone using the documented operator path - [ ] readyz/admin card reflect off-host staleness; failures alert via the existing backup failure mail - [ ] credentials live in the secret store, never in `instance_settings`, exports or logs ## Technical notes - The api and the backup sidecar share no channel today; the established primitive is pg NOTIFY/LISTEN (`PAGE_RESTORE_CHANNEL` pattern, listener example in `apps/collab/src/restore-listener.ts`) — the sidecar gains a listener for "run backup now" / "restore set X" commands, the api publishes them and tracks progress via `status.json` (extend the shared contract). - WebDAV needs no heavy dependency (plain HTTP PUT/GET/PROPFIND with basic auth); chunked upload for very large sets is a stretch goal. - Maintenance mode: SetupGuard-style global 503 gate (#80) flipped by the restore orchestrator. - New admin settings go into the `INSTANCE_SETTINGS` registry (`apps/api/src/settings/instance-settings.service.ts`) — non-secret keys only. ## Dependencies Builds on #83 (sidecar), #85 (readyz), #86 (admin panel). Supersedes the generic need behind #84, which stays open as a personal, tunnel-blocked extra. **Size**: ~3 days --- *Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add `de` **and** `en`), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.*

fable-5 added this to the M8 — Self-hosting & operations milestone 2026-07-12 09:36:43 +02:00

fable-5 added the

backend

deployment

frontend

labels 2026-07-12 09:36:43 +02:00

fable-5 referenced this issue

2026-07-12 09:37:01 +02:00

Off-host backup mirror to BASEL #84

fable-5 referenced this issue from a commit

2026-07-12 10:39:29 +02:00

Nextcloud backup target: admin-configured, manual + scheduled uploads, in-app restore (#103)

fable-5 commented

2026-07-12 10:51:41 +02:00

Implemented in 5cef359 (CI fully green, deployed to Test + Int).

What shipped

Admin-configured Nextcloud target (Admin → System → Backups): WebDAV base URL, username, app password (secret store, never the DB), folder, test connection button (verifies credentials and creates missing folder segments via MKCOL). Shared minimal WebDAV client (@dorfteich/shared/webdav, PROPFIND/MKCOL/PUT/GET/DELETE, no new dependency).
Uploads: after each successful local set the sidecar packs ONE self-contained bundle dorfteich-backup-<id>.tar.gz (db dump + files archive + manifest.json) and uploads it; schedule off/daily/weekly (manual runs always upload); remote prune honours its own retention and never deletes the newest bundle.
Triggers: "Back up now" button → backup_command pg NOTIFY → sidecar (same LISTEN/NOTIFY primitive as the collab bus); nightly local run unchanged.
Retention settings: backup.localRetentionDays (admin override; env stays authoritative until first save) and backup.remoteRetentionDays in the instance-settings registry.
In-app restore: picker lists local (ro backups mount) and Nextcloud sets; type-to-confirm; sidecar orchestrates — restore-status.json → maintenance mode (api answers 503 maintenance_mode everywhere except health + public GET /backup/restore-status) → collab closes/refuses sessions via backup_maintenance NOTIFY → terminate connections → same restore path as restore.sh → api restarts itself for a clean boot. SPA shows a maintenance status page for everyone and reloads when the instance is back. restore.sh + runbook stay the disaster path; the runbook documents fetching a bundle from Nextcloud by hand.
Status surfaces: status.json gained a remote block (contract in shared backup-status.ts); admin card shows upload state; readyz gained a warning-level backup_remote check (26 h daily / 170 h weekly); upload failures alert via a new backupUploadFailed mail (de+en).

Hardening found by the live verify (fresh stack + fake WebDAV server, full configure→backup→upload→remote-restore cycle): the restore's connection termination crashed the api through an unhandled worker rejection — conversion/mail/scheduler sweeps now catch transient DB failures; and main.ts waits while restore-status.json says running so a container restarting mid-restore can never race migrate deploy against pg_restore.

Notes / deviations

The upload schedule interprets "custom hour" as: uploads ride on the nightly run (BACKUP_TIME); a separate upload-only clock did not seem worth the extra setting.
Restore picker also offers local sets (free, useful without Nextcloud); the issue only required remote.
e2e: 8-test api pack (backup-admin.e2e.db.test.ts) incl. maintenance-gate semantics + staleness; sidecar tests run against an in-process WebDAV server (streamed PUT, prune, bundle roundtrip); collab listener unit-tested.

Acceptance criteria all met; credentials verified absent from instance_settings, exports and logs.

Implemented in `5cef359` (CI fully green, deployed to Test + Int). **What shipped** - **Admin-configured Nextcloud target** (Admin → System → Backups): WebDAV base URL, username, app password (secret store, never the DB), folder, **test connection** button (verifies credentials and creates missing folder segments via MKCOL). Shared minimal WebDAV client (`@dorfteich/shared/webdav`, PROPFIND/MKCOL/PUT/GET/DELETE, no new dependency). - **Uploads**: after each successful local set the sidecar packs ONE self-contained bundle `dorfteich-backup-<id>.tar.gz` (db dump + files archive + manifest.json) and uploads it; schedule off/daily/weekly (manual runs always upload); remote prune honours its own retention and never deletes the newest bundle. - **Triggers**: "Back up now" button → `backup_command` pg NOTIFY → sidecar (same LISTEN/NOTIFY primitive as the collab bus); nightly local run unchanged. - **Retention settings**: `backup.localRetentionDays` (admin override; env stays authoritative until first save) and `backup.remoteRetentionDays` in the instance-settings registry. - **In-app restore**: picker lists local (ro backups mount) and Nextcloud sets; type-to-confirm; sidecar orchestrates — `restore-status.json` → maintenance mode (api answers 503 `maintenance_mode` everywhere except health + public `GET /backup/restore-status`) → collab closes/refuses sessions via `backup_maintenance` NOTIFY → terminate connections → same restore path as `restore.sh` → api restarts itself for a clean boot. SPA shows a maintenance status page for everyone and reloads when the instance is back. `restore.sh` + runbook stay the disaster path; the runbook documents fetching a bundle from Nextcloud by hand. - **Status surfaces**: `status.json` gained a `remote` block (contract in shared `backup-status.ts`); admin card shows upload state; readyz gained a warning-level `backup_remote` check (26 h daily / 170 h weekly); upload failures alert via a new `backupUploadFailed` mail (de+en). **Hardening found by the live verify** (fresh stack + fake WebDAV server, full configure→backup→upload→remote-restore cycle): the restore's connection termination crashed the api through an unhandled worker rejection — conversion/mail/scheduler sweeps now catch transient DB failures; and `main.ts` waits while `restore-status.json` says `running` so a container restarting mid-restore can never race `migrate deploy` against `pg_restore`. **Notes / deviations** - The upload schedule interprets "custom hour" as: uploads ride on the nightly run (`BACKUP_TIME`); a separate upload-only clock did not seem worth the extra setting. - Restore picker also offers **local** sets (free, useful without Nextcloud); the issue only required remote. - e2e: 8-test api pack (`backup-admin.e2e.db.test.ts`) incl. maintenance-gate semantics + staleness; sidecar tests run against an in-process WebDAV server (streamed PUT, prune, bundle roundtrip); collab listener unit-tested. Acceptance criteria all met; credentials verified absent from `instance_settings`, exports and logs.

fable-5 closed this issue

2026-07-12 10:51:41 +02:00

fable-5 referenced this issue from a commit

2026-07-12 12:20:33 +02:00

Backup mirror to BASEL: rsync of the sets after every successful run (#84)

fable-5 referenced this issue

2026-07-12 12:33:32 +02:00

Off-host backup mirror to BASEL #84

Sign in to join this conversation.