stwaidele/dorfteich
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 22 Wiki Activity

Off-host backup mirror to BASEL #84

New Issue
Closed
opened 2026-07-04 14:52:45 +02:00 by fable-5 · 2 comments
fable-5 commented 2026-07-04 14:52:45 +02:00
Collaborator
Copy Link

Context

Prod backups are mirrored nightly to the BASEL host over WireGuard (ADR 0015, operator standard with the dedicated-user improvement).

Scope

Extend the sidecar (Prod profile): rsync of the backup directory to dorfteich-backup@BASEL:/home/RAID/BACKUPS/dorfteich-prod/ after the local run, SSH key from a mounted secret, remote retention alignment, mirror outcome in status.json; document the BASEL-side setup (dedicated user dorfteich-backup with home under /home/, authorized_keys, directory permissions) in deploy/backup-basel.md for execution with the repo owner — explicitly avoiding the Debian backup system-user pitfalls noted in the operator's conventions.

Acceptance criteria

  • mirror run transfers new sets and is idempotent (re-run transfers nothing)
  • mirror failure is visible in status.json and the failure mail, while local backups continue
  • BASEL doc contains the complete user/key/permission setup incl. the nologin-shell warning
  • no credentials in the image or repo (mounted secret only)

Technical notes

  • ADR 0015 (deviation notes for Test/Int), global backup conventions.

Dependencies

Depends on #83.

Size: ~1 day

Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add de and en), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.

## Context Prod backups are mirrored nightly to the BASEL host over WireGuard (ADR 0015, operator standard with the dedicated-user improvement). ## Scope Extend the sidecar (Prod profile): rsync of the backup directory to `dorfteich-backup@BASEL:/home/RAID/BACKUPS/dorfteich-prod/` after the local run, SSH key from a mounted secret, remote retention alignment, mirror outcome in status.json; document the BASEL-side setup (dedicated user `dorfteich-backup` with home under `/home/`, authorized_keys, directory permissions) in `deploy/backup-basel.md` for execution with the repo owner — explicitly avoiding the Debian `backup` system-user pitfalls noted in the operator's conventions. ## Acceptance criteria - [ ] mirror run transfers new sets and is idempotent (re-run transfers nothing) - [ ] mirror failure is visible in status.json and the failure mail, while local backups continue - [ ] BASEL doc contains the complete user/key/permission setup incl. the nologin-shell warning - [ ] no credentials in the image or repo (mounted secret only) ## Technical notes - ADR 0015 (deviation notes for Test/Int), global backup conventions. ## Dependencies Depends on #83. **Size**: ~1 day --- *Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add `de` **and** `en`), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.*
fable-5 added this to the M8 — Self-hosting & operations milestone 2026-07-04 14:52:45 +02:00
fable-5 added the
deployment
 label 2026-07-04 14:52:45 +02:00
fable-5 added the
blocked
 label 2026-07-12 09:37:01 +02:00
fable-5 commented 2026-07-12 09:37:01 +02:00
Author
Collaborator
Copy Link

Status: blocked — dieser Mirror braucht den WireGuard-Tunnel ONE→BASEL, der wegen des episodischen Inbound-Paketverlusts an der BASEL-Site (Analyse: Telekom-Policing bzw. Fritz!Box-7690-Verdacht, siehe Handoff-Wireguard.md) derzeit nicht stabil steht.

Der generische Off-Host-Backup-Bedarf für Selbst-Hoster wird stattdessen von #103 übernommen (admin-konfigurierbares Nextcloud-Ziel inkl. In-App-Restore). #84 bleibt als persönliches Zusatzziel offen: eine BASEL-Kopie lässt sich nach #103 wahlweise auch über eine Nextcloud auf BASEL abbilden — falls der rsync-Weg dann noch gebraucht wird, hier neu bewerten.

Status: **blocked** — dieser Mirror braucht den WireGuard-Tunnel ONE→BASEL, der wegen des episodischen Inbound-Paketverlusts an der BASEL-Site (Analyse: Telekom-Policing bzw. Fritz!Box-7690-Verdacht, siehe Handoff-Wireguard.md) derzeit nicht stabil steht. Der generische Off-Host-Backup-Bedarf für Selbst-Hoster wird stattdessen von #103 übernommen (admin-konfigurierbares Nextcloud-Ziel inkl. In-App-Restore). #84 bleibt als persönliches Zusatzziel offen: eine BASEL-Kopie lässt sich nach #103 wahlweise auch über eine Nextcloud auf BASEL abbilden — falls der rsync-Weg dann noch gebraucht wird, hier neu bewerten.
fable-5 commented 2026-07-12 12:33:32 +02:00
Author
Collaborator
Copy Link

Implemented in 52192eb (+ 203f7c9 go-live checklist tick) — unblocked by the tunnel fix, CI fully green, mirror is live on the Test stage.

What shipped

  • Sidecar mirror step (mirror.ts), env-only configuration (operator territory, unlike the admin-configured Nextcloud target #103): BACKUP_MIRROR_TARGET (rsync-over-ssh), BACKUP_MIRROR_SSH_KEY (private key on the secrets volume, never in image or repo), BACKUP_MIRROR_SSH_PORT. rsync + openssh-client added to the sidecar image.
  • Runs after the prune of every successful run, so --delete aligns the remote retention with the local one — the newest-complete-set guarantee carries over. Only set files travel (db-*.dump, files-*.tar.gz). Host key pinned via accept-new into .mirror_known_hosts on the backups volume; fixed remote modes (dirs 750, files 640).
  • status.json gained an additive mirror block (outcome, transferred count, lastSuccessAt carried across failures) — shown on the admin backup card; failures alert via the new backupMirrorFailed mail (de+en) while local backups continue.
  • deploy/backup-basel.md: complete BASEL walkthrough — dedicated user dorfteich-backup with a /home/ home and a bash login shell (explicitly avoiding the Debian backup/UID-34 pitfalls: nologin rejects rsync sessions, /var/backups home), key placement through the api container, .env values, verification.

Acceptance criteria

  • ✔ idempotent: verified live — re-runs transfer 0 files for existing sets (unit-tested against real rsync too)
  • ✔ mirror failure visible in status.json + alert mail while the local run stays succeeded (failure path exercised live during verification)
  • ✔ BASEL doc incl. the nologin warning
  • ✔ no credentials in image or repo (key generated on the hosts, placed on the secrets volume, mode 600)

Provisioned + verified

  • BASEL: user dorfteich-backup (uid 1001, /home/dorfteich-backup, bash), per-stage keys in authorized_keys, target dirs dorfteich-test/dorfteich-prod (750).
  • Test stage: compose + .env + key installed, on-demand run green — the nightly sets (incl. last night's) now sit on BASEL, status.json → mirror.lastRun = succeeded.
  • Prod: .env, compose and key are already in place; the running v0.1.2 sidecar ignores the new vars, so the mirror activates automatically with the next release deploy — noted on the go-live checklist (verify status.json → mirror after the first prod deploy).

Restore from the mirror = copy the set pair back into the backups volume + ./restore.sh <id> (runbook).

Implemented in `52192eb` (+ `203f7c9` go-live checklist tick) — unblocked by the tunnel fix, CI fully green, **mirror is live on the Test stage**. **What shipped** - Sidecar mirror step (`mirror.ts`), env-only configuration (operator territory, unlike the admin-configured Nextcloud target #103): `BACKUP_MIRROR_TARGET` (rsync-over-ssh), `BACKUP_MIRROR_SSH_KEY` (private key on the **secrets volume**, never in image or repo), `BACKUP_MIRROR_SSH_PORT`. rsync + openssh-client added to the sidecar image. - Runs **after the prune** of every successful run, so `--delete` aligns the remote retention with the local one — the newest-complete-set guarantee carries over. Only set files travel (`db-*.dump`, `files-*.tar.gz`). Host key pinned via `accept-new` into `.mirror_known_hosts` on the backups volume; fixed remote modes (dirs 750, files 640). - `status.json` gained an additive `mirror` block (outcome, transferred count, `lastSuccessAt` carried across failures) — shown on the admin backup card; failures alert via the new `backupMirrorFailed` mail (de+en) while local backups continue. - **`deploy/backup-basel.md`**: complete BASEL walkthrough — dedicated user `dorfteich-backup` with a `/home/` home and a **bash login shell** (explicitly avoiding the Debian `backup`/UID-34 pitfalls: nologin rejects rsync sessions, `/var/backups` home), key placement through the api container, `.env` values, verification. **Acceptance criteria** - ✔ idempotent: verified live — re-runs transfer 0 files for existing sets (unit-tested against real rsync too) - ✔ mirror failure visible in `status.json` + alert mail while the local run stays succeeded (failure path exercised live during verification) - ✔ BASEL doc incl. the nologin warning - ✔ no credentials in image or repo (key generated on the hosts, placed on the secrets volume, mode 600) **Provisioned + verified** - BASEL: user `dorfteich-backup` (uid 1001, `/home/dorfteich-backup`, bash), per-stage keys in `authorized_keys`, target dirs `dorfteich-test`/`dorfteich-prod` (750). - Test stage: compose + `.env` + key installed, on-demand run green — the nightly sets (incl. last night's) now sit on BASEL, `status.json → mirror.lastRun = succeeded`. - **Prod**: `.env`, compose and key are already in place; the running v0.1.2 sidecar ignores the new vars, so the mirror **activates automatically with the next release deploy** — noted on the go-live checklist (verify `status.json → mirror` after the first prod deploy). Restore from the mirror = copy the set pair back into the backups volume + `./restore.sh <id>` (runbook).
fable-5 removed the
blocked
 label 2026-07-12 12:33:32 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
area:auth

   
area:docs

   
area:export

   
area:ops

   
area:storage

   
area:supply-chain

   
auth

Authentication, permissions, quotas

  
backend

API server (apps/api)

  
blocked

Blocked on external dependency/infrastructure

  
collab

Collaboration server (apps/collab)

  
deployment

Docker, CI/CD, hosting

  
docs

Documentation

  
effort:L

   
effort:M

   
effort:S

   
frontend

SPA (apps/web)

  
plugins

Plugin system and SDK

  
qa

Tests, fixtures, e2e

  
vs-nfd

   
vs-nfd:blocker
No Label
area:auth
area:docs
area:export
area:ops
area:storage
area:supply-chain
auth
backend
blocked
collab
deployment
docs
effort:L
effort:M
effort:S
frontend
plugins
qa
vs-nfd
vs-nfd:blocker
Milestone
Clear milestone
No items
No Milestone
M8 — Self-hosting & operations
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: stwaidele/dorfteich#84
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?