Claude Fable 5
5cef359b8f
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m45s
CD / Build and push images (push) Successful in 3m49s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m35s
CI / Import/export fidelity gate (push) Successful in 47s
Off-host backups for every self-hoster, configured entirely in the admin UI — supersedes the host-specific mirror plan behind #84. shared: - webdav.ts (new package entry like token-crypto): minimal WebDAV client with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT (streamed), GET, DELETE; Nextcloud DAV path derived from the plain server URL, explicit DAV bases pass through - backup-status.ts: additive remote-upload status in status.json, the restore-status.json contract (running/succeeded/failed + staleness bound), the backup_command/backup_maintenance NOTIFY channels, and the one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz) - backup-set.ts moved here from apps/backup (api lists local sets) backup sidecar: - reads the backup.* instance settings directly from the database (admin changes apply next run; local retention row overrides the env) and the app password from the secret store - after each successful set: bundle dump + files archive + manifest into ONE self-contained tar.gz, upload via WebDAV per schedule (off/daily/weekly; manual runs always upload), prune remote bundles — never the newest — and record the outcome in status.json; upload failures alert via a new backupUploadFailed mail (de+en) - command listener on backup_command (run / restore) with a serial queue against the nightly timer - restore orchestrator: restore-status.json → maintenance NOTIFY → grace → (remote: download + manifest-verify bundle) → terminate other DB connections → shared perform-restore path (same code as restore.sh) → final status + maintenance exit api: - MaintenanceGuard (global, registered before the setup gate): 503 maintenance_mode while restore-status says running; health endpoints and the new public GET /backup/restore-status stay exempt; a stale running state (crashed sidecar) unblocks after 30 min - MaintenanceStateService watches the file and restarts the api after a successful restore (fresh caches, migrate-on-start for older dumps); main.ts refuses to touch the database while a restore runs — a container restarting mid-restore must not race pg_restore with migrate deploy - worker sweeps (conversion, mail outbox, scheduler) catch transient database failures instead of dying on an unhandled rejection — the restore's connection termination crashed the api in verification - backup admin endpoints under /admin/system/backup: settings (live connection test before save, password write-only into the secret store), nextcloud/test, sets (local via the ro backups mount + remote via WebDAV), run + restore (type-to-confirm backstop, source validation) — commands travel as NOTIFY payloads; audit actions backup.settings_changed/run_triggered/restore_requested - readyz: new warning-level backup_remote check while a target is configured (26 h daily / 170 h weekly bound) collab: - maintenance listener: on enter, persist + close every live session and refuse new connections until exit (failsafe timeout 30 min) — no in-memory document may write pre-restore content back afterwards web: - Admin → System backup section: status card with remote facts and a "Back up now" button, the Nextcloud settings form with test button, and the restore picker (local + remote sets, type-to-confirm) - global maintenance screen: any 503 maintenance_mode flips the SPA to a status page polling the exempt endpoint, reloading when the instance returns Verified end-to-end against a live stack (fresh DB, native api + sidecar, fake WebDAV server): configure → test → manual backup → bundle upload → readyz/sets/status surfaces → remote restore with maintenance gate, marker rollback and api restart; suites: shared 21, backup 9, collab 11, api 58 files green, lint + i18n:check + typecheck clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
178 lines
7.8 KiB
Markdown
178 lines
7.8 KiB
Markdown
# Self-hosting Dorfteich
|
|
|
|
Everything you need to install, run, update, and back up your own Dorfteich
|
|
with Docker — this guide is the complete contract: if a step here does not
|
|
work, that is a bug (issue #88).
|
|
|
|
> Docs are English-only by decision (issue #88): the product UI is fully
|
|
> localized (de/en), operator documentation is not — one authoritative text
|
|
> beats two drifting ones.
|
|
|
|
## Requirements
|
|
|
|
- Docker Engine with the Compose plugin (`docker compose version` ≥ 2.20).
|
|
- 2 GB RAM, ~2 GB disk for images plus room for your content and backups.
|
|
- A domain pointing at the host — TLS via your own reverse proxy **or** the
|
|
bundled `caddy` profile (below).
|
|
- Outbound SMTP relay (optional at install time: the setup wizard can
|
|
configure it later, or you skip mail entirely at first).
|
|
|
|
## Install
|
|
|
|
1. Create a directory and fetch the two reference files from the repository
|
|
(`deploy/compose/`): `docker-compose.yml`, `.env.example` — plus
|
|
`Caddyfile` if you want the `caddy` profile.
|
|
|
|
```sh
|
|
mkdir dorfteich && cd dorfteich
|
|
# copy docker-compose.yml, .env.example (and Caddyfile) here
|
|
cp .env.example .env && chmod 600 .env
|
|
```
|
|
|
|
2. Edit `.env` — the minimum:
|
|
- `POSTGRES_PASSWORD`, `COLLAB_TOKEN_SECRET`: long random strings
|
|
(`openssl rand -base64 32`).
|
|
- `IMAGE_PREFIX=gitea.101010.cloud/stwaidele/dorfteich` and `TAG`:
|
|
releases are semver tags (`v1.2.3`); until the first public release,
|
|
`test` tracks the latest verified build.
|
|
- `APP_BASE_URL=https://wiki.example.com` — must be exactly what
|
|
browsers will use; e-mail links and the CSRF origin check derive
|
|
from it.
|
|
- Every other variable is documented inline in `.env.example` with its
|
|
default and effect; nothing outside that file configures the stack.
|
|
|
|
3. Start:
|
|
|
|
```sh
|
|
docker compose up -d # behind your own reverse proxy
|
|
docker compose --profile caddy up -d # or with the bundled TLS ingress
|
|
```
|
|
|
|
- **Own proxy:** route `/*` → `127.0.0.1:$WEB_PORT`, `/api/*` →
|
|
`$API_PORT`, `/collab*` → `$COLLAB_PORT` (**WebSocket upgrade
|
|
required** on `/collab`).
|
|
- **`caddy` profile:** set `DOMAIN=wiki.example.com` in `.env`; Caddy
|
|
publishes 80/443 and obtains Let's Encrypt certificates automatically
|
|
(both ports must be reachable from the internet). `DOMAIN=localhost`
|
|
issues an internal-CA certificate — for smoke tests only.
|
|
|
|
4. Open `https://your-domain/` — a fresh instance shows the **first-run
|
|
setup wizard**.
|
|
|
|
## First-run wizard
|
|
|
|
Six steps, all in the browser (issue #80/#81): language → the Site-Admin
|
|
account (created verified, you are signed in immediately) → instance name
|
|
and default language → SMTP relay (**Save runs a live test** and sends a
|
|
test mail to you; _Skip_ leaves mail unconfigured — sign-up verification
|
|
will not work until an admin adds a relay) → registration mode
|
|
(open/closed) → summary + finish. The wizard locks itself permanently on
|
|
completion. Until it completes, the api answers everything but the wizard
|
|
and health endpoints with `503 setup_required` — that is not an error.
|
|
|
|
Unattended installs skip the wizard by pre-seeding: set the
|
|
`SETUP_ADMIN_*` variables in `.env` before the first start (see
|
|
`.env.example`).
|
|
|
|
## Updating
|
|
|
|
```sh
|
|
# edit .env: TAG=v1.3.0
|
|
docker compose pull && docker compose up -d
|
|
```
|
|
|
|
Database migrations run automatically at api start. Release notes flag
|
|
releases with a `migration` label and any manual steps. **Downgrade
|
|
window: one minor release** — `TAG` back + `pull` + `up -d` is supported
|
|
one step back; further back, restore the backup taken before the update
|
|
instead (the nightly sidecar gives you one at most 24 h old).
|
|
|
|
## Backups & restore
|
|
|
|
Enabled by default (ADR 0015): the `backup` sidecar dumps the database and
|
|
archives the uploads/plugins volumes nightly at `BACKUP_TIME` onto the
|
|
`backups` volume, prunes by `BACKUP_RETENTION_DAYS`, writes `status.json`,
|
|
and — with `BACKUP_MAIL_TO` set — mails you on failure.
|
|
|
|
- On-demand backup: `docker compose run --rm -e BACKUP_RUN_ONCE=1 backup`,
|
|
or the **Back up now** button under _Admin → System_.
|
|
- List sets: `docker compose exec backup ls /backups`
|
|
- Restore: `./restore.sh <backup-id>` (fetch `deploy/backup/restore.sh`
|
|
next to your compose file) — details in
|
|
`docs/operations/restore-runbook.md`.
|
|
|
|
### Off-host copies to a Nextcloud
|
|
|
|
Get the backups off the host — a backup on the same disk protects against
|
|
mistakes, not against losing the host. Any Nextcloud you can reach works
|
|
as the target; configure it entirely in the admin UI (_Admin → System →
|
|
Backups_):
|
|
|
|
1. In Nextcloud, create an **app password** for the account that should
|
|
hold the backups (Settings → Security → Devices & sessions).
|
|
2. In Dorfteich, enable _Upload backups to Nextcloud_, enter the plain
|
|
Nextcloud address (e.g. `https://cloud.example.com`), the username, the
|
|
app password and a folder, and use **Test connection** — it verifies
|
|
the credentials and creates the folder. The password is kept in the
|
|
secret store on the `secrets` volume, never in the database.
|
|
3. Pick the upload schedule (after every nightly backup, weekly, or manual
|
|
only) and the retention for both sides. After each successful upload,
|
|
old remote bundles beyond the retention are pruned — never the newest
|
|
one.
|
|
|
|
Each upload is ONE self-contained archive
|
|
(`dorfteich-backup-<id>.tar.gz` = database dump + files archive +
|
|
manifest) — everything needed to rebuild the instance after total loss.
|
|
`readyz` warns (`backup_remote` check) when the off-host copy grows stale,
|
|
and upload failures alert through the backup failure mail.
|
|
|
|
**Restore from the admin UI:** _Admin → System → Backups → Restore_ lists
|
|
local and Nextcloud sets. Restoring asks you to re-type the backup id,
|
|
then the instance enters maintenance mode (everything answers 503 plus a
|
|
status page), restores itself through the backup sidecar, and restarts.
|
|
If the app itself is gone, use the operator path in
|
|
`docs/operations/restore-runbook.md` instead — it documents fetching a
|
|
bundle from Nextcloud by hand.
|
|
|
|
## Health & troubleshooting
|
|
|
|
- `GET /api/v1/readyz` is the instance's own diagnosis. HTTP 503 =
|
|
database/migrations broken (the instance cannot serve). HTTP 200 with
|
|
`"status":"degraded"` = a warning-level check: `converter`/`renderer`
|
|
down (import/export/PDF degrade, everything else works) or `backup`
|
|
stale (last success older than 26 h). Each check carries a `detail`.
|
|
Monitor set: `deploy/monitoring.md`.
|
|
- Logs: `docker compose logs api` (or `web`, `collab`, `backup`, `db`) —
|
|
structured JSON, rotated by Docker.
|
|
- **Proxy pitfalls:** editor never connects / "offline" although the page
|
|
loads → the proxy does not upgrade WebSockets on `/collab`. All
|
|
mutations fail with 403 `csrf_origin_mismatch` → `APP_BASE_URL` does not
|
|
match the URL in the browser (scheme and host must be identical).
|
|
E-mail links point at the wrong host → same variable.
|
|
- Wizard reappears after a restart → the database volume was not
|
|
persisted; never run without the `db-data` volume.
|
|
- `docker compose ps` shows `unhealthy` → that container's liveness check
|
|
fails; a _degraded_ readyz alone never marks containers unhealthy and
|
|
never restarts anything.
|
|
|
|
## Building from source instead
|
|
|
|
Clone the repository and build the images locally — the reference compose
|
|
carries the build contexts already:
|
|
|
|
```sh
|
|
docker compose build && docker compose up -d
|
|
```
|
|
|
|
Same layout, same volumes; you trade the registry pull for a local
|
|
toolchain (Node 22 build stages run inside Docker, nothing else needed).
|
|
|
|
## Verified install
|
|
|
|
The guide is verified by a scripted clean-machine run
|
|
(`deploy/self-hosting-verify.sh`): fresh directory, reference compose +
|
|
`.env.example` only, `--profile caddy` with an internal-CA certificate,
|
|
asserting that the wizard answers over TLS. Run it yourself on any Docker
|
|
host — it uses its own compose project name and high ports, then removes
|
|
everything.
|