stwaidele/dorfteich
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 22 Wiki Activity

Restore drills (log) #98

New Issue
Open
opened 2026-07-11 20:54:18 +02:00 by fable-5 · 5 comments
fable-5 commented 2026-07-11 20:54:18 +02:00
Collaborator
Copy Link

Log of the monthly automated restore drills (issue #87, ADR 0015): the Restore drill workflow restores the latest backup set into a scratch environment, verifies it, and reports each outcome as a comment here. Runbook: docs/operations/restore-runbook.md. This issue stays open as the pinned drill log.

Log of the monthly automated restore drills (issue #87, ADR 0015): the `Restore drill` workflow restores the latest backup set into a scratch environment, verifies it, and reports each outcome as a comment here. Runbook: `docs/operations/restore-runbook.md`. This issue stays open as the pinned drill log.
fable-5 pinned this 2026-07-11 20:54:19 +02:00
gitea-actions commented 2026-07-11 21:01:32 +02:00
Copy Link

Restore drill success — source dorfteich-test_backups, run 201

drill: drilling backup set 20260711-154136 from dorfteich-test_backups
{"level":30,"time":1783796476556,"service":"backup-restore","backupId":"20260711-154136","msg":"restoring database dump"}
{"level":30,"time":1783796476771,"service":"backup-restore","backupId":"20260711-154136","msg":"restoring uploads/plugins archive"}
{"level":30,"time":1783796476774,"service":"backup-restore","backupId":"20260711-154136","msg":"restore complete — start the stack and verify /readyz"}
drill: readyz: database ok
drill: readyz: migrations ok
drill: row counts: 3 users, 12 pages (incl. trash)
drill: rendered pages in content cache: 12
drill: public api render check: 200
drill: media byte-check: skipped (no attachments in this backup)
drill: OK — set 20260711-154136 restored and verified (3 users, 12 pages, 12 rendered)
drill: tearing down scratch environment dorfteich-drill-1783796471-172
**Restore drill success** — source `dorfteich-test_backups`, run 201 ``` drill: drilling backup set 20260711-154136 from dorfteich-test_backups {"level":30,"time":1783796476556,"service":"backup-restore","backupId":"20260711-154136","msg":"restoring database dump"} {"level":30,"time":1783796476771,"service":"backup-restore","backupId":"20260711-154136","msg":"restoring uploads/plugins archive"} {"level":30,"time":1783796476774,"service":"backup-restore","backupId":"20260711-154136","msg":"restore complete — start the stack and verify /readyz"} drill: readyz: database ok drill: readyz: migrations ok drill: row counts: 3 users, 12 pages (incl. trash) drill: rendered pages in content cache: 12 drill: public api render check: 200 drill: media byte-check: skipped (no attachments in this backup) drill: OK — set 20260711-154136 restored and verified (3 users, 12 pages, 12 rendered) drill: tearing down scratch environment dorfteich-drill-1783796471-172 ```
gitea-actions commented 2026-07-12 19:59:00 +02:00
Copy Link

Restore drill failure — source dorfteich-prod_backups, run 282

drill: drilling backup set 20260712-135349 from dorfteich-prod_backups
{"level":30,"time":1783879132282,"service":"backup-restore","backupId":"20260712-135349","msg":"restoring database dump"}
{"level":30,"time":1783879132546,"service":"backup-restore","backupId":"20260712-135349","msg":"restoring uploads/plugins archive"}
{"level":30,"time":1783879132550,"service":"backup-restore","backupId":"20260712-135349","msg":"restore complete — start the stack and verify /readyz"}
drill: readyz: database ok
drill: readyz: migrations ok
drill: FAILED — restored database has no users
drill: tearing down scratch environment dorfteich-drill-1783879118-172
**Restore drill failure** — source `dorfteich-prod_backups`, run 282 ``` drill: drilling backup set 20260712-135349 from dorfteich-prod_backups {"level":30,"time":1783879132282,"service":"backup-restore","backupId":"20260712-135349","msg":"restoring database dump"} {"level":30,"time":1783879132546,"service":"backup-restore","backupId":"20260712-135349","msg":"restoring uploads/plugins archive"} {"level":30,"time":1783879132550,"service":"backup-restore","backupId":"20260712-135349","msg":"restore complete — start the stack and verify /readyz"} drill: readyz: database ok drill: readyz: migrations ok drill: FAILED — restored database has no users drill: tearing down scratch environment dorfteich-drill-1783879118-172 ```
gitea-actions commented 2026-07-12 20:38:02 +02:00
Copy Link

Restore drill success — source dorfteich-prod_backups, run 283

drill: drilling backup set 20260712-183629 from dorfteich-prod_backups
{"level":30,"time":1783881473015,"service":"backup-restore","backupId":"20260712-183629","msg":"restoring database dump"}
{"level":30,"time":1783881473351,"service":"backup-restore","backupId":"20260712-183629","msg":"restoring uploads/plugins archive"}
{"level":30,"time":1783881473356,"service":"backup-restore","backupId":"20260712-183629","msg":"restore complete — start the stack and verify /readyz"}
drill: readyz: database ok
drill: readyz: migrations ok
drill: row counts: 2 users, 16 pages (incl. trash)
drill: rendered pages in content cache: 16
drill: public api render check: 200
drill: media byte-check: skipped (no attachments in this backup)
drill: OK — set 20260712-183629 restored and verified (2 users, 16 pages, 16 rendered)
drill: tearing down scratch environment dorfteich-drill-1783881468-171
**Restore drill success** — source `dorfteich-prod_backups`, run 283 ``` drill: drilling backup set 20260712-183629 from dorfteich-prod_backups {"level":30,"time":1783881473015,"service":"backup-restore","backupId":"20260712-183629","msg":"restoring database dump"} {"level":30,"time":1783881473351,"service":"backup-restore","backupId":"20260712-183629","msg":"restoring uploads/plugins archive"} {"level":30,"time":1783881473356,"service":"backup-restore","backupId":"20260712-183629","msg":"restore complete — start the stack and verify /readyz"} drill: readyz: database ok drill: readyz: migrations ok drill: row counts: 2 users, 16 pages (incl. trash) drill: rendered pages in content cache: 16 drill: public api render check: 200 drill: media byte-check: skipped (no attachments in this backup) drill: OK — set 20260712-183629 restored and verified (2 users, 16 pages, 16 rendered) drill: tearing down scratch environment dorfteich-drill-1783881468-171 ```
gitea-actions commented 2026-08-01 04:19:18 +02:00
Copy Link

Restore drill success — source dorfteich-prod_backups, run 652

drill: drilling backup set 20260801-010000 from dorfteich-prod_backups
{"level":30,"time":1785550744999,"service":"backup-restore","backupId":"20260801-010000","msg":"restoring database dump"}
{"level":30,"time":1785550745879,"service":"backup-restore","backupId":"20260801-010000","msg":"restoring uploads/plugins archive"}
{"level":30,"time":1785550746880,"service":"backup-restore","backupId":"20260801-010000","msg":"restore complete — start the stack and verify /readyz"}
drill: readyz: database ok
drill: readyz: migrations ok
drill: row counts: 5 users, 236 pages (incl. trash)
drill: rendered pages in content cache: 236
drill: public api render check: 200
drill: media byte-check: 7797846c-1a33-4d6a-98b6-6338db7603f1/7252e3b5-4054-4e2a-91d2-68988deb7e43 (337034 bytes) matches
drill: OK — set 20260801-010000 restored and verified (5 users, 236 pages, 236 rendered)
drill: tearing down scratch environment dorfteich-drill-1785550738-171
**Restore drill success** — source `dorfteich-prod_backups`, run 652 ``` drill: drilling backup set 20260801-010000 from dorfteich-prod_backups {"level":30,"time":1785550744999,"service":"backup-restore","backupId":"20260801-010000","msg":"restoring database dump"} {"level":30,"time":1785550745879,"service":"backup-restore","backupId":"20260801-010000","msg":"restoring uploads/plugins archive"} {"level":30,"time":1785550746880,"service":"backup-restore","backupId":"20260801-010000","msg":"restore complete — start the stack and verify /readyz"} drill: readyz: database ok drill: readyz: migrations ok drill: row counts: 5 users, 236 pages (incl. trash) drill: rendered pages in content cache: 236 drill: public api render check: 200 drill: media byte-check: 7797846c-1a33-4d6a-98b6-6338db7603f1/7252e3b5-4054-4e2a-91d2-68988deb7e43 (337034 bytes) matches drill: OK — set 20260801-010000 restored and verified (5 users, 236 pages, 236 rendered) drill: tearing down scratch environment dorfteich-drill-1785550738-171 ```
gitea-actions commented 2026-09-01 04:18:16 +02:00
Copy Link

Restore drill failure — source dorfteich-prod_backups, run 761

drill: drilling backup set 20260901-010000 from dorfteich-prod_backups
{"level":30,"time":1788229092805,"service":"backup-restore","backupId":"20260901-010000","msg":"restoring database dump"}
{"level":30,"time":1788229093402,"service":"backup-restore","backupId":"20260901-010000","msg":"restoring the data-directory archive"}
{"level":50,"time":1788229094390,"service":"backup-restore","backupId":"20260901-010000","error":"Error: tar failed: tar: fonts: Cannot mkdir: Permission denied\ntar: branding: Cannot mkdir: Permission denied\ntar: Exiting with failure status due to previous errors","msg":"restore failed"}
drill: FAILED — restore of 20260901-010000 did not complete
drill: tearing down scratch environment dorfteich-drill-1788229087-172
**Restore drill failure** — source `dorfteich-prod_backups`, run 761 ``` drill: drilling backup set 20260901-010000 from dorfteich-prod_backups {"level":30,"time":1788229092805,"service":"backup-restore","backupId":"20260901-010000","msg":"restoring database dump"} {"level":30,"time":1788229093402,"service":"backup-restore","backupId":"20260901-010000","msg":"restoring the data-directory archive"} {"level":50,"time":1788229094390,"service":"backup-restore","backupId":"20260901-010000","error":"Error: tar failed: tar: fonts: Cannot mkdir: Permission denied\ntar: branding: Cannot mkdir: Permission denied\ntar: Exiting with failure status due to previous errors","msg":"restore failed"} drill: FAILED — restore of 20260901-010000 did not complete drill: tearing down scratch environment dorfteich-drill-1788229087-172 ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
area:auth

   
area:docs

   
area:export

   
area:ops

   
area:storage

   
area:supply-chain

   
auth

Authentication, permissions, quotas

  
backend

API server (apps/api)

  
blocked

Blocked on external dependency/infrastructure

  
collab

Collaboration server (apps/collab)

  
deployment

Docker, CI/CD, hosting

  
docs

Documentation

  
effort:L

   
effort:M

   
effort:S

   
frontend

SPA (apps/web)

  
plugins

Plugin system and SDK

  
qa

Tests, fixtures, e2e

  
vs-nfd

   
vs-nfd:blocker
No Label
area:auth
area:docs
area:export
area:ops
area:storage
area:supply-chain
auth
backend
blocked
collab
deployment
docs
effort:L
effort:M
effort:S
frontend
plugins
qa
vs-nfd
vs-nfd:blocker
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: stwaidele/dorfteich#98
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?