All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m45s
CD / Build and push images (push) Successful in 3m49s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m35s
CI / Import/export fidelity gate (push) Successful in 47s
Off-host backups for every self-hoster, configured entirely in the admin UI — supersedes the host-specific mirror plan behind #84. shared: - webdav.ts (new package entry like token-crypto): minimal WebDAV client with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT (streamed), GET, DELETE; Nextcloud DAV path derived from the plain server URL, explicit DAV bases pass through - backup-status.ts: additive remote-upload status in status.json, the restore-status.json contract (running/succeeded/failed + staleness bound), the backup_command/backup_maintenance NOTIFY channels, and the one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz) - backup-set.ts moved here from apps/backup (api lists local sets) backup sidecar: - reads the backup.* instance settings directly from the database (admin changes apply next run; local retention row overrides the env) and the app password from the secret store - after each successful set: bundle dump + files archive + manifest into ONE self-contained tar.gz, upload via WebDAV per schedule (off/daily/weekly; manual runs always upload), prune remote bundles — never the newest — and record the outcome in status.json; upload failures alert via a new backupUploadFailed mail (de+en) - command listener on backup_command (run / restore) with a serial queue against the nightly timer - restore orchestrator: restore-status.json → maintenance NOTIFY → grace → (remote: download + manifest-verify bundle) → terminate other DB connections → shared perform-restore path (same code as restore.sh) → final status + maintenance exit api: - MaintenanceGuard (global, registered before the setup gate): 503 maintenance_mode while restore-status says running; health endpoints and the new public GET /backup/restore-status stay exempt; a stale running state (crashed sidecar) unblocks after 30 min - MaintenanceStateService watches the file and restarts the api after a successful restore (fresh caches, migrate-on-start for older dumps); main.ts refuses to touch the database while a restore runs — a container restarting mid-restore must not race pg_restore with migrate deploy - worker sweeps (conversion, mail outbox, scheduler) catch transient database failures instead of dying on an unhandled rejection — the restore's connection termination crashed the api in verification - backup admin endpoints under /admin/system/backup: settings (live connection test before save, password write-only into the secret store), nextcloud/test, sets (local via the ro backups mount + remote via WebDAV), run + restore (type-to-confirm backstop, source validation) — commands travel as NOTIFY payloads; audit actions backup.settings_changed/run_triggered/restore_requested - readyz: new warning-level backup_remote check while a target is configured (26 h daily / 170 h weekly bound) collab: - maintenance listener: on enter, persist + close every live session and refuse new connections until exit (failsafe timeout 30 min) — no in-memory document may write pre-restore content back afterwards web: - Admin → System backup section: status card with remote facts and a "Back up now" button, the Nextcloud settings form with test button, and the restore picker (local + remote sets, type-to-confirm) - global maintenance screen: any 503 maintenance_mode flips the SPA to a status page polling the exempt endpoint, reloading when the instance returns Verified end-to-end against a live stack (fresh DB, native api + sidecar, fake WebDAV server): configure → test → manual backup → bundle upload → readyz/sets/status surfaces → remote restore with maintenance gate, marker rollback and api restart; suites: shared 21, backup 9, collab 11, api 58 files green, lint + i18n:check + typecheck clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
274 lines
10 KiB
TypeScript
274 lines
10 KiB
TypeScript
/**
|
|
* Minimal WebDAV client for the Nextcloud backup target (issue #103), shared
|
|
* by the api (connection test, remote set listing) and the backup sidecar
|
|
* (upload, prune, download). Plain `fetch` with basic auth — no heavy DAV
|
|
* dependency; the subset used here (PROPFIND/MKCOL/PUT/GET/DELETE) is stable
|
|
* across Nextcloud versions and generic WebDAV servers.
|
|
*
|
|
* Like `token-crypto`, this module is a separate package entry (not part of
|
|
* the barrel) because it is server-only.
|
|
*/
|
|
|
|
export interface WebDavTarget {
|
|
/**
|
|
* The Nextcloud base URL (e.g. `https://cloud.example.com`) — the DAV
|
|
* path `remote.php/dav/files/<username>` is derived. A URL that already
|
|
* contains `remote.php` or `/dav/` is used as the DAV base verbatim, so
|
|
* generic WebDAV servers work too.
|
|
*/
|
|
baseUrl: string;
|
|
username: string;
|
|
password: string;
|
|
/** Target folder under the DAV base, may contain `/` for nesting. */
|
|
folder: string;
|
|
}
|
|
|
|
export interface WebDavEntry {
|
|
/** Decoded file or collection name (last path segment). */
|
|
name: string;
|
|
isCollection: boolean;
|
|
sizeBytes: number | null;
|
|
lastModified: string | null;
|
|
}
|
|
|
|
export type WebDavResult<T> = { ok: true; value: T } | { ok: false; error: string };
|
|
|
|
type FetchLike = (url: string, init?: RequestInit) => Promise<Response>;
|
|
|
|
/** Sane bound for a wrong host that answers slowly — uploads set their own. */
|
|
const REQUEST_TIMEOUT_MS = 20_000;
|
|
|
|
function trimSlashes(value: string): string {
|
|
return value.replace(/^\/+|\/+$/g, '');
|
|
}
|
|
|
|
/** Folder segments, each URI-encoded; rejects `.`/`..` traversal segments. */
|
|
export function folderSegments(folder: string): string[] {
|
|
const segments = trimSlashes(folder.trim())
|
|
.split('/')
|
|
.filter((segment) => segment.length > 0);
|
|
if (segments.some((segment) => segment === '.' || segment === '..')) {
|
|
throw new Error('folder must not contain "." or ".." segments');
|
|
}
|
|
return segments;
|
|
}
|
|
|
|
/** The DAV base URL (without the folder), normalized without trailing slash. */
|
|
export function webdavBaseUrl(target: Pick<WebDavTarget, 'baseUrl' | 'username'>): string {
|
|
const base = target.baseUrl.replace(/\/+$/, '');
|
|
if (/remote\.php|\/dav\//i.test(base)) return base;
|
|
return `${base}/remote.php/dav/files/${encodeURIComponent(target.username)}`;
|
|
}
|
|
|
|
/** Absolute URL of the target folder (no trailing slash). */
|
|
export function webdavFolderUrl(target: WebDavTarget): string {
|
|
const segments = folderSegments(target.folder).map(encodeURIComponent);
|
|
return [webdavBaseUrl(target), ...segments].join('/');
|
|
}
|
|
|
|
/** Absolute URL of a file inside the target folder. */
|
|
export function webdavFileUrl(target: WebDavTarget, name: string): string {
|
|
if (name.includes('/')) throw new Error('file name must not contain "/"');
|
|
return `${webdavFolderUrl(target)}/${encodeURIComponent(name)}`;
|
|
}
|
|
|
|
export function basicAuthHeader(target: Pick<WebDavTarget, 'username' | 'password'>): string {
|
|
const credentials = `${target.username}:${target.password}`;
|
|
return `Basic ${Buffer.from(credentials, 'utf8').toString('base64')}`;
|
|
}
|
|
|
|
function decodeXmlEntities(value: string): string {
|
|
return value
|
|
.replace(/</g, '<')
|
|
.replace(/>/g, '>')
|
|
.replace(/"/g, '"')
|
|
.replace(/'/g, "'")
|
|
.replace(/&#(\d+);/g, (_, code: string) => String.fromCodePoint(Number(code)))
|
|
.replace(/&/g, '&');
|
|
}
|
|
|
|
/** First text content of `<prefix:tag>` inside `block`, prefix-agnostic. */
|
|
function xmlText(block: string, tag: string): string | null {
|
|
const match = new RegExp(`<(?:[A-Za-z0-9-]+:)?${tag}[^>]*>([^<]*)</`, 'i').exec(block);
|
|
return match ? decodeXmlEntities(match[1]!.trim()) : null;
|
|
}
|
|
|
|
/**
|
|
* Parses a PROPFIND multistatus body into entries. Deliberately tolerant:
|
|
* namespace prefixes vary between servers (`d:`, `D:`, none), and unknown
|
|
* properties are ignored. The entry for the requested collection itself
|
|
* (whose href equals the request path) is excluded.
|
|
*/
|
|
export function parsePropfind(xml: string, requestPath: string): WebDavEntry[] {
|
|
const entries: WebDavEntry[] = [];
|
|
const responseBlocks =
|
|
xml.match(/<(?:[A-Za-z0-9-]+:)?response[\s>][\s\S]*?<\/(?:[A-Za-z0-9-]+:)?response>/gi) ?? [];
|
|
const normalizedRequest = trimSlashes(decodeURIComponent(requestPath));
|
|
for (const block of responseBlocks) {
|
|
const href = xmlText(block, 'href');
|
|
if (!href) continue;
|
|
const path = trimSlashes(decodeURIComponent(href));
|
|
if (path === normalizedRequest) continue;
|
|
const name = path.split('/').pop() ?? '';
|
|
if (!name) continue;
|
|
const lengthText = xmlText(block, 'getcontentlength');
|
|
const sizeBytes = lengthText !== null && /^\d+$/.test(lengthText) ? Number(lengthText) : null;
|
|
entries.push({
|
|
name,
|
|
isCollection: /<(?:[A-Za-z0-9-]+:)?collection\b/i.test(block),
|
|
sizeBytes,
|
|
lastModified: xmlText(block, 'getlastmodified'),
|
|
});
|
|
}
|
|
return entries;
|
|
}
|
|
|
|
async function davRequest(
|
|
fetchLike: FetchLike,
|
|
target: WebDavTarget,
|
|
url: string,
|
|
init: RequestInit & { timeoutMs?: number },
|
|
): Promise<Response> {
|
|
const { timeoutMs, ...rest } = init;
|
|
return fetchLike(url, {
|
|
...rest,
|
|
headers: {
|
|
Authorization: basicAuthHeader(target),
|
|
...(rest.headers ?? {}),
|
|
},
|
|
signal: AbortSignal.timeout(timeoutMs ?? REQUEST_TIMEOUT_MS),
|
|
});
|
|
}
|
|
|
|
function describeFailure(action: string, response: Response): string {
|
|
const auth = response.status === 401 || response.status === 403;
|
|
return auth
|
|
? `${action}: authentication failed (HTTP ${response.status}) — check username and app password`
|
|
: `${action}: HTTP ${response.status} ${response.statusText}`.trim();
|
|
}
|
|
|
|
function describeError(action: string, error: unknown): string {
|
|
const message = error instanceof Error ? error.message : String(error);
|
|
return `${action}: ${message}`;
|
|
}
|
|
|
|
/**
|
|
* Verifies the target is usable: credentials accepted and the folder exists,
|
|
* creating missing folder segments via MKCOL on the way (like the setup
|
|
* wizard's SMTP test, nothing else is touched).
|
|
*/
|
|
export async function webdavCheck(
|
|
target: WebDavTarget,
|
|
fetchLike: FetchLike = fetch,
|
|
): Promise<WebDavResult<void>> {
|
|
let url = webdavBaseUrl(target);
|
|
try {
|
|
const probe = await davRequest(fetchLike, target, url, {
|
|
method: 'PROPFIND',
|
|
headers: { Depth: '0' },
|
|
});
|
|
if (!probe.ok) return { ok: false, error: describeFailure('connect', probe) };
|
|
|
|
for (const segment of folderSegments(target.folder)) {
|
|
url = `${url}/${encodeURIComponent(segment)}`;
|
|
const exists = await davRequest(fetchLike, target, url, {
|
|
method: 'PROPFIND',
|
|
headers: { Depth: '0' },
|
|
});
|
|
if (exists.ok) continue;
|
|
if (exists.status !== 404)
|
|
return { ok: false, error: describeFailure('check folder', exists) };
|
|
const created = await davRequest(fetchLike, target, url, { method: 'MKCOL' });
|
|
if (!created.ok) return { ok: false, error: describeFailure('create folder', created) };
|
|
}
|
|
return { ok: true, value: undefined };
|
|
} catch (error) {
|
|
return { ok: false, error: describeError('connect', error) };
|
|
}
|
|
}
|
|
|
|
/** Lists the target folder (depth 1), excluding sub-collections' contents. */
|
|
export async function webdavList(
|
|
target: WebDavTarget,
|
|
fetchLike: FetchLike = fetch,
|
|
): Promise<WebDavResult<WebDavEntry[]>> {
|
|
const url = webdavFolderUrl(target);
|
|
try {
|
|
const response = await davRequest(fetchLike, target, url, {
|
|
method: 'PROPFIND',
|
|
headers: { Depth: '1' },
|
|
});
|
|
if (!response.ok) return { ok: false, error: describeFailure('list folder', response) };
|
|
const requestPath = new URL(url).pathname;
|
|
return { ok: true, value: parsePropfind(await response.text(), requestPath) };
|
|
} catch (error) {
|
|
return { ok: false, error: describeError('list folder', error) };
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Uploads a file into the target folder. The body is a Buffer or a byte
|
|
* stream (pass `contentLength` for streams so the server can reject early
|
|
* on quota). No practical timeout — bundles can be large and slow links are
|
|
* fine; the caller's run wraps the whole upload.
|
|
*/
|
|
export async function webdavPut(
|
|
target: WebDavTarget,
|
|
name: string,
|
|
body: Buffer | ReadableStream<Uint8Array>,
|
|
options: { contentLength?: number; timeoutMs?: number } = {},
|
|
fetchLike: FetchLike = fetch,
|
|
): Promise<WebDavResult<void>> {
|
|
try {
|
|
const response = await davRequest(fetchLike, target, webdavFileUrl(target, name), {
|
|
method: 'PUT',
|
|
body: body as RequestInit['body'],
|
|
headers: options.contentLength ? { 'Content-Length': String(options.contentLength) } : {},
|
|
timeoutMs: options.timeoutMs ?? 6 * 60 * 60 * 1000,
|
|
// Node's fetch requires half-duplex for streamed request bodies.
|
|
...(body instanceof Buffer ? {} : { duplex: 'half' as const }),
|
|
} as RequestInit & { timeoutMs?: number });
|
|
if (!response.ok) return { ok: false, error: describeFailure(`upload ${name}`, response) };
|
|
return { ok: true, value: undefined };
|
|
} catch (error) {
|
|
return { ok: false, error: describeError(`upload ${name}`, error) };
|
|
}
|
|
}
|
|
|
|
/** Fetches a file; the caller streams `response.body` to disk. */
|
|
export async function webdavGet(
|
|
target: WebDavTarget,
|
|
name: string,
|
|
fetchLike: FetchLike = fetch,
|
|
): Promise<WebDavResult<Response>> {
|
|
try {
|
|
const response = await davRequest(fetchLike, target, webdavFileUrl(target, name), {
|
|
method: 'GET',
|
|
timeoutMs: 6 * 60 * 60 * 1000,
|
|
});
|
|
if (!response.ok) return { ok: false, error: describeFailure(`download ${name}`, response) };
|
|
return { ok: true, value: response };
|
|
} catch (error) {
|
|
return { ok: false, error: describeError(`download ${name}`, error) };
|
|
}
|
|
}
|
|
|
|
export async function webdavDelete(
|
|
target: WebDavTarget,
|
|
name: string,
|
|
fetchLike: FetchLike = fetch,
|
|
): Promise<WebDavResult<void>> {
|
|
try {
|
|
const response = await davRequest(fetchLike, target, webdavFileUrl(target, name), {
|
|
method: 'DELETE',
|
|
});
|
|
// 404 counts as deleted — prune must be idempotent across retries.
|
|
if (!response.ok && response.status !== 404) {
|
|
return { ok: false, error: describeFailure(`delete ${name}`, response) };
|
|
}
|
|
return { ok: true, value: undefined };
|
|
} catch (error) {
|
|
return { ok: false, error: describeError(`delete ${name}`, error) };
|
|
}
|
|
}
|