dorfteich/packages/shared/src/webdav.ts
Claude Fable 5 5cef359b8f
All checks were successful
CI / Lint, typecheck, test (push) Successful in 3m45s
CD / Build and push images (push) Successful in 3m49s
CI / Build container images (push) Has been skipped
CD / Deploy to Test (push) Successful in 9s
CD / Smoke tests against Test (push) Successful in 1m18s
CD / Promote to Int (push) Successful in 11s
CI / Auth e2e pack (push) Successful in 5m35s
CI / Import/export fidelity gate (push) Successful in 47s
Nextcloud backup target: admin-configured, manual + scheduled uploads, in-app restore (#103)
Off-host backups for every self-hoster, configured entirely in the admin
UI — supersedes the host-specific mirror plan behind #84.

shared:
- webdav.ts (new package entry like token-crypto): minimal WebDAV client
  with basic auth — PROPFIND (tolerant multistatus parser), MKCOL, PUT
  (streamed), GET, DELETE; Nextcloud DAV path derived from the plain
  server URL, explicit DAV bases pass through
- backup-status.ts: additive remote-upload status in status.json, the
  restore-status.json contract (running/succeeded/failed + staleness
  bound), the backup_command/backup_maintenance NOTIFY channels, and the
  one-bundle-per-set naming (dorfteich-backup-<id>.tar.gz)
- backup-set.ts moved here from apps/backup (api lists local sets)

backup sidecar:
- reads the backup.* instance settings directly from the database (admin
  changes apply next run; local retention row overrides the env) and the
  app password from the secret store
- after each successful set: bundle dump + files archive + manifest into
  ONE self-contained tar.gz, upload via WebDAV per schedule
  (off/daily/weekly; manual runs always upload), prune remote bundles —
  never the newest — and record the outcome in status.json; upload
  failures alert via a new backupUploadFailed mail (de+en)
- command listener on backup_command (run / restore) with a serial queue
  against the nightly timer
- restore orchestrator: restore-status.json → maintenance NOTIFY →
  grace → (remote: download + manifest-verify bundle) → terminate other
  DB connections → shared perform-restore path (same code as restore.sh)
  → final status + maintenance exit

api:
- MaintenanceGuard (global, registered before the setup gate): 503
  maintenance_mode while restore-status says running; health endpoints
  and the new public GET /backup/restore-status stay exempt; a stale
  running state (crashed sidecar) unblocks after 30 min
- MaintenanceStateService watches the file and restarts the api after a
  successful restore (fresh caches, migrate-on-start for older dumps);
  main.ts refuses to touch the database while a restore runs — a
  container restarting mid-restore must not race pg_restore with
  migrate deploy
- worker sweeps (conversion, mail outbox, scheduler) catch transient
  database failures instead of dying on an unhandled rejection — the
  restore's connection termination crashed the api in verification
- backup admin endpoints under /admin/system/backup: settings (live
  connection test before save, password write-only into the secret
  store), nextcloud/test, sets (local via the ro backups mount + remote
  via WebDAV), run + restore (type-to-confirm backstop, source
  validation) — commands travel as NOTIFY payloads; audit actions
  backup.settings_changed/run_triggered/restore_requested
- readyz: new warning-level backup_remote check while a target is
  configured (26 h daily / 170 h weekly bound)

collab:
- maintenance listener: on enter, persist + close every live session and
  refuse new connections until exit (failsafe timeout 30 min) — no
  in-memory document may write pre-restore content back afterwards

web:
- Admin → System backup section: status card with remote facts and a
  "Back up now" button, the Nextcloud settings form with test button,
  and the restore picker (local + remote sets, type-to-confirm)
- global maintenance screen: any 503 maintenance_mode flips the SPA to a
  status page polling the exempt endpoint, reloading when the instance
  returns

Verified end-to-end against a live stack (fresh DB, native api + sidecar,
fake WebDAV server): configure → test → manual backup → bundle upload →
readyz/sets/status surfaces → remote restore with maintenance gate,
marker rollback and api restart; suites: shared 21, backup 9, collab 11,
api 58 files green, lint + i18n:check + typecheck clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EwZ4jR4KFAPvpjWevfUGX1
2026-07-12 10:39:18 +02:00

274 lines
10 KiB
TypeScript

/**
* Minimal WebDAV client for the Nextcloud backup target (issue #103), shared
* by the api (connection test, remote set listing) and the backup sidecar
* (upload, prune, download). Plain `fetch` with basic auth — no heavy DAV
* dependency; the subset used here (PROPFIND/MKCOL/PUT/GET/DELETE) is stable
* across Nextcloud versions and generic WebDAV servers.
*
* Like `token-crypto`, this module is a separate package entry (not part of
* the barrel) because it is server-only.
*/
export interface WebDavTarget {
/**
* The Nextcloud base URL (e.g. `https://cloud.example.com`) — the DAV
* path `remote.php/dav/files/<username>` is derived. A URL that already
* contains `remote.php` or `/dav/` is used as the DAV base verbatim, so
* generic WebDAV servers work too.
*/
baseUrl: string;
username: string;
password: string;
/** Target folder under the DAV base, may contain `/` for nesting. */
folder: string;
}
export interface WebDavEntry {
/** Decoded file or collection name (last path segment). */
name: string;
isCollection: boolean;
sizeBytes: number | null;
lastModified: string | null;
}
export type WebDavResult<T> = { ok: true; value: T } | { ok: false; error: string };
type FetchLike = (url: string, init?: RequestInit) => Promise<Response>;
/** Sane bound for a wrong host that answers slowly — uploads set their own. */
const REQUEST_TIMEOUT_MS = 20_000;
function trimSlashes(value: string): string {
return value.replace(/^\/+|\/+$/g, '');
}
/** Folder segments, each URI-encoded; rejects `.`/`..` traversal segments. */
export function folderSegments(folder: string): string[] {
const segments = trimSlashes(folder.trim())
.split('/')
.filter((segment) => segment.length > 0);
if (segments.some((segment) => segment === '.' || segment === '..')) {
throw new Error('folder must not contain "." or ".." segments');
}
return segments;
}
/** The DAV base URL (without the folder), normalized without trailing slash. */
export function webdavBaseUrl(target: Pick<WebDavTarget, 'baseUrl' | 'username'>): string {
const base = target.baseUrl.replace(/\/+$/, '');
if (/remote\.php|\/dav\//i.test(base)) return base;
return `${base}/remote.php/dav/files/${encodeURIComponent(target.username)}`;
}
/** Absolute URL of the target folder (no trailing slash). */
export function webdavFolderUrl(target: WebDavTarget): string {
const segments = folderSegments(target.folder).map(encodeURIComponent);
return [webdavBaseUrl(target), ...segments].join('/');
}
/** Absolute URL of a file inside the target folder. */
export function webdavFileUrl(target: WebDavTarget, name: string): string {
if (name.includes('/')) throw new Error('file name must not contain "/"');
return `${webdavFolderUrl(target)}/${encodeURIComponent(name)}`;
}
export function basicAuthHeader(target: Pick<WebDavTarget, 'username' | 'password'>): string {
const credentials = `${target.username}:${target.password}`;
return `Basic ${Buffer.from(credentials, 'utf8').toString('base64')}`;
}
function decodeXmlEntities(value: string): string {
return value
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&apos;/g, "'")
.replace(/&#(\d+);/g, (_, code: string) => String.fromCodePoint(Number(code)))
.replace(/&amp;/g, '&');
}
/** First text content of `<prefix:tag>` inside `block`, prefix-agnostic. */
function xmlText(block: string, tag: string): string | null {
const match = new RegExp(`<(?:[A-Za-z0-9-]+:)?${tag}[^>]*>([^<]*)</`, 'i').exec(block);
return match ? decodeXmlEntities(match[1]!.trim()) : null;
}
/**
* Parses a PROPFIND multistatus body into entries. Deliberately tolerant:
* namespace prefixes vary between servers (`d:`, `D:`, none), and unknown
* properties are ignored. The entry for the requested collection itself
* (whose href equals the request path) is excluded.
*/
export function parsePropfind(xml: string, requestPath: string): WebDavEntry[] {
const entries: WebDavEntry[] = [];
const responseBlocks =
xml.match(/<(?:[A-Za-z0-9-]+:)?response[\s>][\s\S]*?<\/(?:[A-Za-z0-9-]+:)?response>/gi) ?? [];
const normalizedRequest = trimSlashes(decodeURIComponent(requestPath));
for (const block of responseBlocks) {
const href = xmlText(block, 'href');
if (!href) continue;
const path = trimSlashes(decodeURIComponent(href));
if (path === normalizedRequest) continue;
const name = path.split('/').pop() ?? '';
if (!name) continue;
const lengthText = xmlText(block, 'getcontentlength');
const sizeBytes = lengthText !== null && /^\d+$/.test(lengthText) ? Number(lengthText) : null;
entries.push({
name,
isCollection: /<(?:[A-Za-z0-9-]+:)?collection\b/i.test(block),
sizeBytes,
lastModified: xmlText(block, 'getlastmodified'),
});
}
return entries;
}
async function davRequest(
fetchLike: FetchLike,
target: WebDavTarget,
url: string,
init: RequestInit & { timeoutMs?: number },
): Promise<Response> {
const { timeoutMs, ...rest } = init;
return fetchLike(url, {
...rest,
headers: {
Authorization: basicAuthHeader(target),
...(rest.headers ?? {}),
},
signal: AbortSignal.timeout(timeoutMs ?? REQUEST_TIMEOUT_MS),
});
}
function describeFailure(action: string, response: Response): string {
const auth = response.status === 401 || response.status === 403;
return auth
? `${action}: authentication failed (HTTP ${response.status}) — check username and app password`
: `${action}: HTTP ${response.status} ${response.statusText}`.trim();
}
function describeError(action: string, error: unknown): string {
const message = error instanceof Error ? error.message : String(error);
return `${action}: ${message}`;
}
/**
* Verifies the target is usable: credentials accepted and the folder exists,
* creating missing folder segments via MKCOL on the way (like the setup
* wizard's SMTP test, nothing else is touched).
*/
export async function webdavCheck(
target: WebDavTarget,
fetchLike: FetchLike = fetch,
): Promise<WebDavResult<void>> {
let url = webdavBaseUrl(target);
try {
const probe = await davRequest(fetchLike, target, url, {
method: 'PROPFIND',
headers: { Depth: '0' },
});
if (!probe.ok) return { ok: false, error: describeFailure('connect', probe) };
for (const segment of folderSegments(target.folder)) {
url = `${url}/${encodeURIComponent(segment)}`;
const exists = await davRequest(fetchLike, target, url, {
method: 'PROPFIND',
headers: { Depth: '0' },
});
if (exists.ok) continue;
if (exists.status !== 404)
return { ok: false, error: describeFailure('check folder', exists) };
const created = await davRequest(fetchLike, target, url, { method: 'MKCOL' });
if (!created.ok) return { ok: false, error: describeFailure('create folder', created) };
}
return { ok: true, value: undefined };
} catch (error) {
return { ok: false, error: describeError('connect', error) };
}
}
/** Lists the target folder (depth 1), excluding sub-collections' contents. */
export async function webdavList(
target: WebDavTarget,
fetchLike: FetchLike = fetch,
): Promise<WebDavResult<WebDavEntry[]>> {
const url = webdavFolderUrl(target);
try {
const response = await davRequest(fetchLike, target, url, {
method: 'PROPFIND',
headers: { Depth: '1' },
});
if (!response.ok) return { ok: false, error: describeFailure('list folder', response) };
const requestPath = new URL(url).pathname;
return { ok: true, value: parsePropfind(await response.text(), requestPath) };
} catch (error) {
return { ok: false, error: describeError('list folder', error) };
}
}
/**
* Uploads a file into the target folder. The body is a Buffer or a byte
* stream (pass `contentLength` for streams so the server can reject early
* on quota). No practical timeout — bundles can be large and slow links are
* fine; the caller's run wraps the whole upload.
*/
export async function webdavPut(
target: WebDavTarget,
name: string,
body: Buffer | ReadableStream<Uint8Array>,
options: { contentLength?: number; timeoutMs?: number } = {},
fetchLike: FetchLike = fetch,
): Promise<WebDavResult<void>> {
try {
const response = await davRequest(fetchLike, target, webdavFileUrl(target, name), {
method: 'PUT',
body: body as RequestInit['body'],
headers: options.contentLength ? { 'Content-Length': String(options.contentLength) } : {},
timeoutMs: options.timeoutMs ?? 6 * 60 * 60 * 1000,
// Node's fetch requires half-duplex for streamed request bodies.
...(body instanceof Buffer ? {} : { duplex: 'half' as const }),
} as RequestInit & { timeoutMs?: number });
if (!response.ok) return { ok: false, error: describeFailure(`upload ${name}`, response) };
return { ok: true, value: undefined };
} catch (error) {
return { ok: false, error: describeError(`upload ${name}`, error) };
}
}
/** Fetches a file; the caller streams `response.body` to disk. */
export async function webdavGet(
target: WebDavTarget,
name: string,
fetchLike: FetchLike = fetch,
): Promise<WebDavResult<Response>> {
try {
const response = await davRequest(fetchLike, target, webdavFileUrl(target, name), {
method: 'GET',
timeoutMs: 6 * 60 * 60 * 1000,
});
if (!response.ok) return { ok: false, error: describeFailure(`download ${name}`, response) };
return { ok: true, value: response };
} catch (error) {
return { ok: false, error: describeError(`download ${name}`, error) };
}
}
export async function webdavDelete(
target: WebDavTarget,
name: string,
fetchLike: FetchLike = fetch,
): Promise<WebDavResult<void>> {
try {
const response = await davRequest(fetchLike, target, webdavFileUrl(target, name), {
method: 'DELETE',
});
// 404 counts as deleted — prune must be idempotent across retries.
if (!response.ok && response.status !== 404) {
return { ok: false, error: describeFailure(`delete ${name}`, response) };
}
return { ok: true, value: undefined };
} catch (error) {
return { ok: false, error: describeError(`delete ${name}`, error) };
}
}