import { INestApplication } from '@nestjs/common'; import { PrismaClient } from '@prisma/client'; import { unzipSync } from 'fflate'; import request from 'supertest'; import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'; import { AuthTokensService } from '../auth/auth-tokens.service'; import { FilesService } from '../files/files.service'; import { createTestApp, sessionCookieOf } from '../testing/test-app'; import { createTestPrisma, grantOwnerAdmin, hasTestDb, uniqueSuffix } from '../testing/test-db'; import { UsersService } from '../users/users.service'; import { ConversionWorker } from './conversion-worker.service'; import { DataExportService } from './data-export.service'; /** GDPR data export (issue #68): a signed-in account assembles its own profile, * memberships, and the Markdown of every pond it owns into one ZIP — with a * download link that expires — and never another user's content. */ const PNG_BASE64 = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAAC0lEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg=='; function zipEntries(buffer: Buffer): Record { return unzipSync(new Uint8Array(buffer)); } function textOf(entries: Record, name: string): string { return Buffer.from(entries[name]!).toString('utf8'); } async function downloadZip(server: unknown, path: string, cookie: string): Promise { const res = await request(server as never) .get(path) .set('Cookie', cookie) .buffer(true) .parse((r, cb) => { const chunks: Buffer[] = []; r.on('data', (c: Buffer) => chunks.push(c)); r.on('end', () => cb(null, Buffer.concat(chunks))); }) .expect(200); return res.body as Buffer; } describe.skipIf(!hasTestDb)('data export (e2e, issue #68)', () => { let app: INestApplication; let prisma: PrismaClient; let worker: ConversionWorker; let files: FilesService; const suffix = uniqueSuffix(); const password = 'exportiere alle meine daten 1'; let ownerId: string; let ownerCookie: string; let personalSlug: string; let ownedSharedSlug: string; let foreignSlug: string; const api = () => request(app.getHttpServer()); async function login(username: string): Promise { const res = await api() .post('/api/v1/auth/login') .send({ usernameOrEmail: username, password }) .expect(200); return sessionCookieOf(res); } async function createUser(username: string): Promise { const user = await app.get(UsersService).createUser({ username, email: `${username}@example.org`, displayName: username, password, locale: 'en', }); // Verify via the endpoint (not a direct flag flip) so the personal pond and // its owner-admin grant are created exactly as in production (#52). const token = await app.get(AuthTokensService).issue(user.id, 'EMAIL_VERIFICATION', 600); await api().post('/api/v1/auth/verify-email').send({ token }).expect(204); return user.id; } async function seedPage(pondId: string, title: string, markdown: string): Promise { await prisma.page.create({ data: { pondId, title, slug: title.toLowerCase().replace(/[^a-z0-9]+/g, '-'), ydocState: new Uint8Array(), sortKey: title, createdBy: ownerId, contentCache: { create: { plainText: markdown, markdown, html: '', outline: [] } }, }, }); } beforeAll(async () => { prisma = createTestPrisma(); await prisma.rateLimit.deleteMany({}); app = await createTestApp(); worker = app.get(ConversionWorker); files = app.get(FilesService); ownerId = await createUser(`odette-owner-${suffix}`); ownerCookie = await login(`odette-owner-${suffix}`); const personal = await prisma.pond.findFirstOrThrow({ where: { ownerId, type: 'PERSONAL' }, }); personalSlug = personal.slug; // A shared pond the owner created — its content belongs in the export. const shared = await prisma.pond.create({ data: { slug: `owned-shared-${suffix}`, name: 'Owned Shared', type: 'SHARED', ownerId, usage: { create: {} }, }, }); ownedSharedSlug = shared.slug; await grantOwnerAdmin(prisma, shared.id, ownerId); // A stored image referenced from a personal-pond page (media inclusion). const image = await files.upload({ id: ownerId } as never, personal.id, { buffer: Buffer.from(PNG_BASE64, 'base64'), size: 70, originalname: 'dot.png', }); await seedPage(personal.id, 'Diary', `# Diary\n\nMine. ![dot](${image.id})`); await seedPage(shared.id, 'Shared Note', '# Shared Note\n\nAlso mine.'); // A foreign user's pond the owner is only a *reader* of: it must show up in // memberships, but its content must never appear in the export. const foreignId = await createUser(`fred-foreign-${suffix}`); const foreign = await prisma.pond.create({ data: { slug: `foreign-secret-${suffix}`, name: 'Foreign Secret', type: 'SHARED', ownerId: foreignId, usage: { create: {} }, }, }); foreignSlug = foreign.slug; await prisma.page.create({ data: { pondId: foreign.id, title: 'Foreign Page', slug: 'foreign-page', ydocState: new Uint8Array(), sortKey: 'Foreign Page', createdBy: foreignId, contentCache: { create: { plainText: 'TOP SECRET', markdown: 'TOP SECRET', html: '', outline: [] }, }, }, }); await prisma.roleGrant.create({ data: { pondId: foreign.id, subjectType: 'USER', subjectId: ownerId, role: 'READER', scopeType: 'POND', effect: 'ALLOW', createdBy: foreignId, }, }); }); afterAll(async () => { await prisma.conversionJob.deleteMany({ where: { owner: { username: { contains: suffix } } } }); const where = { pond: { owner: { username: { contains: suffix } } } }; await prisma.attachment.deleteMany({ where }); await prisma.roleGrant.deleteMany({ where: { pond: { owner: { username: { contains: suffix } } } }, }); await prisma.page.deleteMany({ where }); await prisma.pond.deleteMany({ where: { owner: { username: { contains: suffix } } } }); await prisma.rateLimit.deleteMany({}); await prisma.user.deleteMany({ where: { username: { contains: suffix } } }); await prisma.$disconnect(); await app.close(); }); // Each test starts from a clean rate-limit table so the per-account export // limit (exercised deliberately in one test) never trips the others. beforeEach(async () => { await prisma.rateLimit.deleteMany({}); }); it('exports the caller profile, memberships, and owned pond content only', async () => { const enqueued = await api() .post('/api/v1/users/me/data-export') .set('Cookie', ownerCookie) .expect(201); expect(enqueued.body.kind).toBe('data_export'); expect(enqueued.body.status).toBe('pending'); await worker.drain(); const done = await api() .get(`/api/v1/jobs/${enqueued.body.id}`) .set('Cookie', ownerCookie) .expect(200); expect(done.body.status).toBe('succeeded'); // The download link carries an expiry (a future timestamp). expect(done.body.expiresAt).toBeTruthy(); expect(new Date(done.body.expiresAt).getTime()).toBeGreaterThan(Date.now()); const zip = await downloadZip( app.getHttpServer(), `/api/v1/jobs/${enqueued.body.id}/result`, ownerCookie, ); const entries = zipEntries(zip); const names = Object.keys(entries); // Profile: the caller's own account fields. const profile = JSON.parse(textOf(entries, 'profile.json')); expect(profile.username).toBe(`odette-owner-${suffix}`); expect(profile.email).toBe(`odette-owner-${suffix}@example.org`); // Personal + owned-shared pond content is present, media included. expect(names).toContain(`ponds/${personalSlug}/diary.md`); expect(names.some((n) => n.startsWith(`ponds/${personalSlug}/media/`))).toBe(true); expect(names).toContain(`ponds/${ownedSharedSlug}/shared-note.md`); // Memberships list the foreign pond the owner reads… const memberships = JSON.parse(textOf(entries, 'memberships.json')) as { pondSlug: string; }[]; expect(memberships.map((m) => m.pondSlug)).toContain(foreignSlug); // …but none of the foreign pond's content appears anywhere in the archive. expect(names.some((n) => n.startsWith(`ponds/${foreignSlug}/`))).toBe(false); for (const name of names) { if (name.endsWith('.md')) expect(textOf(entries, name)).not.toContain('TOP SECRET'); } }); it('404s the download once the link has expired', async () => { const enqueued = await api() .post('/api/v1/users/me/data-export') .set('Cookie', ownerCookie) .expect(201); await worker.drain(); // Backdate the expiry to simulate an elapsed link. await prisma.conversionJob.update({ where: { id: enqueued.body.id }, data: { expiresAt: new Date(Date.now() - 1000) }, }); await api() .get(`/api/v1/jobs/${enqueued.body.id}/result`) .set('Cookie', ownerCookie) .expect(404); }); it('purges the stored bytes of an expired export', async () => { const enqueued = await api() .post('/api/v1/users/me/data-export') .set('Cookie', ownerCookie) .expect(201); await worker.drain(); await prisma.conversionJob.update({ where: { id: enqueued.body.id }, data: { expiresAt: new Date(Date.now() - 1000) }, }); const purged = await app.get(DataExportService).purgeExpired(); expect(purged).toBeGreaterThanOrEqual(1); const row = await prisma.conversionJob.findUniqueOrThrow({ where: { id: enqueued.body.id } }); expect(row.result).toBeNull(); }); it('rate-limits repeated requests from the same account', async () => { await createUser(`irene-idle-${suffix}`); const cookie = await login(`irene-idle-${suffix}`); // The limit is a few requests per window; the run past it is rejected. for (let i = 0; i < 3; i += 1) { await api().post('/api/v1/users/me/data-export').set('Cookie', cookie).expect(201); } await api().post('/api/v1/users/me/data-export').set('Cookie', cookie).expect(429); }); it('keeps a data-export job owner-scoped (a foreign user cannot poll it)', async () => { const enqueued = await api() .post('/api/v1/users/me/data-export') .set('Cookie', ownerCookie) .expect(201); await createUser(`sam-stranger-${suffix}`); const strangerCookie = await login(`sam-stranger-${suffix}`); await api().get(`/api/v1/jobs/${enqueued.body.id}`).set('Cookie', strangerCookie).expect(404); }); });