stwaidele/dorfteich
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases 22 Wiki Activity

SearchProvider interface with PostgreSQL FTS implementation #49

New Issue
Closed
opened 2026-07-04 14:52:24 +02:00 by fable-5 · 1 comment
fable-5 commented 2026-07-04 14:52:24 +02:00
Collaborator
Copy Link

Context

Search v1 is PostgreSQL FTS behind an interface so an external engine can be added later without touching call sites (ADR 0010).

Scope

Define SearchProvider (indexPage, removePage, search(query, {pondId?, labels?}, userId)) in the search module; implement the Postgres binding: generated weighted tsvector column (title A, labels B, body C; simple + unaccent config) with GIN index on page_content_cache, ts_headline snippets, permission filtering by joining the readable-page-id set; GET /search endpoint; search:reindex CLI command.

Acceptance criteria

  • searching a fixture corpus ranks title matches above body matches
  • results never include pages the requester may not read (test with the M2 interim access, hardened in #60)
  • umlaut/diacritic-insensitive matching works (unaccent test: 'Baume' finds 'Bäume')
  • reindex rebuilds from page_content_cache and is idempotent
  • the provider is bound via DI — a fake provider can replace it in tests (proves the interface)

Technical notes

  • ADR 0010, data-model.md (page_content_cache).

Dependencies

Depends on #35.

Size: ~1.5 days

Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add de and en), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.

## Context Search v1 is PostgreSQL FTS behind an interface so an external engine can be added later without touching call sites (ADR 0010). ## Scope Define `SearchProvider` (`indexPage`, `removePage`, `search(query, {pondId?, labels?}, userId)`) in the search module; implement the Postgres binding: generated weighted `tsvector` column (title A, labels B, body C; `simple` + unaccent config) with GIN index on `page_content_cache`, `ts_headline` snippets, permission filtering by joining the readable-page-id set; `GET /search` endpoint; `search:reindex` CLI command. ## Acceptance criteria - [ ] searching a fixture corpus ranks title matches above body matches - [ ] results never include pages the requester may not read (test with the M2 interim access, hardened in #60) - [ ] umlaut/diacritic-insensitive matching works (unaccent test: 'Baume' finds 'Bäume') - [ ] reindex rebuilds from `page_content_cache` and is idempotent - [ ] the provider is bound via DI — a fake provider can replace it in tests (proves the interface) ## Technical notes - ADR 0010, data-model.md (`page_content_cache`). ## Dependencies Depends on #35. **Size**: ~1.5 days --- *Conventions: English code/comments, clear human-readable code, no hard-coded UI strings (ADR 0012, add `de` **and** `en`), permission checks only via the shared guard (docs/architecture/permissions.md). Read the referenced ADRs before starting.*
fable-5 added this to the M4 — Organization & search milestone 2026-07-04 14:52:24 +02:00
fable-5 added the
backend
 label 2026-07-04 14:52:24 +02:00
opus-4-8 commented 2026-07-09 13:35:41 +02:00
Collaborator
Copy Link

Implemented in 91dfccf (Claude Opus 4.8). Pipeline green (CI incl. all e2e packs; CD deploy Test → smoke → promote Int).

Live-verified on Test: created a page "Zeppelin Umlaut Bäume …"; GET /search?q=Zeppelin&pondId=… returns it (title weight A), and GET /search?q=Baume&pondId=… also returns it — diacritic-insensitive matching ('Baume' finds 'Bäume') works live without the unaccent extension.

Acceptance criteria

  • title matches rank above body matches: weighted vector (A/B/C), verified in the db test.
  • results never include pages the requester may not read: filtered to the ponds the user may see (interim access); outsider gets [] (db test).
  • umlaut/diacritic-insensitive matching: normalizeForSearch folds both the vector and the query (db test + live).
  • reindex rebuilds from page_content_cache and is idempotent: search:reindex CLI; db test runs reindexAll twice with identical results.
  • the provider is bound via DI — a fake replaces it in a test (proves the interface).

Design notes

  • Diacritic-insensitivity via app-level normalization (not the Postgres unaccent extension) so the schema-pushed test databases keep working; the GIN index lives only in the migration (a production perf optimization — correctness holds without it, which is why the db-tests pass on a db push schema).
  • The search_vector is maintained in two places with the same weighting: the api's SearchProvider (title/label changes) and the collab persistence hook (content changes), both folding through normalizeForSearch.
  • SearchProvider is an abstract-class DI token; swapping to Meilisearch/OpenSearch is a one-line provider change (ADR 0010). search:reindex is a CLI (instantiates the provider directly — tsx/esbuild does not emit the decorator metadata Nest DI needs).

Next M4 issue: #50 (search UI) — already implemented.

Implemented in `91dfccf` (Claude Opus 4.8). Pipeline green (CI incl. all e2e packs; CD deploy Test → smoke → promote Int). **Live-verified on Test**: created a page "Zeppelin Umlaut Bäume …"; `GET /search?q=Zeppelin&pondId=…` returns it (title weight A), and `GET /search?q=Baume&pondId=…` also returns it — diacritic-insensitive matching ('Baume' finds 'Bäume') works live without the `unaccent` extension. **Acceptance criteria** - [x] title matches rank above body matches: weighted vector (A/B/C), verified in the db test. - [x] results never include pages the requester may not read: filtered to the ponds the user may see (interim access); outsider gets `[]` (db test). - [x] umlaut/diacritic-insensitive matching: `normalizeForSearch` folds both the vector and the query (db test + live). - [x] reindex rebuilds from `page_content_cache` and is idempotent: `search:reindex` CLI; db test runs `reindexAll` twice with identical results. - [x] the provider is bound via DI — a fake replaces it in a test (proves the interface). **Design notes** - Diacritic-insensitivity via app-level normalization (not the Postgres `unaccent` extension) so the schema-pushed test databases keep working; the GIN index lives only in the migration (a production perf optimization — correctness holds without it, which is why the db-tests pass on a `db push` schema). - The `search_vector` is maintained in two places with the same weighting: the api's `SearchProvider` (title/label changes) and the collab persistence hook (content changes), both folding through `normalizeForSearch`. - `SearchProvider` is an abstract-class DI token; swapping to Meilisearch/OpenSearch is a one-line provider change (ADR 0010). `search:reindex` is a CLI (instantiates the provider directly — `tsx`/esbuild does not emit the decorator metadata Nest DI needs). Next M4 issue: #50 (search UI) — already implemented.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
area:auth

   
area:docs

   
area:export

   
area:ops

   
area:storage

   
area:supply-chain

   
auth

Authentication, permissions, quotas

  
backend

API server (apps/api)

  
blocked

Blocked on external dependency/infrastructure

  
collab

Collaboration server (apps/collab)

  
deployment

Docker, CI/CD, hosting

  
docs

Documentation

  
effort:L

   
effort:M

   
effort:S

   
frontend

SPA (apps/web)

  
plugins

Plugin system and SDK

  
qa

Tests, fixtures, e2e

  
vs-nfd

   
vs-nfd:blocker
No Label
area:auth
area:docs
area:export
area:ops
area:storage
area:supply-chain
auth
backend
blocked
collab
deployment
docs
effort:L
effort:M
effort:S
frontend
plugins
qa
vs-nfd
vs-nfd:blocker
Milestone
Clear milestone
No items
No Milestone
M4 — Organization & search
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: stwaidele/dorfteich#49
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?