docs(s31): close 1 review MAJOR — vector store recorded as an unwired authoring primitive

This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 12:54:28 +02:00
commit da8779fc7f
2 changed files with 30 additions and 10 deletions

View file

@ -146,11 +146,20 @@ is opt-in (`--semantic-retrieval`) rather than the default. If you wire one in,
`semretrieval` currently imports no network module at all, and a guard test asserts exactly that;
a real client belongs behind the `Embedder` protocol in *your* module, not inside this one.
Optional persistence: `save_vector_store(dir, verdicts, embedder)` / `load_vector_store(dir)`
write a byte-deterministic `vectors.npy` + `vectors.jsonl` pair (sorted by verdict id, atomic
replace). It is a **rebuildable cache, never authoritative** — a missing store loads as `None`
and the caller degrades to structural ranking, while a row/line mismatch raises rather than
silently mis-ranking. `*.npy` is gitignored.
**Optional persistence — an unwired authoring primitive.** `save_vector_store(dir, verdicts,
embedder)` / `load_vector_store(dir)` write and read a byte-deterministic `vectors.npy` +
`vectors.jsonl` pair (sorted by verdict id, atomic replace). A missing store loads as `None`; a
row/line mismatch raises rather than silently mis-ranking. `*.npy` is gitignored.
They have **no caller in `src/`**, and ranking never reads a persisted matrix: `HybridRanker.rank`
re-invokes the embedder for every candidate on every call, so a persisted store would be bypassed
even if one existed. They are offered to extenders, on the same footing as `write_verdict`,
`promote_verdict` and `build_mcp_server` — public primitives the core deliberately does not wire
into `run_project`. Wiring the cache in only becomes worth anything once the ranker is changed to
consult it, which is a redesign rather than a hookup.
*Known limitation:* the empty-store branch hardcodes `EMBED_DIM`, so a third-party `Embedder` of a
different dimension writes a shape-inconsistent empty store. Stated, not fixed.
## Bevisst ikke bygget (90 %-kuttlista)

View file

@ -286,11 +286,20 @@ class HybridRanker:
return sorted(candidates, key=lambda v: (-round(score(v), 9), v.id))[:k]
# --- Vector store: a rebuildable cache, never authoritative -----------------------------------
# --- Vector store: an UNWIRED authoring primitive, offered to extenders -------------------------
# Exactly two files. The `.npy` holds the matrix; the `.jsonl` sidecar holds the row->verdict-id
# mapping, one object per line in the repo's deterministic on-disk idiom (`outbox._dump`). There
# is deliberately no third `meta.json`: the numpy version that fixes the `.npy` header is already
# pinned in `uv.lock`, so a provenance file would only be a second place to drift.
#
# NOTHING IN `src/` CALLS THESE, and that is the chosen disposition — the same one the repo already
# gives `write_verdict`, `promote_verdict` and `build_mcp_server`: a public primitive that is NOT
# wired into `run_project`. Wiring it would buy nothing today, because `HybridRanker.rank` re-embeds
# every candidate on every call and would bypass a persisted matrix regardless; making the cache
# load-bearing means redesigning the ranker to consult it, which is a separate piece of work.
#
# Known limitation: the empty-store branch below hardcodes `EMBED_DIM`, so a third-party `Embedder`
# of a different dimension writes a shape-inconsistent empty store. Recorded, not fixed.
_VECTORS_NPY = "vectors.npy"
_VECTORS_JSONL = "vectors.jsonl"
@ -337,10 +346,12 @@ def save_vector_store(
def load_vector_store(directory: str | Path) -> tuple[list[str], np.ndarray] | None:
"""Load the store as ``(ids, matrix)``, or ``None`` when it has not been built.
Absence is tolerated because the store is a rebuildable cache a caller without one simply
degrades to structural ranking. A row/line MISMATCH is not tolerated: it would silently map
every vector to the wrong verdict id and mis-rank without any error, so it raises
``ValueError`` (the fail-fast posture of ``ledger.SavingsLedger.load``)."""
Absence is tolerated because the store is a rebuildable cache. A row/line MISMATCH is not
tolerated: it would silently map every vector to the wrong verdict id and mis-rank without any
error, so it raises ``ValueError`` (the fail-fast posture of ``ledger.SavingsLedger.load``).
Unwired: nothing in ``src/`` calls this. See the section note above the ranking path never
consults a persisted matrix, so ``None`` is not a degradation from anything."""
path = Path(directory)
npy_path, jsonl_path = path / _VECTORS_NPY, path / _VECTORS_JSONL
if not npy_path.is_file() or not jsonl_path.is_file():