docs(s31): close 1 review MAJOR — vector store recorded as an unwired authoring primitive
This commit is contained in:
parent
fb4c593924
commit
da8779fc7f
2 changed files with 30 additions and 10 deletions
|
|
@ -146,11 +146,20 @@ is opt-in (`--semantic-retrieval`) rather than the default. If you wire one in,
|
|||
`semretrieval` currently imports no network module at all, and a guard test asserts exactly that;
|
||||
a real client belongs behind the `Embedder` protocol in *your* module, not inside this one.
|
||||
|
||||
Optional persistence: `save_vector_store(dir, verdicts, embedder)` / `load_vector_store(dir)`
|
||||
write a byte-deterministic `vectors.npy` + `vectors.jsonl` pair (sorted by verdict id, atomic
|
||||
replace). It is a **rebuildable cache, never authoritative** — a missing store loads as `None`
|
||||
and the caller degrades to structural ranking, while a row/line mismatch raises rather than
|
||||
silently mis-ranking. `*.npy` is gitignored.
|
||||
**Optional persistence — an unwired authoring primitive.** `save_vector_store(dir, verdicts,
|
||||
embedder)` / `load_vector_store(dir)` write and read a byte-deterministic `vectors.npy` +
|
||||
`vectors.jsonl` pair (sorted by verdict id, atomic replace). A missing store loads as `None`; a
|
||||
row/line mismatch raises rather than silently mis-ranking. `*.npy` is gitignored.
|
||||
|
||||
They have **no caller in `src/`**, and ranking never reads a persisted matrix: `HybridRanker.rank`
|
||||
re-invokes the embedder for every candidate on every call, so a persisted store would be bypassed
|
||||
even if one existed. They are offered to extenders, on the same footing as `write_verdict`,
|
||||
`promote_verdict` and `build_mcp_server` — public primitives the core deliberately does not wire
|
||||
into `run_project`. Wiring the cache in only becomes worth anything once the ranker is changed to
|
||||
consult it, which is a redesign rather than a hookup.
|
||||
|
||||
*Known limitation:* the empty-store branch hardcodes `EMBED_DIM`, so a third-party `Embedder` of a
|
||||
different dimension writes a shape-inconsistent empty store. Stated, not fixed.
|
||||
|
||||
## Bevisst ikke bygget (90 %-kuttlista)
|
||||
|
||||
|
|
|
|||
|
|
@ -286,11 +286,20 @@ class HybridRanker:
|
|||
return sorted(candidates, key=lambda v: (-round(score(v), 9), v.id))[:k]
|
||||
|
||||
|
||||
# --- Vector store: a rebuildable cache, never authoritative -----------------------------------
|
||||
# --- Vector store: an UNWIRED authoring primitive, offered to extenders -------------------------
|
||||
# Exactly two files. The `.npy` holds the matrix; the `.jsonl` sidecar holds the row->verdict-id
|
||||
# mapping, one object per line in the repo's deterministic on-disk idiom (`outbox._dump`). There
|
||||
# is deliberately no third `meta.json`: the numpy version that fixes the `.npy` header is already
|
||||
# pinned in `uv.lock`, so a provenance file would only be a second place to drift.
|
||||
#
|
||||
# NOTHING IN `src/` CALLS THESE, and that is the chosen disposition — the same one the repo already
|
||||
# gives `write_verdict`, `promote_verdict` and `build_mcp_server`: a public primitive that is NOT
|
||||
# wired into `run_project`. Wiring it would buy nothing today, because `HybridRanker.rank` re-embeds
|
||||
# every candidate on every call and would bypass a persisted matrix regardless; making the cache
|
||||
# load-bearing means redesigning the ranker to consult it, which is a separate piece of work.
|
||||
#
|
||||
# Known limitation: the empty-store branch below hardcodes `EMBED_DIM`, so a third-party `Embedder`
|
||||
# of a different dimension writes a shape-inconsistent empty store. Recorded, not fixed.
|
||||
|
||||
_VECTORS_NPY = "vectors.npy"
|
||||
_VECTORS_JSONL = "vectors.jsonl"
|
||||
|
|
@ -337,10 +346,12 @@ def save_vector_store(
|
|||
def load_vector_store(directory: str | Path) -> tuple[list[str], np.ndarray] | None:
|
||||
"""Load the store as ``(ids, matrix)``, or ``None`` when it has not been built.
|
||||
|
||||
Absence is tolerated because the store is a rebuildable cache — a caller without one simply
|
||||
degrades to structural ranking. A row/line MISMATCH is not tolerated: it would silently map
|
||||
every vector to the wrong verdict id and mis-rank without any error, so it raises
|
||||
``ValueError`` (the fail-fast posture of ``ledger.SavingsLedger.load``)."""
|
||||
Absence is tolerated because the store is a rebuildable cache. A row/line MISMATCH is not
|
||||
tolerated: it would silently map every vector to the wrong verdict id and mis-rank without any
|
||||
error, so it raises ``ValueError`` (the fail-fast posture of ``ledger.SavingsLedger.load``).
|
||||
|
||||
Unwired: nothing in ``src/`` calls this. See the section note above — the ranking path never
|
||||
consults a persisted matrix, so ``None`` is not a degradation from anything."""
|
||||
path = Path(directory)
|
||||
npy_path, jsonl_path = path / _VECTORS_NPY, path / _VECTORS_JSONL
|
||||
if not npy_path.is_file() or not jsonl_path.is_file():
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue