fix(semretrieval): refuse a non-finite embedding instead of scoring it (kø-(l)/S3.1 MINOR)

`cosine`'s docstring claimed its guard was load-bearing because "a NaN reaching the
ranking sort key would corrupt ordering silently rather than failing loudly" — but the
guard tested `norm == 0.0` only, which a NaN or inf norm passes straight through. The
claim was prose, not behaviour.

Measured, not assumed: `cosine(unit, nan_vector)` AND `cosine(unit, inf_vector)` both
returned `nan`, and a NaN sort key made ranking INPUT-ORDER-DEPENDENT — six permutations
of the same three candidates produced four distinct orderings. That defeats the total
order `HybridRanker` documents ("`id` makes the result independent of input order").

Refuse rather than coerce, and deliberately NOT symmetric with the zero-norm branch: a
zero vector is a legitimate handled state (`FakeEmbedder` returns `np.zeros` by design),
whereas a non-finite component only ever means the INJECTED embedder is broken. Scoring
it `0.0` would launder that into "no semantic similarity" while ranking proceeded on a
forged signal — validation, never repair, mirroring `read_spend`.

Reachable via the documented `Embedder` extension point, not the shipped fake; scoped to
the norms (90% principle — a finite-normed dot-product overflow is not chased).

Also corrects `docs/extending.md`, which stated `SEMANTIC_WEIGHT_DEFAULT = 0.5` while the
code has said `0.25` since the weight was lowered.

625 -> 630 tests. Load-bearing MEASURED against the WHOLE suite, five mutations all red:
detach the guard entirely · coerce to 0.0 instead of raising · check only the first norm ·
drop "non-finite" from the message · (control) detach the zero-norm branch, which fails
ONLY the zero-norm test — the new guard does not mask the existing one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018V9vNBmxAmgJ2JMoHByiHS

docs/extending.md

@@ -194,7 +194,15 @@ protocols and the store delegates to them:
   with a `rank` method to replace ranking wholesale.
 - **`Embedder`** — `__call__(features) -> np.ndarray`. `HybridRanker(embedder, similarity, weight)`
   blends `weight * cosine + (1 - weight) * structural`; both terms live in `[0, 1]`, so `weight`
-  means what it reads as (`SEMANTIC_WEIGHT_DEFAULT = 0.5`).
+  means what it reads as (`SEMANTIC_WEIGHT_DEFAULT = 0.25`).
+
+  **Your vectors must be finite.** `cosine` raises `ValueError` on a NaN or infinite norm rather
+  than scoring it — a non-finite score compares `False` against everything, which leaves the
+  ranking in whatever order the input happened to arrive in and defeats the total order
+  `HybridRanker` otherwise guarantees. A zero vector is fine and scores `0.0`; the asymmetry is
+  deliberate, because zero is a state the shipped `FakeEmbedder` produces on purpose whereas
+  non-finite only ever means the embedder is broken. Coercing it to `0.0` would hide that as
+  "no semantic similarity" and let ranking proceed on a forged signal.
 
 ```python
 store.retriever = HybridRanker(MyEmbedder(), similarity, weight=0.3)

src/portfolio_optimiser/semretrieval.py

@@ -244,12 +244,31 @@ def load_embedder_config(path: str | Path) -> EmbedderConfig:
 
 
 def cosine(a: np.ndarray, b: np.ndarray) -> float:
-    """Cosine similarity, with a zero-norm guard returning ``0.0``.
+    """Cosine similarity: ``0.0`` for a zero norm, ``ValueError`` for a non-finite one.
 
     The guard is load-bearing: a NaN reaching the ranking sort key would corrupt ordering
-    silently rather than failing loudly."""
+    silently rather than failing loudly. Measured, not assumed — NaN compares False against
+    everything, so ``sorted`` leaves it where the input put it, and six permutations of the same
+    three candidates produced four distinct orderings. That defeats the total order
+    ``HybridRanker`` documents ("``id`` makes the result independent of input order").
+
+    The two branches are deliberately NOT symmetric. A zero vector is a legitimate, handled state
+    — ``FakeEmbedder`` returns ``np.zeros`` by design — so it earns a defined score. A non-finite
+    component only ever means the INJECTED embedder is broken (the shipped fake cannot emit one),
+    and coercing it to ``0.0`` would launder that into "no semantic similarity" while ranking
+    proceeded on a forged signal. Validation, never repair, matching ``read_spend``: reading
+    corrupt state as zero hands back a false answer in the caller's own units.
+
+    Scoped to the norms on purpose: a non-finite component always poisons its norm, which is the
+    reachable defect. A finite-normed pair whose dot product overflows is not chased here (90%
+    principle) — the seam this guards is a broken embedder, not float brinkmanship."""
     norm_a = float(np.linalg.norm(a))
     norm_b = float(np.linalg.norm(b))
+    if not np.isfinite(norm_a) or not np.isfinite(norm_b):
+        raise ValueError(
+            f"embedder produced a non-finite vector (norms: {norm_a}, {norm_b}); "
+            "a non-finite score corrupts the ranking order silently"
+        )
     if norm_a == 0.0 or norm_b == 0.0:
         return 0.0
     return float(np.dot(a, b) / (norm_a * norm_b))

tests/test_semretrieval.py

@@ -172,6 +172,25 @@ def test_cosine_of_zero_norm_is_zero() -> None:
     assert cosine(zeros, zeros) == 0.0
 
 
+@pytest.mark.parametrize("bad_value", [np.nan, np.inf, -np.inf])
+def test_cosine_refuses_a_non_finite_vector(bad_value: float) -> None:
+    """The zero-norm guard tests ``norm == 0.0``, which a NaN or inf norm passes straight through:
+    measured, ``cosine(unit, nan_vector)`` and ``cosine(unit, inf_vector)`` BOTH returned ``nan``.
+
+    Refuse rather than repair, and deliberately NOT symmetric with the zero-norm branch above: a
+    zero vector is a legitimate, handled state (``FakeEmbedder`` returns ``np.zeros`` by design),
+    whereas a non-finite component only ever means the injected embedder is broken. Coercing it to
+    ``0.0`` would launder that into "no semantic similarity" and let ranking proceed on a forged
+    signal — the same reasoning that makes ``read_spend`` raise on corrupt content instead of
+    reading it as zero."""
+    vec = FakeEmbedder()(_features())
+    bad = np.full(EMBED_DIM, bad_value, dtype="<f8")
+    with pytest.raises(ValueError, match="non-finite"):
+        cosine(vec, bad)
+    with pytest.raises(ValueError, match="non-finite"):
+        cosine(bad, vec)
+
+
 # --- SC1: the Retriever seam is additive — the DEFAULT ranking is byte-identical to today ---
 
 _QUERY = ProposalFeatures(

tests/test_semretrieval_loadbearing.py

@@ -594,3 +594,86 @@ def test_hybrid_preserves_structural_order_across_one_magnitude_step() -> None:
         f"{len(adverse)} structural orderings overturned by the semantic term at "
         f"weight={SEMANTIC_WEIGHT_DEFAULT}; first three: {adverse[:3]}"
     )
+
+
+# --- The non-finite guard: an injected embedder must not be able to corrupt the sort key --------
+#
+# ``cosine``'s docstring claims its guard is load-bearing because "a NaN reaching the ranking sort
+# key would corrupt ordering silently rather than failing loudly". The guard tested ``norm == 0.0``
+# only, which a NaN or inf norm passes straight through — so the claim was prose, not behaviour.
+# ``FakeEmbedder`` cannot emit a non-finite component, but ``Embedder`` is a documented CODE-LEVEL
+# extension point (``docs/extending.md``), and an injected client is exactly what this seam exists
+# to accept. That makes the reachable caller the injected one, NOT the shipped fake.
+
+
+class _NaNEmbedder:
+    """An injected embedder that has gone wrong on ONE input — the realistic failure, since a
+    client that returned NaN for everything would be caught by the first smoke test."""
+
+    def __init__(self, poison: str) -> None:
+        self._poison = poison
+        self._fake = FakeEmbedder()
+
+    def __call__(self, features: ProposalFeatures) -> np.ndarray:
+        if features.description == self._poison:
+            return np.full(len(self._fake(features)), np.nan, dtype="<f8")
+        return self._fake(features)
+
+
+def _verdict(vid: str, description: str) -> Verdict:
+    return Verdict(
+        id=vid,
+        proposal_features=ProposalFeatures(
+            affected_codes=frozenset({"05.2"}),
+            measure_type="scope_reduction",
+            claimed_saving_nok=200_000,
+            description=description,
+        ),
+        decision="approved",
+        rationale="prior scope reduction on the same codes was approved",
+    )
+
+
+def test_hybrid_rank_refuses_a_non_finite_embedding() -> None:
+    """Detach point: drop the non-finite check in ``cosine`` -> ``rank`` returns a silently
+    corrupted order instead of raising -> RED.
+
+    Refusal is the contract, not coercion-to-zero: a broken embedder must not be laundered into
+    "no semantic similarity" while ranking proceeds on the forged signal."""
+    query = ProposalFeatures(
+        affected_codes=frozenset({"05.2"}),
+        measure_type="scope_reduction",
+        claimed_saving_nok=200_000,
+        description="query prose",
+    )
+    candidates = [_verdict("A", "poisoned"), _verdict("B", "fine"), _verdict("C", "also fine")]
+    ranker = HybridRanker(_NaNEmbedder("poisoned"), similarity)
+
+    try:
+        ranked = ranker.rank(query, candidates, k=3)
+    except ValueError as exc:
+        assert "non-finite" in str(exc)
+    else:
+        raise AssertionError(
+            "rank() returned a NaN-contaminated ordering instead of refusing: "
+            f"{[v.id for v in ranked]}"
+        )
+
+
+def test_a_nan_sort_key_is_input_order_dependent() -> None:
+    """CONTROL — green with or without the guard. Its job is to pin the PREMISE the guard rests
+    on, so the guard above cannot later be dismissed as defensive noise.
+
+    ``HybridRanker``'s docstring promises ``id`` "makes the result independent of input order even
+    when two candidates score identically". Measured: a single NaN score breaks that promise —
+    NaN compares False against everything, so ``sorted`` leaves it wherever the input put it. Six
+    permutations of the same three candidates yielded FOUR distinct orderings."""
+    scored = [("A", float("nan")), ("B", 0.9), ("C", 0.5)]
+    orderings = {
+        tuple(item[0] for item in sorted(perm, key=lambda t: (-round(t[1], 9), t[0])))
+        for perm in itertools.permutations(scored)
+    }
+    assert len(orderings) > 1, (
+        "a NaN score no longer perturbs the ranking key — the premise behind "
+        "test_hybrid_rank_refuses_a_non_finite_embedding has changed, re-measure it"
+    )