feat(p20): the requirement that is RIGHT, and a clause number that is not a price

Three seams, one commit: A, B and C touch the same four modules (run.py carries
the debate task, the grounding composition and the announcement; okf.py carries
one reference-number vocabulary read by both A and B), so splitting them into
three commits would have meant hunk-level staging of entangled files. Stated
rather than silently restructured.

A — the declaration answers with the DOCUMENT's own words. Measured: 13
declarations over round 3 and P17b, not one naming a fasit concept, while the
tool answered {"declared": true, ...} by echoing the caller's own arguments. It
now returns the document's title and req_number, read off Bundle.context_files
(so the type: verdict layer can never be named back), plus the sentence saying
what the declaration binds. A path the base carries as no concept answers with
empty strings rather than refusing. The commission's success_criteria now reach
the DEBATE task through mandate.criteria_block, the one renderer, empty when
there are none — which is what keeps every un-commissioned prompt, and the
golden, byte-identical.

B — a clause number is not a price. THE ORDER'S OWN RULE WAS FELLED BY
MEASUREMENT: it asks to refuse a code that IS declared req_number/prosessnr,
and neither of its two known positives is. n500 declares seksjon 10.4.1..10.4.4
but never the bare 10.4; r761 declares 2727 prosessnr and 2753 seksjon, none of
them 1.10.4, which occurs once, as prose ("iht. vegnormal N200 kap. 1.10.4").
The COMPLEMENT fires on both and closes the hole _ground_against_input already
admits in writing -- "it fails OPEN on a coincidental match". Unanchored run +
requirement-shaped code + the base declares a vocabulary + the code is not in
it -> refused, naming the denominator. All five of kontrakt-sorasen's real
process codes ARE declared and pass, which is what keeps the one context set
built on real codes measurable. Replayed over all 24 codes of round 3 + P17b:
exactly the two known positives flip validated -> rejected, 22 unchanged.

C — a parse failure no longer burns the round ledger blind. _fetch_parsed takes
a BUILDER instead of a finished message list, so the retry carries the parse
reason; measured, kontrakt-sorasen-04 spent 11 of 12 rounds re-asking the same
question. And announced_subject names the routed bases instead of saying "the
portfolio" for a two-base commission.

Suite 1807/5 (from 1781, +26, 0 removed), golden demo-transcript.stdout
BYTE-UNCHANGED (shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f),
ruff and mypy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

tests/test_requirement_number_gate_loadbearing.py

@@ -0,0 +1,275 @@
+"""P20 DEL B — a clause number is not a price, and the base's own vocabulary is what says so.
+
+**What was measured.** Three paid rounds and one multi-base pass carried FOUR ``validated``
+proposals whose cost code was a chapter number of a standard. Two survive in the recorded outboxes
+and are this arm's known positives:
+
+* ``10.4`` — tunnel-hauglia round 3, base ``vegnormal-n500-2024``, ``validated``;
+* ``1.10.4`` — lindaas P17b, base ``vegnormal-r761-2025``, ``validated``.
+
+Both are GROUNDED in P7's sense (they occur verbatim in the input) and neither is INERT in
+P18/B1's sense (``10.4`` in 12 of 274 documents, ``1.10.4`` in 1 of 2 756). Stage 0 never ran:
+no vegnormal base ships a cost baseline. Nothing in the gate could say what they are.
+
+**THE ORDER'S OWN RULE WAS FELLED BY MEASUREMENT BEFORE ANYTHING WAS BUILT ON IT.** B1 reads: a
+code is a requirement when it has form 2 or 3 AND "står som ``req_number``/``prosessnr`` i
+toppnivå-frontmatter i minst ett av grunnlagets dokumenter" — refuse that. Measured 15.09:
+
+* n500 declares ``seksjon: 10.4.1`` … ``10.4.4`` and ``req_number: Krav 10.4.3—2``. The bare
+  ``10.4`` is declared NOWHERE — it is a section PREFIX;
+* r761 declares 2 727 ``prosessnr`` and 2 753 ``seksjon``. ``1.10.4`` is NONE of them: it occurs
+  once, as prose, in "Krav til materialer skal være iht. vegnormal N200 Vegbygging kap. 1.10.4".
+
+The ordered rule therefore fires on NEITHER of its own known positives. The COMPLEMENT fires on
+BOTH, and it closes a hole ``_ground_against_input`` already admits in writing — "it fails OPEN …
+on a coincidental match". For one shape, a clause number, the base hands us the vocabulary needed
+to tell a real reference from a coincidence, and that is the rule built here.
+
+The complement is also what SPARES the one context set built on real process codes: all five of
+``contexts/kontrakt-sorasen-2027``'s codes are declared ``prosessnr`` and pass. Under the ordered
+rule every one of them would have been refused on an unanchored r761 run, and the set's positive
+arms would have become unmeasurable — the R761 risk the order names, arriving through the door it
+was pointed away from.
+
+Measured over EVERY code of round 3 and P17b (24 codes, 10 runs): exactly two are
+requirement-shaped, they are the two known positives, and the replay flips exactly those two.
+
+What each arm pins:
+
+(a) known positive — the ``10.4`` proposal, replayed against the base it actually ran on, is
+    ``rejected``, and the reason names the denominator;
+(b) known positive — ditto ``1.10.4`` on r761;
+(c) known negative — a code the base DOES declare (sorasen's real ``12.1``) still validates;
+(d) known negative — the gate is OFF when the run is anchored, even for a clause-shaped code;
+(e) the generality guard — an input that declares no reference numbers at all cannot trip the rule,
+    which is what leaves every pre-P20 fixture untouched rather than exempted;
+(f) K2's identifier forms are untouched: ``SHA-01`` is not requirement-shaped;
+(g) ``classify_codes``' third value, and its denominator-free reading (``grounding=None``) that
+    ``stress.py`` re-derives with;
+(h) the vocabulary travels WITH the text through ``_grounding_text``, so the gate the generation
+    loop runs sees what the run composed;
+(i) a run composes the vocabulary from the base it opened — asserted end-to-end through
+    ``run_project``, not on the composer.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+from pathlib import Path
+
+import pytest
+
+from portfolio_optimiser import okf
+from portfolio_optimiser.generate import _grounding_text
+from portfolio_optimiser.ir import CostBaseline, SavingsProposal
+from portfolio_optimiser.reference_domain import Project
+from portfolio_optimiser.validator import (
+    Grounding,
+    Rejection,
+    ValidatedProposal,
+    classify_codes,
+    has_requirement_form,
+    validate_proposal,
+)
+
+_DEFAULT_BUNDLE_ROOT = Path.home() / "repos" / "vegnormal-okf" / "build" / "ferdig"
+_ROUND3 = Path("scratchpad/p19-stress/tunnel-hauglia-2027")
+_P17B = Path("scratchpad/p17b-multibase/lindaas")
+
+
+def _base(name: str) -> Path:
+    root = Path(os.environ.get("PORTFOLIO_VEGNORMAL_ROOT", str(_DEFAULT_BUNDLE_ROOT)))
+    if not (root / name).is_dir():
+        pytest.skip(f"knowledge base {name!r} is not mounted under {root}")
+    return root / name
+
+
+def _grounding_over(name: str) -> Grounding:
+    """The delivered base exactly as ``run_project`` composes it — documents AND vocabulary."""
+    bundle = okf.navigate_bundle(str(_base(name)))
+    return Grounding(
+        documents=tuple(
+            "\n".join([f.name, *f.frontmatter.values(), f.body]) for f in bundle.context_files
+        ),
+        declared_references=tuple(
+            ref for f in bundle.context_files for ref in okf.declared_reference_numbers(f)
+        ),
+    )
+
+
+def _recorded(path: Path) -> SavingsProposal:
+    if not path.is_file():
+        pytest.skip(f"the recorded artefact {path} is not present in this checkout")
+    return SavingsProposal.model_validate(json.loads(path.read_text(encoding="utf-8"))["proposal"])
+
+
+def _proposal(code: str, *, saving: float = 1000.0) -> SavingsProposal:
+    return SavingsProposal(
+        project_id="p",
+        measure="m",
+        affected_items=[{"code": code, "quantity": 10.0, "unit_cost": 1000.0}],
+        claimed_saving_nok=saving,
+    )
+
+
+# ---------------------------------------------------------------------------------- known positives
+
+
+def test_the_section_number_that_reached_validated_on_n500_is_refused() -> None:
+    """(a) tunnel-04's ``10.4``, replayed against the base that run actually opened."""
+    proposal = _recorded(_ROUND3 / "tunnel-hauglia-2027-04-a4-enhetspris-ventilator-proposal.json")
+    assert [i.code for i in proposal.affected_items] == ["10.4"]
+    outcome = validate_proposal(proposal, baseline=None, grounding=_grounding_over("n500-2024"))
+    assert isinstance(outcome, Rejection)
+    assert "'10.4'" in outcome.reason
+    assert "not one of the 365 this knowledge base declares" in outcome.reason
+
+
+def test_the_process_number_that_reached_validated_on_r761_is_refused() -> None:
+    """(b) lindaas a4's ``1.10.4`` — a chapter of ANOTHER standard, quoted in one r761 document."""
+    proposal = _recorded(_P17B / "lindaas-01-vegnormal-r761-2025-a4-indeksregulering-proposal.json")
+    assert [i.code for i in proposal.affected_items] == ["1.10.4"]
+    outcome = validate_proposal(proposal, baseline=None, grounding=_grounding_over("r761-2025"))
+    assert isinstance(outcome, Rejection)
+    assert "not one of the 2765 this knowledge base declares" in outcome.reason
+
+
+# ---------------------------------------------------------------------------------- known negatives
+
+
+def test_a_process_code_the_base_declares_still_validates() -> None:
+    """(c) The arm that keeps this a rule about the corpus and not about shapes.
+
+    ``12.1`` is ``contexts/kontrakt-sorasen-2027``'s own first code and a REAL declared
+    ``prosessnr`` of R761. Under the ordered rule it would have been refused; it must not be.
+    """
+    grounding = _grounding_over("r761-2025")
+    assert "12.1" in grounding.reference_vocabulary
+    outcome = validate_proposal(_proposal("12.1"), baseline=None, grounding=grounding)
+    assert isinstance(outcome, ValidatedProposal), getattr(outcome, "reason", "")
+
+
+def test_every_sorasen_code_is_in_the_bases_vocabulary() -> None:
+    """(c) The whole context set, not one sample: five real codes, five declared numbers."""
+    codes = [
+        code
+        for approach in json.loads(
+            Path("contexts/kontrakt-sorasen-2027/mandate.json").read_text(encoding="utf-8")
+        )["approaches"]
+        for code in approach.get("affected_codes", [])
+    ]
+    vocabulary = _grounding_over("r761-2025").reference_vocabulary
+    shaped = [c for c in codes if has_requirement_form(c)]
+    assert len(shaped) == 5, shaped
+    assert [c for c in shaped if c not in vocabulary] == []
+
+
+def test_the_fasit_references_are_classified_requirement() -> None:
+    """(g) known negative (c) of the order: a fasit reference IS a requirement, and says so."""
+    fasit = json.loads(
+        Path("contexts/kontrakt-sorasen-2027/fasit.json").read_text(encoding="utf-8")
+    )
+    refs = sorted({c["ref"] for entry in fasit["must_cite"] for c in entry["concepts"]})
+    forms = classify_codes(refs, _grounding_over("r761-2025"))
+    assert set(forms.values()) == {"requirement"}, forms
+
+
+def test_an_anchored_run_is_untouched_by_the_rule() -> None:
+    """(d) Stage 0 has already ruled; the weaker stage must not overrule the stronger."""
+    grounding = Grounding(documents=("12.9 is a clause",), declared_references=("12.1", "12.2"))
+    baseline = CostBaseline(project_id="p", items={"12.9": {"quantity": 10.0, "unit_cost": 1000.0}})
+    outcome = validate_proposal(_proposal("12.9"), baseline=baseline, grounding=grounding)
+    assert isinstance(outcome, ValidatedProposal), getattr(outcome, "reason", "")
+    # The control: the SAME code and the SAME text, unanchored, is refused.
+    unanchored = validate_proposal(_proposal("12.9"), baseline=None, grounding=grounding)
+    assert isinstance(unanchored, Rejection)
+
+
+def test_an_input_that_declares_no_reference_numbers_cannot_trip_the_rule() -> None:
+    """(e) The generality guard — and the reason every pre-P20 fixture is untouched."""
+    grounding = Grounding(documents=("a document mentioning 12.9 once",))
+    assert grounding.reference_vocabulary == frozenset()
+    outcome = validate_proposal(_proposal("12.9"), baseline=None, grounding=grounding)
+    assert isinstance(outcome, ValidatedProposal), getattr(outcome, "reason", "")
+
+
+def test_a_cost_line_identifier_is_not_requirement_shaped() -> None:
+    """(f) K2's 50 identifiers and this repo's own code: shape, measured."""
+    assert not has_requirement_form("SHA-01")
+    assert not has_requirement_form("ENERGI-TOTAL-EL")
+    assert not has_requirement_form("65 ASFALTDEKKER")
+    assert has_requirement_form("10.4") and has_requirement_form("Krav 4.1.2—1")
+
+
+def test_classify_codes_without_a_grounding_is_the_pre_p20_answer() -> None:
+    """(g) ``stress.py`` re-derives for runs written before the field existed."""
+    assert classify_codes(["12.1", "SHA-01", "impulsventilator"]) == {
+        "12.1": "identifier",
+        "SHA-01": "identifier",
+        "impulsventilator": "prose",
+    }
+    grounding = Grounding(documents=("x",), declared_references=("12.1",))
+    assert classify_codes(["12.1"], grounding) == {"12.1": "requirement"}
+
+
+# ---------------------------------------------------------------------------------- the wiring
+
+
+def test_the_vocabulary_travels_with_the_text_into_the_generation_gate() -> None:
+    """(h) ``_grounding_text`` composes the run's three sources; the vocabulary must survive it."""
+    delivered = Grounding(documents=("d",), declared_references=("12.1",))
+    project = Project(
+        id="p", name="n", description="d", currency="NOK", cost_items=(), docs_dir="."
+    )
+    composed = _grounding_text(project, None, delivered)
+    assert composed.reference_vocabulary == frozenset({"12.1"})
+
+
+def test_a_run_composes_the_vocabulary_from_the_base_it_opened(tmp_path: Path) -> None:
+    """(i) End-to-end through ``run_project``: the stamp says ``requirement`` for a declared code.
+
+    Asserted on the ARTEFACT a run leaves, never on the composer — a vocabulary wired nowhere would
+    satisfy every arm above and none of this one.
+    """
+    import asyncio
+
+    from agent_framework import BaseChatClient
+
+    from portfolio_optimiser.run import run_project
+    from portfolio_optimiser.simulation import ScriptedChatClient
+
+    base = tmp_path / "mini"
+    base.mkdir()
+    (base / "index.md").write_text(
+        "---\nbundle_id: mini\n---\n\n- [Krav](krav.md) — one clause.\n", encoding="utf-8"
+    )
+    (base / "krav.md").write_text(
+        "---\ntype: Krav\ntitle: Krav 4.1.2-1\nprosessnr: '12.1'\n---\n\nEn kostlinje 12.1.\n",
+        encoding="utf-8",
+    )
+    (base / "validator-input.json").write_text(
+        json.dumps({"project_id": "mini-p", "measure": "m", "affected_codes": ["12.1"]}),
+        encoding="utf-8",
+    )
+    reply = (
+        '{"measure":"m","affected_items":[{"code":"12.1","quantity":10,"unit_cost":1000}],'
+        '"claimed_saving_nok":1000}'
+    )
+
+    def factory(role: str) -> BaseChatClient:
+        return ScriptedChatClient(
+            "Reasoning holds.\nVERDICT: APPROVE" if role == "checker" else reply, role=role
+        )
+
+    result = asyncio.run(
+        run_project(
+            "mini-p",
+            "local",
+            docs_dir=str(base),
+            bundle_dir=str(base),
+            client_factory=factory,
+        )
+    )
+    assert result.provenance.code_forms == {"12.1": "requirement"}
+    assert result.provenance.validator_decision == "validated"