open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 4 Activity

feat(p21): the PROJECT carries the price, so a run against a road normal can be anchored

Browse source 
Four paid stress rounds ran entirely UN-ANCHORED, all of them, because the one file
loader reads cost-baseline.json out of the BUNDLE and no vegnormal ships one: N100,
N200, N500 and R761 are knowledge, and knowledge carries requirements, never amounts.
The validator's stage 0 -- the one stage that tells an invented cost line from a line
this project actually buys -- was skipped in every single run, so "validated" could not
mean what it says. P20 G1/G2 measured real R761 process numbers (12.11 three times on
Soraasen, 1.1.1 on Lindaas) validating with amounts nobody had anywhere.

--cost-baseline FILE is PM decision (e), taken over the three alternatives P20 wrote
down. A LOADED object, never a path (prepass_payload's rule): the CLI owns the file and
loads it ONCE, so the notice, the stamp and every base of an --across-bundle pass all
descend from one read. ONE parse, two doors -- load_cost_baseline delegates to
load_cost_baseline_file -- while safe_resolve stays on the bundle door alone, because a
project's own schedule is legitimately outside every base. No tolerant twin: this path
exists only because an operator NAMED a file.

DEL B: five anchored context sets, a1-a3 with their line and a4 with none, so stage 0 is
what catches the falsification arm. THE ORDER'S OWN ARM (h) WAS FELLED BY MEASUREMENT:
"no baseline code is a requirement number the base declares" is measured 0 of 4 on the
project-coded sets and 5 of 5 on kontrakt-sorasen -- which is what R761 Prosesskoden IS,
a bill of quantities priced BY process code. The complement keeps both, and the order's
own mutation still bites.

DEL B3: the judge reports anchored (off the run's own stamp), priced per row, and WHICH
falsifier caught the falsification arm.

Load-bearing MEASURED, five mutations all red against the WHOLE suite, green control
1850/5 (from 1809/5, superset, 0 removed), golden byte-unchanged:
A3(i) the flag is read but the baseline is unused (3 red) . A3(ii) only the first base
gets it (1) . A3(iii) report_forbidden drops it (1) . B2(i) a4 gets a line (1, arm (g)
alone) . B2(ii) a code swapped to 12.11 (2, arms (f) and (h)).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-15 10:49:10 +02:00
commit 7b4f85d77c
20 changed files with 1259 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

29
src/portfolio_optimiser/okf.py
View file

@ -1601,6 +1601,35 @@ def load_cost_baseline(bundle_dir: str, name: str = _COST_BASELINE) -> CostBasel
resolved = Path(safe_resolve(bundle_dir, name))
if not resolved.is_file():
raise FileNotFoundError(f"cost baseline not found in bundle: {name!r}")
return load_cost_baseline_file(str(resolved))
def load_cost_baseline_file(path: str) -> CostBaseline:
"""Load a cost baseline from a file that is NOT inside a knowledge base: the PROJECT's own price
schedule (P21, ``--cost-baseline``).
The measured reason it exists. Four paid rounds (P16/P18/P19/P17b/P20) ran entirely UN-ANCHORED,
because the only file loader reads ``cost-baseline.json`` out of the bundle directory and no road
normal carries a price schedule: a vegnormal is KNOWLEDGE, and the price belongs to the PROJECT.
Stage 0 was therefore skipped in every one of them, and "validated" could not mean anything —
P20 G1/G2 measured real process numbers validating with invented amounts. This is the third door
into ``CostBaseline`` alongside the bundle file and ``derive_cost_baseline``, and the only one
whose input is the project rather than the corpus.
**The SAME parse, never a second one** (kø-(p)): ``load_cost_baseline`` resolves inside the
bundle and then delegates here, so the two doors cannot disagree about what a baseline file is.
What differs is the resolution — ``safe_resolve`` is the ONE in-/out-of-bundle test and stays on
the bundle door alone, because a project's own schedule is legitimately outside every base.
Fail-fast, the error CLASSES of ``load_cost_baseline``: a missing file raises
``FileNotFoundError`` and malformed content raises ``pydantic.ValidationError``. There is no
optional twin, and that is deliberate: the bundle file is absent by default (a base authored
before the amendment is legitimately un-anchored), whereas this path exists only because an
operator NAMED a file — tolerating its absence would answer an explicit order with a silently
un-anchored run (``load_mandate``'s rule)."""
resolved = Path(path)
if not resolved.is_file():
raise FileNotFoundError(f"cost baseline not found: {path!r}")
return CostBaseline.model_validate_json(resolved.read_text(encoding="utf-8"))

170
src/portfolio_optimiser/run.py
View file

@ -88,7 +88,7 @@ from portfolio_optimiser.generate import (
generate_via_llm,
grounding_offer,
)
from portfolio_optimiser.ir import SavingsProposal
from portfolio_optimiser.ir import CostBaseline, SavingsProposal
from portfolio_optimiser.mandate import (
OWN_PROPOSAL_ID,
Approach,
@ -822,6 +822,36 @@ def cost_baseline_notice(anchored: bool) -> str | None:
return None if anchored else _UNANCHORED_NOTICE
def cost_baseline_source_notice(path: str | None, lines: int) -> str | None:
"""Render where this run's cost baseline came from, or ``None`` when nobody named a file (P21).
A SECOND renderer beside ``cost_baseline_notice``, never a widening of it, because the two say
DIFFERENT facts and cannot disagree: that one warns that stage 0 is SKIPPED, this one names the
file an operator chose and how many lines it carries. Supplying ``--cost-baseline`` implies
anchored, so exactly one of the two can ever render.
**A POSITIVE line, and that is a deliberate departure from the omission rule** its neighbours
follow (``cost_baseline_notice``, ``skipped_links_notice``, ``unkeyed_verdicts_notice``) — the
same departure ``proposal_review_notice`` makes, for the same reason. Silence here is
AMBIGUOUS: an operator who passed a project price schedule cannot tell "your file anchored this
run" from "the bundle happened to ship its own" or from "the flag was dropped somewhere", and
the whole point of the flag is that stage 0 now judges. Without a file the renderer returns
``None``, so the omission is kept exactly where it is unambiguous.
It takes the ALREADY-RESOLVED path and count rather than re-reading the file: a renderer that
opened it again would be a second resolution of the same fact, free to drift from the baseline
the run was actually given (``cost_baseline_notice``'s rule).
English, like every other line this CLI prints."""
if path is None:
return None
plural = "" if lines == 1 else "s"
return (
f" Cost baseline: {lines} line{plural} from {path} — the validator's stage 0 reconciles "
"every proposed cost line against this project's own schedule"
)
def grounding_offer_notice(offer: GroundingOffer | None) -> str | None:
"""Render the one line that says what this run's delivered input can ground, or ``None`` when
there is nothing to warn about (P8).
@ -1018,6 +1048,13 @@ async def run_project(
#: file loader, byte-identically.
derive_cost_baseline: bool = False,
require_cost_baseline: bool = False,
#: P21: the PROJECT's own price schedule, supplied by the caller instead of read out of the
#: knowledge base. A LOADED object, never a path — ``prepass_payload``'s rule, and
#: ``mandate=``/``dimension=``' before it: the CLI owns the file, the library seam takes the
#: validated artefact. ``None`` (the default) leaves every existing run on the bundle loader,
#: byte-identically. Mutually exclusive with ``derive_cost_baseline``: two sources for one
#: baseline would have to silently pick one, and the picked one would be a policy nobody wrote.
cost_baseline: CostBaseline | None = None,
dimension: Dimension | None = None,
store: VerdictStore | None = None,
verdict_dir: str | None = None,
@ -1081,6 +1118,17 @@ async def run_project(
semantics, so over a structural tie the resulting order is deterministic but arbitrary;
retrieval *quality* arrives only with an embedder injected via ``embedder=`` or
``--embedder-config``. Default false keeps the structural ranking exactly as before."""
# 0. Fail-fast: TWO sources for ONE baseline, refused rather than merged (P21). A run holding
# both would have to pick silently, and the picked one would be a policy nobody wrote down —
# ``--prepass-payload``/``--prepass-seed``'s rule, and ``portfolio_meter``/``meter_factory``'s
# before it. Checked HERE and not only at the CLI, because the library seam has the same two
# arguments and a library caller must not be able to reach a state the CLI refuses by name.
if cost_baseline is not None and derive_cost_baseline:
raise ValueError(
"cost_baseline and derive_cost_baseline are two sources for one baseline (a file the "
"caller supplies, and a schedule derived from the knowledge base); pass exactly one"
)
# 0. Fail-fast: an outbox write is byte-deterministic and keyed on run_id — no wall-clock default.
if outbox_dir is not None and run_id is None:
raise ValueError(
@ -1173,8 +1221,15 @@ async def run_project(
# silently downgraded order, which is what ``load_mandate`` fail-fasts against. This one
# resolution serves BOTH the full run and the ``live_dry_run`` report below, so the dry-run
# arm cannot drift away from what a real run would anchor on.
# P21 takes precedence over BOTH bundle-side sources, and it is the only one whose input
# is the PROJECT rather than the corpus: a vegnormal is knowledge and carries no prices, so
# before this every paid run measured (P16/P18/P19/P17b/P20) was un-anchored and stage 0
# never ran. Mutual exclusion with ``derive_cost_baseline`` is enforced at the top of this
# function, so the ``if`` below is an ordering and not a silent pick.
baseline = (
okf.derive_cost_baseline(bundle, project_id=project_id)
cost_baseline
if cost_baseline is not None
else okf.derive_cost_baseline(bundle, project_id=project_id)
if derive_cost_baseline
else okf.load_optional_cost_baseline(bundle_dir)
)
@ -2429,6 +2484,12 @@ async def run_mandate_across_bundles(
#: silent drop on the paid path while the free drill honoured them, which is the F4 class.
derive_cost_baseline: bool = False,
require_cost_baseline: bool = False,
#: P21, and threaded UNCHANGED into every base: ONE project has ONE price schedule, so the same
#: baseline anchors each base's run. That is the one anchoring argument which is NOT a bundle
#: concern — the two above are read out of the base being handed over, this one is the project's
#: own, and giving base k a baseline and base k+1 none would anchor half a commission while the
#: stamp reported anchoring for the half that happened to run first.
cost_baseline: CostBaseline | None = None,
) -> MultiBaseResult:
"""Evaluate ONE commission across SEVERAL knowledge bases — the multi-base dispatch (§ C.7).
@ -2531,6 +2592,7 @@ async def run_mandate_across_bundles(
run_id=base_run_id or None,
derive_cost_baseline=derive_cost_baseline,
require_cost_baseline=require_cost_baseline,
cost_baseline=cost_baseline,
notify=lambda verdict: minted_here.append(verdict.id),
verdict_input=verdict_input,
verdict_dir=verdict_dir,
@ -2967,6 +3029,20 @@ def main(argv: list[str] | None = None) -> int:
"guesses: an unpriced or ambiguous schedule stops the run"
),
)
parser.add_argument(
"--cost-baseline",
default=None,
metavar="FILE",
help=(
"anchor the validator's stage 0 to THIS PROJECT's own price schedule (P21): FILE is a "
"cost-baseline.json (the same {project_id, items:{code:{quantity,unit_cost}}} shape a "
"bundle may ship) and it is used INSTEAD of one inside --bundle-dir. The price belongs "
"to the project, not to the knowledge base — a road normal carries requirements, never "
"amounts — so without this a run against one is un-anchored and stage 0 is skipped. "
"Applies to every base in --across-bundle mode: one project, one schedule. Mutually "
"exclusive with --derive-cost-baseline; satisfies --require-cost-baseline"
),
)
parser.add_argument(
"--require-cost-baseline",
action="store_true",
@ -3072,6 +3148,10 @@ def main(argv: list[str] | None = None) -> int:
# Same reason, same rung: report mode returns above every run dispatch, so leaving it
# out is a SILENT DROP of a guarantee the operator asked for by name.
"--require-cost-baseline": args.require_cost_baseline,
# P21, same rung and same reason: report mode returns ABOVE every dispatch that could
# honour a project price schedule, so an omission here would accept the file, anchor
# nothing, and exit 0 — a silent drop rather than a refusal (the F4 class).
"--cost-baseline": args.cost_baseline is not None,
# Same reason, one flag later: report mode returns above the S7b dispatch too.
"--proposals-from-mandate": args.proposals_from_mandate,
"PROJECT_ID": args.project_id is not None,
@ -3183,6 +3263,11 @@ def main(argv: list[str] | None = None) -> int:
# anchored by construction — so here the flag could only ever pass. BY NAME rather
# than falling through to the --bundle-dir requirement, its neighbours' reason.
"--require-cost-baseline": args.require_cost_baseline,
# P21: ONE project's price schedule, and a portfolio pass keys on PROJECTS — each of
# which already carries its own ``cost_items`` and is anchored by construction. A
# single file could only ever be right for one row out of N. BY NAME rather than
# falling through to the --bundle-dir requirement, its neighbours' reason.
"--cost-baseline": args.cost_baseline,
# It reads ONE base's schedule and settles ONE commission against it, so it sits on the
# same side of the partition as the flag it requires. BY NAME rather than falling
# through to "requires --derive-cost-baseline": an operator who wrote --portfolio
@ -3364,6 +3449,37 @@ def main(argv: list[str] | None = None) -> int:
)
return 1
# P21, and its neighbours' reason exactly: on the road path the baseline IS the project's own
# ``cost_items``, so a file here would be a SECOND source for one fact with nothing to break
# the tie — and a flag that can only ever be shadowed is a claim the surface makes about
# itself. Refused BY NAME rather than left to surface as a project lookup that ignored it.
if (
not args.portfolio
and not args.across_bundle
and args.cost_baseline is not None
and args.bundle_dir is None
):
print(
"run refused: --cost-baseline requires --bundle-dir or --across-bundle (the road path "
"is already anchored by the reference project's own cost_items, so a second schedule "
"there would have nothing to anchor that those do not)",
file=sys.stderr,
)
return 1
# TWO sources for ONE baseline, refused rather than merged — the same decision
# ``run_project`` enforces at its own seam, said HERE by name so an operator who typed both
# hears which two flags conflict instead of getting a library ValueError's traceback. Not
# nested under either flag's branch, for the F4 reason its neighbours are not.
if args.cost_baseline is not None and args.derive_cost_baseline:
print(
"run refused: --cost-baseline and --derive-cost-baseline are two sources for one "
"baseline (a file you supply, and a schedule derived from a table in the knowledge "
"base); pass exactly one",
file=sys.stderr,
)
return 1
# The SEEDING arm's three refusals, placed ABOVE the replacing arm's block on purpose: given
# both flags, the block below would answer with "--prepass-payload and --explore cannot be
# combined", which names neither of the two flags the operator actually put in conflict. At
@ -3815,6 +3931,20 @@ def main(argv: list[str] | None = None) -> int:
print(f"run refused: {exc}", file=sys.stderr)
return 1
# The PROJECT's price schedule, loaded fail-fast alongside the commission and for the same
# reason: a baseline that cannot be read is not a run to start UN-anchored instead. Degrading
# it to "no baseline" would answer an operator who asked for stage 0 by name with a run in
# which stage 0 is skipped — ``load_mandate``'s rule, and the exact silence four paid rounds
# were measured inside. Loaded ONCE and passed as an object, so the notice below, the stamp
# and every base of an ``--across-bundle`` pass all descend from one read (kø-(p)).
cost_baseline: CostBaseline | None = None
if args.cost_baseline is not None:
try:
cost_baseline = okf.load_cost_baseline_file(args.cost_baseline)
except (FileNotFoundError, ValidationError, ValueError) as exc:
print(f"run refused: {exc}", file=sys.stderr)
return 1
# The declared cut, loaded fail-fast alongside the commission and for the same reason: a
# payload that cannot be read is not a run to start with a navigating debate instead. Missing,
# not JSON, or not the shape the models require — all three land on the refusal surface with
@ -4191,6 +4321,15 @@ def main(argv: list[str] | None = None) -> int:
print(f"run refused: {exc}", file=sys.stderr)
return 1
# P21, printed ONCE for the pass rather than per base, and that is the opposite placement
# from its neighbours below FOR A REASON: they read each base's OWN stamp, whereas ONE
# project has ONE price schedule and the same baseline anchors every base here. Per base it
# would read as N schedules, which is the claim this flag exists to deny.
source_notice = cost_baseline_source_notice(
args.cost_baseline, 0 if cost_baseline is None else len(cost_baseline.items)
)
if source_notice is not None:
print(source_notice)
if args.live_dry_run:
for bundle_id, bundle_dir, project_id in resolved:
try:
@ -4210,6 +4349,7 @@ def main(argv: list[str] | None = None) -> int:
max_tokens=args.max_tokens,
derive_cost_baseline=args.derive_cost_baseline,
require_cost_baseline=args.require_cost_baseline,
cost_baseline=cost_baseline,
live_dry_run=True,
)
)
@ -4269,6 +4409,7 @@ def main(argv: list[str] | None = None) -> int:
),
derive_cost_baseline=args.derive_cost_baseline,
require_cost_baseline=args.require_cost_baseline,
cost_baseline=cost_baseline,
outbox_for=outbox_for,
)
)
@ -4302,6 +4443,15 @@ def main(argv: list[str] | None = None) -> int:
multi=multi,
)
# P21, printed ONCE for the pass rather than per base, and that is the opposite placement
# from its neighbours below FOR A REASON: they read each base's OWN stamp, whereas ONE
# project has ONE price schedule and the same baseline anchors every base here. Per base it
# would read as N schedules, which is the claim this flag exists to deny.
source_notice = cost_baseline_source_notice(
args.cost_baseline, 0 if cost_baseline is None else len(cost_baseline.items)
)
if source_notice is not None:
print(source_notice)
for bundle_run in multi.runs:
print(f"--- {bundle_run.bundle_id} ({bundle_run.project_id}) ---")
print(
@ -4440,6 +4590,7 @@ def main(argv: list[str] | None = None) -> int:
verdict_input=_verdict_input_from_args(args),
derive_cost_baseline=args.derive_cost_baseline,
require_cost_baseline=args.require_cost_baseline,
cost_baseline=cost_baseline,
mcp_servers=mcp_servers,
live_dry_run=True,
)
@ -4470,6 +4621,14 @@ def main(argv: list[str] | None = None) -> int:
notice = cost_baseline_notice(report.cost_baseline_anchored)
if notice is not None:
print(notice)
# P21's positive half, on the FREE trip: the operator who named a project price schedule
# learns on the drill that it was read and how many lines it carries, rather than paying
# for a run to find out. Silence without the flag (omission, never an empty row).
source_notice = cost_baseline_source_notice(
args.cost_baseline, 0 if cost_baseline is None else len(cost_baseline.items)
)
if source_notice is not None:
print(source_notice)
# P8, printed next to the line it qualifies: "stage 0 is skipped" says the gate lost a
# falsifier; this says what the input could have offered it instead. On the FREE trip, so
# an operator learns a run cannot be grounded without paying three attempts to find out.
@ -4519,6 +4678,7 @@ def main(argv: list[str] | None = None) -> int:
semantic_retrieval=args.semantic_retrieval,
derive_cost_baseline=args.derive_cost_baseline,
require_cost_baseline=args.require_cost_baseline,
cost_baseline=cost_baseline,
client_factory=scripted_client_factory,
mandate=mandate,
mcp_servers=mcp_servers,
@ -4567,6 +4727,12 @@ def main(argv: list[str] | None = None) -> int:
notice = cost_baseline_notice(result.provenance.cost_baseline_anchored)
if notice is not None:
print(notice)
# Same renderer on the paid run, so the drill and the run it rehearses say the same thing.
source_notice = cost_baseline_source_notice(
args.cost_baseline, 0 if cost_baseline is None else len(cost_baseline.items)
)
if source_notice is not None:
print(source_notice)
# Same renderer on the full run, read off the run's OWN measurement: a run that spent every
# attempt being refused as ungrounded is exactly where the input-side fact costs the most.
offer_notice = grounding_offer_notice(result.grounding_offer)

57
src/portfolio_optimiser/stress.py
View file

@ -76,7 +76,7 @@ from typing import Any
from portfolio_optimiser import okf
from portfolio_optimiser.mandate import Mandate, load_mandate
from portfolio_optimiser.validator import classify_codes
from portfolio_optimiser.validator import classify_codes, rejection_stage
#: Where the vegnormal bases are mounted, unless ``--bundle-root`` says otherwise. Read at CALL
#: time (the ``shared_root()`` idiom) so a test or an operator can move the mount without a reimport.
@ -125,6 +125,12 @@ class ApproachVerdict:
#: wrote one, and RE-DERIVED with the same classifier when it did not, so rounds 1 and 2 -
#: written before the field existed - can be re-judged with the same instrument.
prose_codes: tuple[str, ...]
#: P21 B3 - this row's ``affected_item`` codes ARE lines of the project's own price schedule.
#: Measured against ``contexts/<set>/cost-baseline.json`` (the schedule the run is given with
#: ``--cost-baseline``) and reported whether or not the run was anchored, so rounds written
#: before the schedule existed can be re-judged with the same instrument. ``False`` for a row
#: with no proposal, and for one whose codes the project does not buy.
priced: bool
#: P19 D2 - WHY this row was not evaluated: ``rounds`` / ``tokens`` when a cap cut the run
#: short, ``absent`` when the artefact is simply missing and no coverage file says otherwise,
#: and ``""`` for a row that WAS evaluated. Before this, "no artefact" could not be told from
@ -141,6 +147,14 @@ class RefusalVerdict:
approach_id: str
passed: bool
detail: str
#: P21 B3 - WHICH falsifier refused it, from ``validator.rejection_stage`` over the artefact's
#: own reason. ``stage0-baseline`` is the answer this whole order exists to make reachable: it
#: is the only stage that knows what the PROJECT buys, and before a project price schedule it
#: was skipped in every paid run. ``""`` when the arm produced no rejection to classify (it was
#: validated, or never evaluated) - an honest absence rather than a stage nobody reached.
#: REQUIRED without a default (``cost_baseline_anchored``'s rule): every construction site has
#: to say which falsifier spoke, and a default would let one of the three forget.
stage: str
@dataclass(frozen=True)
@ -172,6 +186,12 @@ class ContextSetVerdict:
#: P19 D2 - ``BudgetExceeded.kind`` when a cap cut the run short, ``""`` when nothing did, and
#: ``"absent"`` when the run wrote no coverage file at all (every run before today).
stop_reason: str
#: P21 B3 - whether the run's OWN stamp says stage 0 had a baseline to reconcile against, read
#: off ``provenance.cost_baseline_anchored`` rather than re-derived from a file: the judge
#: reports what the run DID, and a second resolution here would be free to disagree with it.
#: ``False`` when no artefact carried one (every round before P21). REQUIRED without a
#: default, for the reason ``ProvenanceStamp.cost_baseline_anchored`` is.
anchored: bool
tool_calls_seen: int
citations_seen: int
approach_rows_seen: int
@ -299,6 +319,19 @@ def score_context_set(
baseline = okf.load_optional_cost_baseline(str(base))
baseline_codes = set(baseline.items) if baseline is not None else set()
# P21 B3: the PROJECT's own price schedule, which is where the prices live — a road normal
# carries requirements and no amounts, so the ``load_optional_cost_baseline`` above finds
# nothing on every one of the four bases (measured). Read from the SET, which is the same file
# the run is given with ``--cost-baseline``, and read whether or not the run was anchored: that
# is what lets rounds written before the schedule existed be re-judged with this instrument.
# It never overwrites ``baseline_codes`` above — the hallucination arm's allowance is about
# what the BASE could ground, and merging the two would let a priced code launder a
# hallucinated one.
priced_codes: set[str] = set()
project_schedule = okf.load_optional_cost_baseline(str(context))
if project_schedule is not None:
priced_codes = set(project_schedule.items)
must_cite = {row["approach_id"]: row.get("concepts", []) for row in fasit.get("must_cite", [])}
refuse_ids = {row["approach_id"] for row in fasit.get("must_refuse", [])}
@ -353,6 +386,9 @@ def score_context_set(
str(_read_json(coverage_path).get("stop_reason", "")) if coverage_path.is_file() else ""
)
coverage_seen = coverage_path.is_file()
# P21 B3: read off the run's OWN stamp, accumulated over the artefacts below. A run is anchored
# or it is not, so ANY artefact saying so is the run saying so.
anchored = False
# ---- per approach ------------------------------------------------------------------------
rows: list[ApproachVerdict] = []
@ -385,6 +421,7 @@ def score_context_set(
requirement_source=_attributable(approach, declared_paths)[1],
requirement_hit=bool(set(_attributable(approach, declared_paths)[0]) & wanted),
prose_codes=(),
priced=False,
not_evaluated_reason=stop_reason or "absent",
ferdig=False,
)
@ -394,6 +431,7 @@ def score_context_set(
rows_seen += 1
payload = _read_json(proposal_path)
token_usage = max(token_usage, int(payload.get("provenance", {}).get("token_usage", 0)))
anchored = anchored or bool(payload.get("provenance", {}).get("cost_baseline_anchored"))
proposal = payload.get("proposal", {})
citations = payload.get("provenance", {}).get("citations", [])
citations_seen += len(citations)
@ -463,6 +501,7 @@ def score_context_set(
requirement_source=requirement_source,
requirement_hit=requirement_hit,
prose_codes=prose_codes,
priced=bool(codes) and all(c in priced_codes for c in codes),
not_evaluated_reason="",
ferdig=(
grounded
@ -491,20 +530,29 @@ def score_context_set(
commissioned = next((a for a in judged_approaches if a.id == rid), None)
refuse_codes = set(commissioned.affected_codes) if commissioned is not None else set()
leaked = sorted(refuse_codes & validated_codes)
# P21 B3: WHICH falsifier spoke, from the arm's own artefact. ``validator.rejection_stage``
# owns the classification because it owns the sentences (kø-(p)); this only reads the
# reason the run wrote down.
stage = ""
_, arm_outcome = _artefacts(outbox, run_id, rid)
if arm_outcome.is_file():
reason = str(_read_json(arm_outcome).get("reason", ""))
if reason:
stage = rejection_stage(reason)
if rid in validated_ids:
refusals.append(
RefusalVerdict(
rid, False, f"{rid} was VALIDATED - the base carries no ground for it"
rid, False, f"{rid} was VALIDATED - the base carries no ground for it", stage
)
)
elif leaked:
refusals.append(
RefusalVerdict(
rid, False, f"{rid}'s codes {leaked} ride inside a validated proposal"
rid, False, f"{rid}'s codes {leaked} ride inside a validated proposal", stage
)
)
else:
refusals.append(RefusalVerdict(rid, True, "no validated outcome rests on it"))
refusals.append(RefusalVerdict(rid, True, "no validated outcome rests on it", stage))
judged = [r for r in rows if r.approach_id not in refuse_ids]
return ContextSetVerdict(
@ -517,6 +565,7 @@ def score_context_set(
requirements_declared=tuple(declared_paths),
token_usage=token_usage,
stop_reason=stop_reason if coverage_seen else "absent",
anchored=anchored,
filter_calls=sum(1 for c in tool_calls if str(c.get("filter", ""))),
paged_calls=sum(
1 for c in tool_calls if int(c.get("offset", 0) or 0) or int(c.get("limit", 0) or 0)

36
src/portfolio_optimiser/validator.py
View file

@ -670,6 +670,42 @@ def validate_proposal(
return ValidatedProposal(proposal=proposal, p10=p10, p50=p50, p90=p90, nominal_feasible=nominal)
#: P21 B3 — which falsifier a ``Rejection`` came from, keyed on the SENTENCES this module writes.
#:
#: It lives HERE, next to the wordings, and never in the judge: a second copy of "what a stage 0
#: refusal looks like" would be free to drift from the sentence the validator actually emits, and
#: the reader most likely to be misled is the one re-judging a paid run months later (kø-(p)).
#:
#: In PIPELINE order, which is also the only order that can be right: ``validate_proposal`` returns
#: at the FIRST failing stage, so one reason carries violations from exactly one of them.
_REJECTION_STAGES: Final = (
("stage0-baseline", ("cost baseline (", "tolerance around the baseline ")),
("stage0b-grounding", ("ungrounded identifier ",)),
("stage4-p90", ("exceeds P90 feasible",)),
("stage4b-nominal", ("exceeds the nominal feasible",)),
("stage5-method-cap", ("method cap",)),
)
def rejection_stage(reason: str) -> str:
"""Which stage of the deterministic gate wrote ``reason`` — ``"other"`` when none did.
A REPORT, never a gate: nothing branches on the answer, so an unrecognised sentence costs a
label and not a verdict. That is why ``"other"`` is an honest answer here and would not be one
inside the pipeline.
The measured reason it exists (P21 B3). Before a project price schedule, the ``must_refuse``
arm of every context set fell — when it fell at all — on stage 0b, P7's grounding check, which
can only say "this identifier is not in the delivered text". Stage 0 is the one stage that
knows what the PROJECT buys, and it was skipped in every paid run measured, because no road
normal ships a ``cost-baseline.json``. Saying which stage caught the falsification arm is how a
reader can tell an anchored refusal from an un-anchored one that happened to land."""
for stage, markers in _REJECTION_STAGES:
if any(marker in reason for marker in markers):
return stage
return "other"
def self_repair(
generate: Callable[[int], SavingsProposal],
*,