open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 4 Activity

feat(p21): the PROJECT carries the price, so a run against a road normal can be anchored

Browse source 
Four paid stress rounds ran entirely UN-ANCHORED, all of them, because the one file
loader reads cost-baseline.json out of the BUNDLE and no vegnormal ships one: N100,
N200, N500 and R761 are knowledge, and knowledge carries requirements, never amounts.
The validator's stage 0 -- the one stage that tells an invented cost line from a line
this project actually buys -- was skipped in every single run, so "validated" could not
mean what it says. P20 G1/G2 measured real R761 process numbers (12.11 three times on
Soraasen, 1.1.1 on Lindaas) validating with amounts nobody had anywhere.

--cost-baseline FILE is PM decision (e), taken over the three alternatives P20 wrote
down. A LOADED object, never a path (prepass_payload's rule): the CLI owns the file and
loads it ONCE, so the notice, the stamp and every base of an --across-bundle pass all
descend from one read. ONE parse, two doors -- load_cost_baseline delegates to
load_cost_baseline_file -- while safe_resolve stays on the bundle door alone, because a
project's own schedule is legitimately outside every base. No tolerant twin: this path
exists only because an operator NAMED a file.

DEL B: five anchored context sets, a1-a3 with their line and a4 with none, so stage 0 is
what catches the falsification arm. THE ORDER'S OWN ARM (h) WAS FELLED BY MEASUREMENT:
"no baseline code is a requirement number the base declares" is measured 0 of 4 on the
project-coded sets and 5 of 5 on kontrakt-sorasen -- which is what R761 Prosesskoden IS,
a bill of quantities priced BY process code. The complement keeps both, and the order's
own mutation still bites.

DEL B3: the judge reports anchored (off the run's own stamp), priced per row, and WHICH
falsifier caught the falsification arm.

Load-bearing MEASURED, five mutations all red against the WHOLE suite, green control
1850/5 (from 1809/5, superset, 0 removed), golden byte-unchanged:
A3(i) the flag is read but the baseline is unused (3 red) . A3(ii) only the first base
gets it (1) . A3(iii) report_forbidden drops it (1) . B2(i) a4 gets a line (1, arm (g)
alone) . B2(ii) a code swapped to 12.11 (2, arms (f) and (h)).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-15 10:49:10 +02:00
commit 7b4f85d77c
20 changed files with 1259 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

57
src/portfolio_optimiser/stress.py
View file

@ -76,7 +76,7 @@ from typing import Any
from portfolio_optimiser import okf
from portfolio_optimiser.mandate import Mandate, load_mandate
from portfolio_optimiser.validator import classify_codes
from portfolio_optimiser.validator import classify_codes, rejection_stage
#: Where the vegnormal bases are mounted, unless ``--bundle-root`` says otherwise. Read at CALL
#: time (the ``shared_root()`` idiom) so a test or an operator can move the mount without a reimport.
@ -125,6 +125,12 @@ class ApproachVerdict:
#: wrote one, and RE-DERIVED with the same classifier when it did not, so rounds 1 and 2 -
#: written before the field existed - can be re-judged with the same instrument.
prose_codes: tuple[str, ...]
#: P21 B3 - this row's ``affected_item`` codes ARE lines of the project's own price schedule.
#: Measured against ``contexts/<set>/cost-baseline.json`` (the schedule the run is given with
#: ``--cost-baseline``) and reported whether or not the run was anchored, so rounds written
#: before the schedule existed can be re-judged with the same instrument. ``False`` for a row
#: with no proposal, and for one whose codes the project does not buy.
priced: bool
#: P19 D2 - WHY this row was not evaluated: ``rounds`` / ``tokens`` when a cap cut the run
#: short, ``absent`` when the artefact is simply missing and no coverage file says otherwise,
#: and ``""`` for a row that WAS evaluated. Before this, "no artefact" could not be told from
@ -141,6 +147,14 @@ class RefusalVerdict:
approach_id: str
passed: bool
detail: str
#: P21 B3 - WHICH falsifier refused it, from ``validator.rejection_stage`` over the artefact's
#: own reason. ``stage0-baseline`` is the answer this whole order exists to make reachable: it
#: is the only stage that knows what the PROJECT buys, and before a project price schedule it
#: was skipped in every paid run. ``""`` when the arm produced no rejection to classify (it was
#: validated, or never evaluated) - an honest absence rather than a stage nobody reached.
#: REQUIRED without a default (``cost_baseline_anchored``'s rule): every construction site has
#: to say which falsifier spoke, and a default would let one of the three forget.
stage: str
@dataclass(frozen=True)
@ -172,6 +186,12 @@ class ContextSetVerdict:
#: P19 D2 - ``BudgetExceeded.kind`` when a cap cut the run short, ``""`` when nothing did, and
#: ``"absent"`` when the run wrote no coverage file at all (every run before today).
stop_reason: str
#: P21 B3 - whether the run's OWN stamp says stage 0 had a baseline to reconcile against, read
#: off ``provenance.cost_baseline_anchored`` rather than re-derived from a file: the judge
#: reports what the run DID, and a second resolution here would be free to disagree with it.
#: ``False`` when no artefact carried one (every round before P21). REQUIRED without a
#: default, for the reason ``ProvenanceStamp.cost_baseline_anchored`` is.
anchored: bool
tool_calls_seen: int
citations_seen: int
approach_rows_seen: int
@ -299,6 +319,19 @@ def score_context_set(
baseline = okf.load_optional_cost_baseline(str(base))
baseline_codes = set(baseline.items) if baseline is not None else set()
# P21 B3: the PROJECT's own price schedule, which is where the prices live — a road normal
# carries requirements and no amounts, so the ``load_optional_cost_baseline`` above finds
# nothing on every one of the four bases (measured). Read from the SET, which is the same file
# the run is given with ``--cost-baseline``, and read whether or not the run was anchored: that
# is what lets rounds written before the schedule existed be re-judged with this instrument.
# It never overwrites ``baseline_codes`` above — the hallucination arm's allowance is about
# what the BASE could ground, and merging the two would let a priced code launder a
# hallucinated one.
priced_codes: set[str] = set()
project_schedule = okf.load_optional_cost_baseline(str(context))
if project_schedule is not None:
priced_codes = set(project_schedule.items)
must_cite = {row["approach_id"]: row.get("concepts", []) for row in fasit.get("must_cite", [])}
refuse_ids = {row["approach_id"] for row in fasit.get("must_refuse", [])}
@ -353,6 +386,9 @@ def score_context_set(
str(_read_json(coverage_path).get("stop_reason", "")) if coverage_path.is_file() else ""
)
coverage_seen = coverage_path.is_file()
# P21 B3: read off the run's OWN stamp, accumulated over the artefacts below. A run is anchored
# or it is not, so ANY artefact saying so is the run saying so.
anchored = False
# ---- per approach ------------------------------------------------------------------------
rows: list[ApproachVerdict] = []
@ -385,6 +421,7 @@ def score_context_set(
requirement_source=_attributable(approach, declared_paths)[1],
requirement_hit=bool(set(_attributable(approach, declared_paths)[0]) & wanted),
prose_codes=(),
priced=False,
not_evaluated_reason=stop_reason or "absent",
ferdig=False,
)
@ -394,6 +431,7 @@ def score_context_set(
rows_seen += 1
payload = _read_json(proposal_path)
token_usage = max(token_usage, int(payload.get("provenance", {}).get("token_usage", 0)))
anchored = anchored or bool(payload.get("provenance", {}).get("cost_baseline_anchored"))
proposal = payload.get("proposal", {})
citations = payload.get("provenance", {}).get("citations", [])
citations_seen += len(citations)
@ -463,6 +501,7 @@ def score_context_set(
requirement_source=requirement_source,
requirement_hit=requirement_hit,
prose_codes=prose_codes,
priced=bool(codes) and all(c in priced_codes for c in codes),
not_evaluated_reason="",
ferdig=(
grounded
@ -491,20 +530,29 @@ def score_context_set(
commissioned = next((a for a in judged_approaches if a.id == rid), None)
refuse_codes = set(commissioned.affected_codes) if commissioned is not None else set()
leaked = sorted(refuse_codes & validated_codes)
# P21 B3: WHICH falsifier spoke, from the arm's own artefact. ``validator.rejection_stage``
# owns the classification because it owns the sentences (kø-(p)); this only reads the
# reason the run wrote down.
stage = ""
_, arm_outcome = _artefacts(outbox, run_id, rid)
if arm_outcome.is_file():
reason = str(_read_json(arm_outcome).get("reason", ""))
if reason:
stage = rejection_stage(reason)
if rid in validated_ids:
refusals.append(
RefusalVerdict(
rid, False, f"{rid} was VALIDATED - the base carries no ground for it"
rid, False, f"{rid} was VALIDATED - the base carries no ground for it", stage
)
)
elif leaked:
refusals.append(
RefusalVerdict(
rid, False, f"{rid}'s codes {leaked} ride inside a validated proposal"
rid, False, f"{rid}'s codes {leaked} ride inside a validated proposal", stage
)
)
else:
refusals.append(RefusalVerdict(rid, True, "no validated outcome rests on it"))
refusals.append(RefusalVerdict(rid, True, "no validated outcome rests on it", stage))
judged = [r for r in rows if r.approach_id not in refuse_ids]
return ContextSetVerdict(
@ -517,6 +565,7 @@ def score_context_set(
requirements_declared=tuple(declared_paths),
token_usage=token_usage,
stop_reason=stop_reason if coverage_seen else "absent",
anchored=anchored,
filter_calls=sum(1 for c in tool_calls if str(c.get("filter", ""))),
paged_calls=sum(
1 for c in tool_calls if int(c.get("offset", 0) or 0) or int(c.get("limit", 0) or 0)