portfolio-optimiser-claude/tests
Kjell Tore Guttormsen b955bf274e test(baseline): the baseline was loaded, and the judge could never reach it
S4.0, the fifth D7 mirroring candidate, measured with scripts/mutation_harness.py
against the real tree. C-F3 says "nothing in the bundle format carries a cost
baseline to reconcile against". Measured, that is too strong: validator-input.json
carries ENERGI-TOTAL-EL at 300 000 NOK and it is loaded on every run path. The
defect is not that the baseline is missing -- it is that no path connects it to
the judge.

A SavingsProposal comes into being in exactly three places (model-authored parse,
bundle baseline, re-read output; AST-measured over all 27 src files, positive
control: the same query finds validate_proposal's one call site). Only the first
reaches the validator. The only field read off the baseline directly is
project_id; the quantities, unit costs and uncertainty bands are schema-validated
and never read again.

Seven mutations, every one green-before/red-after with the restore sha256-verified
from disk:

  M1 baseline swapped for the weakest schema-valid variant -- 3 targets RED,
     all 974 pre-existing tests GREEN in both runs
  M2 the carried annotation narrowed to object -- AST ratchet RED, behaviour GREEN
  M3 the S10 path stops loading the baseline -- loader population RED
  M4 the validator gains a baseline parameter -- signature ratchet RED, golden GREEN
  M5 the call-site argument becomes a keyword -- provenance control RED
  M6 fail-closed code-set gate (the gated fix's own shape) -- boundary RED, golden GREEN
  M7 retrieval stops reading the codes -- disjointness RED

M4 and M6 leaving the golden green is itself the finding that S4.0 is
golden-compatible when D-A pkt. 2 lands.

A measuring trap was caught by the harness rather than by memory: the first M1
flattened the cost lines to 1.0x1.0, which broke the IR invariant (claim 30 000 >
total 1.0) and turned the run-entrance tests red. That redness attributed to
pydantic, not to the seam. The collateral control refused the proof.

New beyond C-F3: the fabricated line's code set is DISJOINT from the bundle's, and
retrieval computes that disjointness to rank experience. The system holds the
evidence that would expose the fabrication and spends it on ranking, never on
deciding.

No src/ change -- the fix is GATED on D-A pkt. 2 plus a commons amendment. Tests
pin the boundary so the gated work must arrive as a visible red test.

974 -> 984, strict superset, 0 node ids lost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-13 00:18:46 +02:00
..
data feat(portfolio): K11 — per-run value report, pure projection over the three layers (parity row 25) [skip-docs] 2026-07-25 06:25:02 +02:00
_scripted.py feat(loop): S8 — D7 agentic loop: budget meter, maker-checker gate, informed refinement 2026-07-03 07:21:02 +02:00
test_budget.py feat(portfolio): C3.5 — pre-call run-total USD budget belt (parity row 16/31) [skip-docs] 2026-07-23 22:45:37 +02:00
test_bygg_energi_mikro.py feat(validator): S6 — deterministic backbone: typed IR, golden-frozen validator, provenance stamp 2026-07-03 06:27:40 +02:00
test_checker_gate_loadbearing.py feat(loop): S8 — D7 agentic loop: budget meter, maker-checker gate, informed refinement 2026-07-03 07:21:02 +02:00
test_cli_paritet_loadbearing.py feat(harness): the procedure rebuilt ten times gets the four controls it kept skipping 2026-08-25 08:57:06 +02:00
test_contracts.py feat(portfolio): K2 — sequential multi-project run (parity row 4) 2026-07-23 21:39:39 +02:00
test_cost_baseline_loadbearing.py test(baseline): the baseline was loaded, and the judge could never reach it 2026-09-13 00:18:46 +02:00
test_costsim.py test(loadbearing): positive controls for the static-guard half of the sibling-vacuity class 2026-08-01 20:01:21 +02:00
test_dry_run_loadbearing.py feat(portfolio): K8 — live-run drill, pre-call artifact capture (parity row 21) [skip-docs] 2026-07-24 06:54:58 +02:00
test_experience_keying_loadbearing.py test(experience): the id was pinned, the keying was not - and the two paths disagree 2026-09-12 23:40:07 +02:00
test_goals.py feat(ledger): K1 — savings ledger + goal contract (parity rows 2-3) 2026-07-17 04:00:14 +02:00
test_hitl_loadbearing.py test(loadbearing): measure the last 8 all-negative tests instead of assuming them 2026-08-09 10:08:12 +02:00
test_ingest.py chore(deps): re-pin llm-ingestion-okf to v0.3.1 + migrate tests to stable error codes 2026-07-20 07:22:09 +02:00
test_ingest_adoption.py chore(deps): re-pin llm-ingestion-okf to v0.3.1 + migrate tests to stable error codes 2026-07-20 07:22:09 +02:00
test_ingest_golden.py feat(ingest): I3 — D7-speil av ingest (filkatalog/CSV), bygget fra commons-spec alene 2026-07-04 06:12:43 +02:00
test_ingest_loadbearing.py test(loadbearing): the spec moved to O2 and the emitter did not — ratchet the gap 2026-08-09 10:19:40 +02:00
test_ingest_spec_loadbearing.py test(ingest-spec): §11's two tables get the §12 row-anchor form 2026-08-20 23:18:23 +02:00
test_ingest_sql.py chore(deps): re-pin llm-ingestion-okf to v0.3.1 + migrate tests to stable error codes 2026-07-20 07:22:09 +02:00
test_ingest_sql_golden.py feat(ingest): I5 — SQL D7-speil, bygget fra commons-spec alene 2026-07-04 07:18:59 +02:00
test_ingest_sql_loadbearing.py test(loadbearing): the spec moved to O2 and the emitter did not — ratchet the gap 2026-08-09 10:19:40 +02:00
test_ingest_stamp_conformance_loadbearing.py test(loadbearing): the spec moved to O2 and the emitter did not — ratchet the gap 2026-08-09 10:19:40 +02:00
test_ir.py fix(validator): C2.6 — finiteness hardening, Infinity can no longer vacuously clear the gate (closes R-2) 2026-07-16 20:10:12 +02:00
test_ledger_loadbearing.py fix(ledger): normalize every load rejection to ValueError at the ledger's own entrance 2026-07-25 15:29:27 +02:00
test_loop.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_method_spec_loadbearing.py test(loadbearing): positive controls for the static-guard half of the sibling-vacuity class 2026-08-01 20:01:21 +02:00
test_method_spec_rule_phrases_loadbearing.py test(loadbearing): bind a rule phrase to every section the shape guard cannot see 2026-08-09 10:25:55 +02:00
test_mutation_harness.py feat(harness): the procedure rebuilt ten times gets the four controls it kept skipping 2026-08-25 08:57:06 +02:00
test_notify_loadbearing.py test(loadbearing): positive controls for the static-guard half of the sibling-vacuity class 2026-08-01 20:01:21 +02:00
test_notify_seam_loadbearing.py feat(portfolio): K10 — notification/notifier seam, opt-in webhook egress (parity row 23) [skip-docs] 2026-07-24 20:16:56 +02:00
test_okf.py test(loadbearing): positive controls for the static-guard half of the sibling-vacuity class 2026-08-01 20:01:21 +02:00
test_okf_unquote_loadbearing.py test(okf): the one unquoting rule was pinned at its edge, not in its three clauses 2026-08-31 23:31:17 +02:00
test_outbox_loadbearing.py feat(portfolio): K5 — outbox persistence, run_id-named pairs (parity row 7) [skip-docs] 2026-07-23 22:31:18 +02:00
test_persona_skill_loadbearing.py feat(learning): S9 — D7 læringssløyfe: verdict-inbox, fail-closed promoteringsgate, artefakt-sourced persona 2026-07-03 07:36:15 +02:00
test_portfolio.py feat(portfolio): K2 — sequential multi-project run (parity row 4) 2026-07-23 21:39:39 +02:00
test_portfolio_learning_loadbearing.py test(loadbearing): measure the last 8 all-negative tests instead of assuming them 2026-08-09 10:08:12 +02:00
test_preflight.py test(sdk): the pin was a permission, so give the premises a proof 2026-08-18 16:57:57 +02:00
test_provenance.py feat(validator): S6 — deterministic backbone: typed IR, golden-frozen validator, provenance stamp 2026-07-03 06:27:40 +02:00
test_provenance_sdk_version_loadbearing.py feat(run): stamp the drill's SDK build in the dry-run run-config [skip-docs] 2026-07-25 15:35:16 +02:00
test_readme_anchors_loadbearing.py fix(readme): the four dead TOC anchors, and a guard that reads the renderer instead of modelling it 2026-08-17 11:28:46 +02:00
test_run_entrance_loadbearing.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_s10_run_layer.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_sdk_isolation.py test(sdk): the pin was a permission, so give the premises a proof 2026-08-18 16:57:57 +02:00
test_sdk_tool_namespace_loadbearing.py test(sdk): the server knows its own name, and the tool never hears it 2026-08-25 08:33:07 +02:00
test_sdk_version_guard.py test(sdk): the pin was a permission, so give the premises a proof 2026-08-18 16:57:57 +02:00
test_simulation_loadbearing.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_step1_expel_loadbearing.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_step5_refine_loadbearing.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_step7_async_loop_loadbearing.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_step8_promotion_loadbearing.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_validator.py fix(validator): C2.6 — finiteness hardening, Infinity can no longer vacuously clear the gate (closes R-2) 2026-07-16 20:10:12 +02:00
test_validator_gate_loadbearing.py test(validator): the gate was pinned in its arithmetic, not in its decision [mutation-ok] 2026-09-07 00:07:19 +02:00
test_valuereport_loadbearing.py test(loadbearing): close the vacuous-negative class across the whole suite 2026-07-31 21:39:28 +02:00
test_valuereport_seam_loadbearing.py feat(portfolio): K11 — per-run value report, pure projection over the three layers (parity row 25) [skip-docs] 2026-07-25 06:25:02 +02:00