MAJOR-3/S7a-3 gjorde utforskningen billig og lot pipelinen staa. Maalt paa K2
(630 konsepter, S7bs eget instrument, kjent-positiv-kontrollen reprodusert
eksakt FOER bruk): okf.bundle_context er 648 962 o200k-tokens og rir i TRE
kopier = 1 947 342 = 99,1 % av en kjoerings prompt-tokens.
Et premiss i maaledokumentet ble presisert foerst: de tre kopiene er tre
DEBATT-turer (proposer x2, checker x1), mens genererings-prompten er 156
tokens, fordi gen_context = debate_output or context. Det avgjorde formen -
generering trengte ingen egen soem, for aa binde `context` binder
siste-utvei-fallbacken ved konstruksjon.
run_project sender naa en PEKER (fast tekst + erklaert bundle_id + antall
konseptdokumenter i scope + stigen, O(1) i korpuset) og gir debatten de SAMME
fire verktoeyene utforskningen bruker - explore.navigator_tools gjenbrukt,
aldri en andre kopi av policyen.
Etter: 753 tokens like-for-like (samme manus, samme fire prompter, -99,96 %)
og 8 942 med en debatt som faktisk gaar stigen (-99,5 %), mot operatoerens
terskel 195 000 = 4,6 % av taket. Validert besparelse og validatorens dom er
UENDRET (850 000 NOK av 3 852 500, 2 av 5 felt paa stage 4 og 5, samme
dom-noekkel), og utforskningens 18 355 er uendret til tokenet.
§4.1a maatte flytte, ikke forsvinne: dimensjonsfilteret bodde i renderingen og
bor naa i VERKTOEYENE, paa begge trinn - en listing som skjuler et fremmed
dokument mens read_file serverer det paa sti er et filter i navnet alene.
okf.in_dimension er eneste predikat.
Sporet er kaller-eid (ExplorationToolRecorder -> RunResult.debate_tool_calls ->
{run_id}-debate.json fra en finally) og skrives ogsaa TOMT: en debatt som
navigerer ingenting ER S2c-regresjonen, saa den maa kunne leses.
Load-bearing MAALT: aatte mutasjoner roede mot HELE suiten, groenn kontroll
1306/5 (fra 1295/5), golden demo-transcript.stdout BYTE-UENDRET
(shasum -a 1 av innholdet = ea8c534773acdbe41ae68f2c55724d69aaf8be4f).
M7 falsifiserte seg selv, ikke gaten - staar som maalt.
Maaling: docs/2026-09-04-s2c-debatt-k2.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
208 lines
9.2 KiB
Python
208 lines
9.2 KiB
Python
"""Step 2 load-bearing seam (SC2 + brief §4.1, målbilde §2/§6): a run's ``dimension`` does TWO
|
|
things, each with a named detach point:
|
|
|
|
- **Candidate constraint (§4.1b):** a validator-VALID candidate (P90-valid, empty assumptions)
|
|
whose ``measure_type`` falls OUTSIDE the run's dimension is rejected WHEN a dimension is set.
|
|
The proposal validates on the numbers, so the ONLY possible rejecter is the ``admits`` scope
|
|
gate (closes the green-but-dead trap). RED if ``admits`` is removed ⇒ the foreign candidate
|
|
slips through. Control: an in-dimension candidate passes.
|
|
- **Context scope (§4.1a), as it stands after S2c:** the promise is unchanged — a dimension-scoped
|
|
run lets the agents read ONLY dimension-matched bundle knowledge — but the debate no longer
|
|
RECEIVES a rendered context, it NAVIGATES the base, so the filter had to move from
|
|
``bundle_context`` to the tools the debate is handed. These two arms gate the FIRST rung: the
|
|
listing the debate can see never names a foreign-dimension document. RED if the ``dimension=``
|
|
arg is dropped anywhere along ``run_project`` → ``navigator_tools`` → ``directory_listing``.
|
|
Control: ``dimension=None`` ⇒ the same document IS listed. The SECOND rung (``read_file``
|
|
refusing a foreign document by path) is gated in
|
|
``tests/test_debate_navigation_cost_loadbearing.py`` — a listing filter with an ungated reader
|
|
behind it is a filter in name only, so the two halves get their own arms and their own
|
|
mutations.
|
|
|
|
Patterns: ``test_checker_gate_loadbearing.py:59/87`` (gate + causality control),
|
|
``conftest.py:184`` (recording client), ``test_step8_promotion_loadbearing.py:51`` (bundle copy).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import shutil
|
|
from collections.abc import Callable
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
from agent_framework import BaseChatClient
|
|
from conftest import SyntheticUsageChatClient
|
|
|
|
from portfolio_optimiser import okf
|
|
from portfolio_optimiser.dimension import Dimension
|
|
from portfolio_optimiser.run import run_project
|
|
from portfolio_optimiser.validator import Rejection, ValidatedProposal
|
|
|
|
BUNDLE_DIR = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
|
|
|
|
_ENERGY_DIM = Dimension(
|
|
id="energi", label="Energi", allowed_measure_types=frozenset({"energy_efficiency"})
|
|
)
|
|
_VERDICT_INPUT = {"decision": "approved", "rationale": "expert reviewed (sim)"}
|
|
|
|
# A marker that appears ONLY in the asfalt-marked concept file's body, so it can reach the prompt
|
|
# solely through un-filtered context — its presence/absence is the §4.1a leak probe.
|
|
_ASFALT_SENTINEL = "ASFALT-LEAK-SENTINEL-x7y8z9"
|
|
|
|
|
|
def _valid_reply(measure: str, code: str) -> str:
|
|
"""A validator-VALID proposal: affected total 300000, degenerate P90 = 0.30 x 300000 = 90000
|
|
>= claimed 30000, empty assumptions -> validates on the numbers regardless of ``measure``/``code``
|
|
(so a rejection can only come from the dimension scope gate)."""
|
|
return (
|
|
'{"measure":"' + measure + '","affected_items":'
|
|
'[{"code":"' + code + '","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":30000}'
|
|
)
|
|
|
|
|
|
def _role_factory(proposer_reply: str, checker_reply: str) -> Callable[[str], BaseChatClient]:
|
|
def factory(role: str) -> BaseChatClient:
|
|
return SyntheticUsageChatClient(
|
|
default_reply=checker_reply if role == "checker" else proposer_reply
|
|
)
|
|
|
|
return factory
|
|
|
|
|
|
# --- Candidate constraint (§4.1b) ----------------------------------------------------------------
|
|
|
|
|
|
async def test_foreign_dimension_candidate_rejected_when_dimension_set() -> None:
|
|
"""LOAD-BEARING: a foreign-dimension candidate that validates on the numbers is rejected by the
|
|
scope gate. RED if ``admits`` is detached (the foreign candidate slips through as validated)."""
|
|
factory = _role_factory(
|
|
_valid_reply("paving_renegotiation", "SENTINEL-FOREIGN"), "VERDICT: APPROVE"
|
|
)
|
|
|
|
result = await run_project(
|
|
"BYGG-KONTOR-NORD",
|
|
"local",
|
|
docs_dir=str(BUNDLE_DIR),
|
|
bundle_dir=str(BUNDLE_DIR),
|
|
verdict_input=_VERDICT_INPUT,
|
|
dimension=_ENERGY_DIM,
|
|
client_factory=factory,
|
|
)
|
|
|
|
assert isinstance(result.outcome, Rejection), (
|
|
"a foreign-dimension candidate slipped through — the admits scope gate is not gating"
|
|
)
|
|
assert "outside dimension" in result.outcome.reason
|
|
# Provenance honesty: the VALIDATOR passed (the numbers are feasible); only the scope gate
|
|
# rejected. validator_decision reflects the numbers ONLY — never the scope gate.
|
|
assert result.provenance.validator_decision == "validated"
|
|
|
|
|
|
async def test_in_dimension_candidate_passes() -> None:
|
|
"""CAUSALITY CONTROL: the SAME shape with an in-dimension ``measure_type`` validates normally —
|
|
proving the rejection above is caused by the dimension scope, not the fixture."""
|
|
factory = _role_factory(
|
|
_valid_reply("energy_efficiency", "ENERGI-TOTAL-EL"), "VERDICT: APPROVE"
|
|
)
|
|
|
|
result = await run_project(
|
|
"BYGG-KONTOR-NORD",
|
|
"local",
|
|
docs_dir=str(BUNDLE_DIR),
|
|
bundle_dir=str(BUNDLE_DIR),
|
|
verdict_input=_VERDICT_INPUT,
|
|
dimension=_ENERGY_DIM,
|
|
client_factory=factory,
|
|
)
|
|
|
|
assert isinstance(result.outcome, ValidatedProposal)
|
|
|
|
|
|
# --- Context scope (§4.1a) -----------------------------------------------------------------------
|
|
|
|
|
|
def _bundle_with_asfalt_file(tmp_path: Path) -> str:
|
|
"""A throwaway copy of the shared bundle with one extra asfalt-marked concept file carrying the
|
|
sentinel in its body, linked from the index so ``navigate_bundle`` reaches it. The shared,
|
|
framework-neutral fixture is never mutated (mirrors ``_copy_bundle``)."""
|
|
dst = tmp_path / "bundle"
|
|
shutil.copytree(BUNDLE_DIR, dst)
|
|
(dst / "asfalt-note.md").write_text(
|
|
f"---\ntype: methodology\ndimension: asfalt\n---\n\n{_ASFALT_SENTINEL} — paving method note\n",
|
|
encoding="utf-8",
|
|
)
|
|
index = dst / "index.md"
|
|
index.write_text(
|
|
index.read_text(encoding="utf-8") + "\n- [asfalt](asfalt-note.md)\n", encoding="utf-8"
|
|
)
|
|
return str(dst)
|
|
|
|
|
|
async def _listing_the_debate_can_see(bundle_dir: str, dimension: Dimension | None) -> str:
|
|
"""Run the bundle path, capture the tools the debate was built with, and return what its
|
|
``read_bundle`` rung answers — the S2c successor to reading the prompt text.
|
|
|
|
The tool is invoked DIRECTLY afterwards rather than through an agent turn, because a
|
|
``ScriptedChatClient`` returns text and never emits a ``function_call``: a test that only drove
|
|
the run would leave the entire tool surface outside the gate (``test_explore_loadbearing``'s
|
|
own measured correction)."""
|
|
import portfolio_optimiser.run as run_module
|
|
|
|
captured: list[list[object]] = []
|
|
original = run_module.fresh_workflow
|
|
|
|
def spy(*args: object, **kwargs: object) -> object:
|
|
captured.append(list(kwargs.get("tools") or [])) # type: ignore[arg-type]
|
|
return original(*args, **kwargs) # type: ignore[arg-type]
|
|
|
|
def factory(role: str) -> BaseChatClient:
|
|
return SyntheticUsageChatClient(
|
|
default_reply=_valid_reply("energy_efficiency", "ENERGI-TOTAL-EL")
|
|
)
|
|
|
|
monkeypatch = pytest.MonkeyPatch()
|
|
try:
|
|
monkeypatch.setattr(run_module, "fresh_workflow", spy)
|
|
await run_project(
|
|
"BYGG-KONTOR-NORD",
|
|
"local",
|
|
docs_dir=bundle_dir,
|
|
bundle_dir=bundle_dir,
|
|
verdict_input=_VERDICT_INPUT,
|
|
dimension=dimension,
|
|
client_factory=factory,
|
|
)
|
|
finally:
|
|
monkeypatch.undo()
|
|
|
|
assert captured, "the debate was never built"
|
|
read_bundle = next(t for t in captured[0] if getattr(t, "name", "") == "read_bundle")
|
|
bundle_id = okf.reconcile_bundle_id(bundle_dir).id
|
|
answer = await read_bundle.invoke(arguments={"bundle_id": bundle_id})
|
|
return "".join(getattr(c, "text", "") or "" for c in answer)
|
|
|
|
|
|
async def test_dimension_scopes_the_agent_context(tmp_path) -> None:
|
|
"""LOAD-BEARING (§4.1a, first rung): under a dimension the listing the debate can see does NOT
|
|
name the foreign-dimension document. RED if ``dimension=`` is dropped anywhere between
|
|
``run_project`` and ``directory_listing``.
|
|
|
|
Asserts on the FILE NAME rather than the body sentinel, because a listing carries names and
|
|
sizes, never bodies — an assert on the sentinel would be green against every implementation
|
|
and would prove nothing (the vacuous form this repo keeps measuring)."""
|
|
bundle_dir = _bundle_with_asfalt_file(tmp_path)
|
|
|
|
scoped = await _listing_the_debate_can_see(bundle_dir, _ENERGY_DIM)
|
|
assert "asfalt-note.md" not in scoped, (
|
|
"another dimension's document is listed to the agents — the tools are not dimension-scoped"
|
|
)
|
|
|
|
|
|
async def test_no_dimension_leaves_context_unscoped(tmp_path) -> None:
|
|
"""CAUSALITY CONTROL: with ``dimension=None`` the asfalt document IS listed — proving its
|
|
absence above is caused by the dimension scope, not by the file being unreachable."""
|
|
bundle_dir = _bundle_with_asfalt_file(tmp_path)
|
|
|
|
unscoped = await _listing_the_debate_can_see(bundle_dir, None)
|
|
assert "asfalt-note.md" in unscoped, (
|
|
"the asfalt file is unreachable even without a filter — the control does not prove causality"
|
|
)
|