feat(toolbox): the judgement through the same door -- validate-proposal, verdict-key, capture-verdict
B's premise applied to the three steps that DECIDE a proposal: a proposal authored outside po -- by a human, or by an agent that is not po -- now meets the blocking deterministic gate, mints the learning key, and is captured as a Verdict, all without a chat client on the way. Three thin adapters, no second implementation. The reason is the one the first four doors were built on, but it bites harder here: the refusal SENTENCE is fed back verbatim into the next attempt by step 5, so a door that reworded it would break the repair loop while still looking correct. The probes assert the sentence, not a substring two stages share. One measurement decided a design detail. The IR writes whole magnitudes as JSON integers (30000), the run path carries the pydantic float, and verdicts._mint_id hashes the raw value -- so minting from the undeclared JSON would hand out a DIFFERENT verdict id than the debate does for the same proposal. The door therefore reads the proposal through SavingsProposal and feeds model_dump() to the public features_from_ir; the probe pins both forms and asserts they differ, so the shortcut cannot come back silently. A blocked proposal exits 3, carrying the verdict rather than an exception envelope. "You asked right and the answer is no" is the same fact whether a file was missing or a claim was infeasible, and a caller that only reads the exit code must not see a blocked proposal as a cleared one. Fasit outside the door in every arm: the base's own checked-in golden suite (written before the toolbox existed, so it cannot have been fitted to it), a cost baseline authored in the test, a method cap computed by hand from the fixture, and the public minting rule. Each refusal arm has an rc-0 control on an argv that would otherwise be accepted. STATED LIMIT: the input-grounding stage (P7, stage 0b) has no flag here. It falsifies a proposal against the rendered prompt the model received, and an outside caller has no such prompt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
c660bd2a82
commit
368367e1c5
4 changed files with 408 additions and 14 deletions
|
|
@ -34,6 +34,11 @@ Python ≥3.10. MAF (`agent-framework-core` 1.16.0, `-orchestrations` 1.1.1 —
|
||||||
`costsim`/`hitl`/`preflight` er operatørverktøy, ikke produktets inngang, og hvert navn her er et
|
`costsim`/`hitl`/`preflight` er operatørverktøy, ikke produktets inngang, og hvert navn her er et
|
||||||
navn frysen må bære. Verktøykassen er der fordi den er det ENE de to andre ikke kan brukes til:
|
navn frysen må bære. Verktøykassen er der fordi den er det ENE de to andre ikke kan brukes til:
|
||||||
hver vei gjennom `run` bygger en chatklient, og stegene den bygger på trenger ingen modell.
|
hver vei gjennom `run` bygger en chatklient, og stegene den bygger på trenger ingen modell.
|
||||||
|
Underkommandoene er stegenes egne navn (`navigate-bundle`, `cost-baseline`, `retrieve-chunks`,
|
||||||
|
`prepass-admit`, `validate-proposal`, `verdict-key`, `capture-verdict`) og hver er en TYNN
|
||||||
|
adapter over den funksjonen kjørestien kaller — aldri en andre implementasjon. Exit 0 kjørte,
|
||||||
|
2 feil kall, 3 NEKTET med grunnen navngitt; et forslag den deterministiske validatoren BLOKKERER
|
||||||
|
er også 3, og bærer dommen (ordrett grunn + stadiet) framfor en unntaks-konvolutt.
|
||||||
Pinnet av `tests/test_console_entry_points.py` mot den INSTALLERTE
|
Pinnet av `tests/test_console_entry_points.py` mot den INSTALLERTE
|
||||||
distribusjonens metadata, ikke mot TOML-en: en `[project.scripts]`-linje som aldri er `uv sync`-et
|
distribusjonens metadata, ikke mot TOML-en: en `[project.scripts]`-linje som aldri er `uv sync`-et
|
||||||
er en påstand, ikke en kommando.
|
er en påstand, ikke en kommando.
|
||||||
|
|
|
||||||
16
README.md
16
README.md
|
|
@ -100,10 +100,24 @@ uv run portfolio-optimiser-toolbox cost-baseline --bundle-dir <base> --project-i
|
||||||
|
|
||||||
# Admit (or refuse, by name) a declared pre-pass cut before it may shape a run
|
# Admit (or refuse, by name) a declared pre-pass cut before it may shape a run
|
||||||
uv run portfolio-optimiser-toolbox prepass-admit --payload <cut.json> --bundle-dir <base>
|
uv run portfolio-optimiser-toolbox prepass-admit --payload <cut.json> --bundle-dir <base>
|
||||||
|
|
||||||
|
# Run the blocking deterministic gate on a proposal written OUTSIDE the framework.
|
||||||
|
# Exit 3 carries the verdict: the verbatim refusal sentence and the stage that wrote it.
|
||||||
|
uv run portfolio-optimiser-toolbox validate-proposal \
|
||||||
|
--proposal shared/examples/bygg-energi-mikro/validator-input.json \
|
||||||
|
--cost-baseline <project-prices.json>
|
||||||
|
|
||||||
|
# The learning key a verdict on that proposal will arrive under — without deciding anything
|
||||||
|
uv run portfolio-optimiser-toolbox verdict-key --proposal <proposal.json>
|
||||||
|
|
||||||
|
# Mint an expert's judgement into the Verdict the store holds (stdout is the on-disk form)
|
||||||
|
uv run portfolio-optimiser-toolbox capture-verdict --proposal <proposal.json> \
|
||||||
|
--decision approved --rationale "why"
|
||||||
```
|
```
|
||||||
|
|
||||||
Each subcommand calls the same function the run path calls — not a copy of it. That is what makes
|
Each subcommand calls the same function the run path calls — not a copy of it. That is what makes
|
||||||
the answers you get here the answers the debate gets.
|
the answers you get here the answers the debate gets: a proposal a human wrote meets the same
|
||||||
|
stages, in the same order, with the same sentence, as one an agent produced.
|
||||||
|
|
||||||
Verify the install by running the whole suite from the clean clone:
|
Verify the install by running the whole suite from the clean clone:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -2,15 +2,18 @@
|
||||||
|
|
||||||
This is B's premise made callable. The framework's own CLI (``portfolio_optimiser.run``) drives a
|
This is B's premise made callable. The framework's own CLI (``portfolio_optimiser.run``) drives a
|
||||||
MAF debate and therefore constructs a chat client on every path through it; an outside caller —
|
MAF debate and therefore constructs a chat client on every path through it; an outside caller —
|
||||||
a human at a terminal, or an agent that is NOT po — cannot reach ``navigate_bundle`` or
|
a human at a terminal, or an agent that is NOT po — cannot reach ``navigate_bundle``,
|
||||||
``retrieve_chunks`` through that door without paying for a model. The steps themselves need no
|
``retrieve_chunks`` or ``validate_proposal`` through that door without paying for a model. The steps themselves need no
|
||||||
model at all. This module exposes exactly those steps, and nothing else.
|
model at all. This module exposes exactly those steps, and nothing else.
|
||||||
|
|
||||||
**One CLI, four subcommands, one core call each.** Each subcommand parses arguments, calls the
|
**One CLI, one core call per subcommand.** Each subcommand parses arguments, calls the
|
||||||
SAME function the run path calls, writes the result as JSON on stdout, and returns an exit code
|
SAME function the run path calls, writes the result as JSON on stdout, and returns an exit code
|
||||||
that says what happened: ``0`` the step ran, ``2`` the call was malformed (argparse), ``3`` the
|
that says what happened: ``0`` the step ran, ``2`` the call was malformed (argparse), ``3`` the
|
||||||
step refused and the refusal is named in the JSON. There is no fourth code and no silent zero —
|
step refused and the refusal is named in the JSON. There is no fourth code and no silent zero —
|
||||||
the exit code is the only signal a calling agent has before it reads a byte.
|
the exit code is the only signal a calling agent has before it reads a byte. A proposal the
|
||||||
|
deterministic gate BLOCKS is a ``3`` as well, and carries the verdict rather than an exception:
|
||||||
|
"you asked right and the answer is no" is the same fact whether a file was missing or a claim
|
||||||
|
was infeasible, and a caller that only reads rc must not see a blocked proposal as a cleared one.
|
||||||
|
|
||||||
**No re-implementation, and that is the load-bearing part.** Every handler below is a thin
|
**No re-implementation, and that is the load-bearing part.** Every handler below is a thin
|
||||||
adapter: it converts strings to the types the core function already takes and converts what came
|
adapter: it converts strings to the types the core function already takes and converts what came
|
||||||
|
|
@ -30,10 +33,26 @@ import argparse
|
||||||
import json
|
import json
|
||||||
import sys
|
import sys
|
||||||
from collections.abc import Mapping, Sequence
|
from collections.abc import Mapping, Sequence
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from portfolio_optimiser import okf, prepass
|
from portfolio_optimiser import okf, prepass
|
||||||
|
from portfolio_optimiser.contracts import FeedbackContract
|
||||||
from portfolio_optimiser.datasource import retrieve_chunks
|
from portfolio_optimiser.datasource import retrieve_chunks
|
||||||
|
from portfolio_optimiser.ir import SavingsProposal
|
||||||
|
from portfolio_optimiser.validator import (
|
||||||
|
ValidatedProposal,
|
||||||
|
rejection_stage,
|
||||||
|
validate_proposal,
|
||||||
|
)
|
||||||
|
from portfolio_optimiser.verdicts import (
|
||||||
|
ProposalFeatures,
|
||||||
|
capture_verdict,
|
||||||
|
features_from_ir,
|
||||||
|
verdict_key,
|
||||||
|
verdict_to_dict,
|
||||||
|
)
|
||||||
|
|
||||||
__all__ = ["main"]
|
__all__ = ["main"]
|
||||||
|
|
||||||
|
|
@ -48,6 +67,19 @@ REFUSED = 3
|
||||||
_REFUSALS = (ValueError, FileNotFoundError, OSError)
|
_REFUSALS = (ValueError, FileNotFoundError, OSError)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Refused:
|
||||||
|
"""A step that RAN, whose answer is NO — carried out with the full result, not an error.
|
||||||
|
|
||||||
|
``validate_proposal`` returning a ``Rejection`` is the deterministic gate doing its job, so
|
||||||
|
the payload is the step's own verdict (decision, the verbatim reason, the stage that wrote
|
||||||
|
it) and not an exception envelope. The exit code is still ``REFUSED``, because a calling
|
||||||
|
agent reads the code before it reads a byte, and a blocked proposal that answered ``0``
|
||||||
|
would be indistinguishable from a validated one to everything that only checks rc."""
|
||||||
|
|
||||||
|
payload: Mapping[str, Any]
|
||||||
|
|
||||||
|
|
||||||
def navigate_bundle_command(args: argparse.Namespace) -> Mapping[str, Any]:
|
def navigate_bundle_command(args: argparse.Namespace) -> Mapping[str, Any]:
|
||||||
"""``navigate-bundle`` — open a knowledge base and report what navigation reached.
|
"""``navigate-bundle`` — open a knowledge base and report what navigation reached.
|
||||||
|
|
||||||
|
|
@ -111,8 +143,120 @@ def prepass_admit_command(args: argparse.Namespace) -> Mapping[str, Any]:
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _proposal_from(path: str) -> SavingsProposal:
|
||||||
|
"""The proposal IR, read the ONE way the repository reads it: Pydantic over the same
|
||||||
|
``SavingsProposal`` the run path constructs.
|
||||||
|
|
||||||
|
Deliberately not ``json.loads`` straight into the feature mapping. The IR writes whole
|
||||||
|
magnitudes as JSON integers (``30000``), the run path carries the pydantic FLOAT, and
|
||||||
|
``verdicts._mint_id`` hashes the raw value — so a door that minted from the undeclared JSON
|
||||||
|
would hand out a DIFFERENT verdict id than the debate does for the same proposal, which is
|
||||||
|
the one thing the learning key may not do (A5)."""
|
||||||
|
return SavingsProposal.model_validate_json(Path(path).read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def _method_caps(path: str) -> Mapping[str, float]:
|
||||||
|
"""The method-cap registry as DATA (F8): measure type -> the method-scoped max fraction.
|
||||||
|
|
||||||
|
Read as the registry the core function already takes, never merged with the built-in one —
|
||||||
|
``validate_proposal`` treats a supplied registry as the whole registry, and a door that
|
||||||
|
quietly unioned them would apply a cap the caller did not ask for."""
|
||||||
|
caps = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||||
|
if not isinstance(caps, dict):
|
||||||
|
raise ValueError(f"method caps must be an object of measure -> fraction, got {type(caps)}")
|
||||||
|
return {str(measure): float(fraction) for measure, fraction in caps.items()}
|
||||||
|
|
||||||
|
|
||||||
|
def _features_of(proposal: SavingsProposal) -> ProposalFeatures:
|
||||||
|
"""The run path's own IR -> features mapping, reached through its PUBLIC name.
|
||||||
|
|
||||||
|
``verdicts.features_from_ir`` is public for exactly this reason (A5, one minting rule), and
|
||||||
|
it is fed ``model_dump()`` rather than the file's own dict so the magnitudes are the
|
||||||
|
validated floats the debate mints from."""
|
||||||
|
return features_from_ir(proposal.model_dump())
|
||||||
|
|
||||||
|
|
||||||
|
def validate_proposal_command(args: argparse.Namespace) -> Mapping[str, Any] | Refused:
|
||||||
|
"""``validate-proposal`` — the blocking deterministic gate, method cap included.
|
||||||
|
|
||||||
|
The whole of B rests on this one: a proposal authored OUTSIDE po — by a human, or by an
|
||||||
|
agent that is not po — must meet the same stages, in the same order, with the same verbatim
|
||||||
|
refusal sentence, as one a MAF agent produced. The sentence matters as much as the verdict:
|
||||||
|
Step 5 feeds it back into the next attempt, so a door that reworded it would break the
|
||||||
|
repair loop while still looking correct.
|
||||||
|
|
||||||
|
``cost_baseline_anchored`` is carried out rather than implied. Stage 0 only runs when a
|
||||||
|
baseline was given, and an UNANCHORED "validated" means something much weaker than an
|
||||||
|
anchored one — four paid rounds were measured reasoning about magnitudes nobody priced.
|
||||||
|
|
||||||
|
STATED LIMIT: the input-grounding stage (P7, stage 0b) has no flag here. It falsifies a
|
||||||
|
proposal against the rendered prompt the model received, and an outside caller has no such
|
||||||
|
prompt; a file pretending to be one would be a different measurement wearing its name."""
|
||||||
|
proposal = _proposal_from(args.proposal)
|
||||||
|
baseline = (
|
||||||
|
None if args.cost_baseline is None else okf.load_cost_baseline_file(args.cost_baseline)
|
||||||
|
)
|
||||||
|
caps = None if args.method_caps is None else _method_caps(args.method_caps)
|
||||||
|
result = validate_proposal(proposal, baseline=baseline, method_caps=caps)
|
||||||
|
common = {
|
||||||
|
"project_id": proposal.project_id,
|
||||||
|
"measure": proposal.measure,
|
||||||
|
"claimed_saving_nok": proposal.claimed_saving_nok,
|
||||||
|
"cost_baseline_anchored": baseline is not None,
|
||||||
|
}
|
||||||
|
if isinstance(result, ValidatedProposal):
|
||||||
|
return {
|
||||||
|
"decision": "validated",
|
||||||
|
**common,
|
||||||
|
"nominal_feasible": result.nominal_feasible,
|
||||||
|
"p10": result.p10,
|
||||||
|
"p50": result.p50,
|
||||||
|
"p90": result.p90,
|
||||||
|
}
|
||||||
|
return Refused(
|
||||||
|
{
|
||||||
|
"decision": "rejected",
|
||||||
|
**common,
|
||||||
|
"reason": result.reason,
|
||||||
|
# Which falsifier wrote the sentence, named by the module that owns the wordings —
|
||||||
|
# never re-derived here, or the door and the validator could disagree about one run.
|
||||||
|
"stage": rejection_stage(result.reason),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def verdict_key_command(args: argparse.Namespace) -> Mapping[str, Any]:
|
||||||
|
"""``verdict-key`` — the id an expert verdict on THIS proposal will arrive under.
|
||||||
|
|
||||||
|
Available without capturing a decision nobody has made yet: a caller can stamp its own
|
||||||
|
artefact with the key, hand the proposal to a human, and find the verdict again when it
|
||||||
|
comes back."""
|
||||||
|
features = _features_of(_proposal_from(args.proposal))
|
||||||
|
return {
|
||||||
|
"verdict_id": verdict_key(features),
|
||||||
|
"affected_codes": sorted(features.affected_codes),
|
||||||
|
"measure_type": features.measure_type,
|
||||||
|
"claimed_saving_nok": features.claimed_saving_nok,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def capture_verdict_command(args: argparse.Namespace) -> Mapping[str, Any]:
|
||||||
|
"""``capture-verdict`` — an expert's judgement, minted into the Verdict the store holds.
|
||||||
|
|
||||||
|
The decision vocabulary is ``FeedbackContract``'s, which is what step 1 of the run path
|
||||||
|
already checks a supplied verdict against: a half-given or invented judgement is refused BY
|
||||||
|
FIELD NAME rather than completed on the expert's behalf. The emitted JSON is
|
||||||
|
``verdict_to_dict`` — the same form ``write_verdict`` puts on disk — so a caller that
|
||||||
|
redirects stdout into the inbox folder has authored a verdict the next run will read."""
|
||||||
|
given = FeedbackContract(decision=args.decision, rationale=args.rationale)
|
||||||
|
verdict = capture_verdict(
|
||||||
|
_features_of(_proposal_from(args.proposal)), given.decision, given.rationale
|
||||||
|
)
|
||||||
|
return verdict_to_dict(verdict)
|
||||||
|
|
||||||
|
|
||||||
def build_parser() -> argparse.ArgumentParser:
|
def build_parser() -> argparse.ArgumentParser:
|
||||||
"""The four doors, each registered by name.
|
"""The doors, each registered by name.
|
||||||
|
|
||||||
``required=True`` on the subparsers: a toolbox invoked with no command must be a usage error,
|
``required=True`` on the subparsers: a toolbox invoked with no command must be a usage error,
|
||||||
never a zero. Measured as a class in this repository — an exit 0 for a call that did nothing
|
never a zero. Measured as a class in this repository — an exit 0 for a call that did nothing
|
||||||
|
|
@ -140,11 +284,30 @@ def build_parser() -> argparse.ArgumentParser:
|
||||||
slipp.add_argument("--payload", required=True, help="the producer's payload JSON")
|
slipp.add_argument("--payload", required=True, help="the producer's payload JSON")
|
||||||
slipp.add_argument("--bundle-dir", required=True, help="the base the cut claims to be of")
|
slipp.add_argument("--bundle-dir", required=True, help="the base the cut claims to be of")
|
||||||
slipp.add_argument("--dimension", default=None, help="restrict admission to one dimension")
|
slipp.add_argument("--dimension", default=None, help="restrict admission to one dimension")
|
||||||
|
|
||||||
|
doem = sub.add_parser("validate-proposal", help="run the blocking deterministic gate")
|
||||||
|
doem.add_argument("--proposal", required=True, help="the proposal IR as JSON")
|
||||||
|
doem.add_argument(
|
||||||
|
"--cost-baseline",
|
||||||
|
default=None,
|
||||||
|
help="the project's own priced lines; without it stage 0 never runs",
|
||||||
|
)
|
||||||
|
doem.add_argument(
|
||||||
|
"--method-caps", default=None, help="method-cap registry JSON (measure -> fraction)"
|
||||||
|
)
|
||||||
|
|
||||||
|
noekkel = sub.add_parser("verdict-key", help="the learning key a verdict will arrive under")
|
||||||
|
noekkel.add_argument("--proposal", required=True, help="the proposal IR as JSON")
|
||||||
|
|
||||||
|
fang = sub.add_parser("capture-verdict", help="mint an expert judgement into a Verdict")
|
||||||
|
fang.add_argument("--proposal", required=True, help="the proposal IR as JSON")
|
||||||
|
fang.add_argument("--decision", required=True, help="the expert's decision")
|
||||||
|
fang.add_argument("--rationale", required=True, help="why — carried into the store verbatim")
|
||||||
return parser
|
return parser
|
||||||
|
|
||||||
|
|
||||||
def dispatch(args: argparse.Namespace) -> Any:
|
def dispatch(args: argparse.Namespace) -> Any:
|
||||||
"""Name the four handlers, one branch each — deliberately not a ``set_defaults(handler=…)``.
|
"""Name each handler, one branch each — deliberately not a ``set_defaults(handler=…)``.
|
||||||
|
|
||||||
The dispatch table argparse offers is one line shorter and hides the only thing a reader of
|
The dispatch table argparse offers is one line shorter and hides the only thing a reader of
|
||||||
this module wants to see: which command reaches which run-path step. B-gate row 1 asks the
|
this module wants to see: which command reaches which run-path step. B-gate row 1 asks the
|
||||||
|
|
@ -158,6 +321,12 @@ def dispatch(args: argparse.Namespace) -> Any:
|
||||||
return retrieve_chunks_command(args)
|
return retrieve_chunks_command(args)
|
||||||
if args.command == "prepass-admit":
|
if args.command == "prepass-admit":
|
||||||
return prepass_admit_command(args)
|
return prepass_admit_command(args)
|
||||||
|
if args.command == "validate-proposal":
|
||||||
|
return validate_proposal_command(args)
|
||||||
|
if args.command == "verdict-key":
|
||||||
|
return verdict_key_command(args)
|
||||||
|
if args.command == "capture-verdict":
|
||||||
|
return capture_verdict_command(args)
|
||||||
raise RuntimeError(f"unregistered command {args.command!r}") # pragma: no cover - argparse
|
raise RuntimeError(f"unregistered command {args.command!r}") # pragma: no cover - argparse
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -181,9 +350,12 @@ def main(argv: Sequence[str] | None = None) -> int:
|
||||||
)
|
)
|
||||||
print()
|
print()
|
||||||
return REFUSED
|
return REFUSED
|
||||||
|
code = 0
|
||||||
|
if isinstance(result, Refused):
|
||||||
|
result, code = result.payload, REFUSED
|
||||||
json.dump(result, sys.stdout, ensure_ascii=False, indent=2)
|
json.dump(result, sys.stdout, ensure_ascii=False, indent=2)
|
||||||
print()
|
print()
|
||||||
return 0
|
return code
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__": # pragma: no cover - dekket av subprosess-probene
|
if __name__ == "__main__": # pragma: no cover - dekket av subprosess-probene
|
||||||
|
|
|
||||||
|
|
@ -25,6 +25,12 @@ import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from portfolio_optimiser import datasource
|
from portfolio_optimiser import datasource
|
||||||
|
from portfolio_optimiser.verdicts import (
|
||||||
|
ProposalFeatures,
|
||||||
|
capture_verdict,
|
||||||
|
verdict_key,
|
||||||
|
verdict_to_dict,
|
||||||
|
)
|
||||||
|
|
||||||
_REPO = Path(__file__).resolve().parents[1]
|
_REPO = Path(__file__).resolve().parents[1]
|
||||||
_MIKRO = _REPO / "shared" / "examples" / "bygg-energi-mikro"
|
_MIKRO = _REPO / "shared" / "examples" / "bygg-energi-mikro"
|
||||||
|
|
@ -87,9 +93,7 @@ def test_navigate_bundle_refuses_a_directory_that_is_no_bundle_with_a_named_code
|
||||||
|
|
||||||
|
|
||||||
def test_cost_baseline_from_outside_derives_exactly_what_the_priced_table_says() -> None:
|
def test_cost_baseline_from_outside_derives_exactly_what_the_priced_table_says() -> None:
|
||||||
proc = _toolbox(
|
proc = _toolbox("cost-baseline", "--bundle-dir", str(_PRISSKJEMA), "--project-id", "K2")
|
||||||
"cost-baseline", "--bundle-dir", str(_PRISSKJEMA), "--project-id", "K2"
|
|
||||||
)
|
|
||||||
assert proc.returncode == 0, proc.stderr
|
assert proc.returncode == 0, proc.stderr
|
||||||
payload = json.loads(proc.stdout)
|
payload = json.loads(proc.stdout)
|
||||||
assert payload["project_id"] == "K2"
|
assert payload["project_id"] == "K2"
|
||||||
|
|
@ -124,9 +128,7 @@ def test_retrieve_chunks_from_outside_matches_the_in_process_seam_exactly() -> N
|
||||||
|
|
||||||
|
|
||||||
def test_retrieve_chunks_refuses_a_top_k_that_asks_for_nothing() -> None:
|
def test_retrieve_chunks_refuses_a_top_k_that_asks_for_nothing() -> None:
|
||||||
proc = _toolbox(
|
proc = _toolbox("retrieve-chunks", "--query", "x", "--docs-dir", str(_MIKRO), "--top-k", "0")
|
||||||
"retrieve-chunks", "--query", "x", "--docs-dir", str(_MIKRO), "--top-k", "0"
|
|
||||||
)
|
|
||||||
assert proc.returncode == 3, proc.stdout
|
assert proc.returncode == 3, proc.stdout
|
||||||
assert "top_k" in json.loads(proc.stdout)["error"]["message"]
|
assert "top_k" in json.loads(proc.stdout)["error"]["message"]
|
||||||
|
|
||||||
|
|
@ -212,3 +214,204 @@ def test_wrong_usage_is_exit_two_and_never_a_silent_zero() -> None:
|
||||||
assert _toolbox().returncode == 2
|
assert _toolbox().returncode == 2
|
||||||
assert _toolbox("ingen-slik-kommando").returncode == 2
|
assert _toolbox("ingen-slik-kommando").returncode == 2
|
||||||
assert _toolbox("navigate-bundle").returncode == 2
|
assert _toolbox("navigate-bundle").returncode == 2
|
||||||
|
|
||||||
|
|
||||||
|
# --- dommen: validate-proposal / verdict-key / capture-verdict ----------------------------------
|
||||||
|
#
|
||||||
|
# Steg 3 i B-gatens kø: de tre stegene som avgjør et forslag. Samme dør, samme regel om fasit —
|
||||||
|
# den hentes UTENFOR døren i hver arm: basens egen innsjekkede golden-suite (validate-proposal),
|
||||||
|
# den publike myntingsregelen `verdicts.verdict_key` (verdict-key) og `verdicts.verdict_to_dict`
|
||||||
|
# over en dom bygget i testen (capture-verdict).
|
||||||
|
|
||||||
|
_IR = _MIKRO / "validator-input.json"
|
||||||
|
_GOLDEN = json.loads((_MIKRO / "golden.json").read_text(encoding="utf-8"))["validator"]
|
||||||
|
|
||||||
|
#: Mikro-basens IR, transkribert fra `validator-input.json` — ikke lest ut av den. En arm som
|
||||||
|
#: leste fila ville flyttet seg med den, og da måler den ikke lenger at døren gir DISSE tallene.
|
||||||
|
_IR_CODE = "ENERGI-TOTAL-EL"
|
||||||
|
_IR_PROJECT = "BYGG-KONTOR-NORD"
|
||||||
|
_IR_QUANTITY = 300000.0
|
||||||
|
_IR_UNIT_COST = 1.0
|
||||||
|
_IR_CLAIMED = 30000.0
|
||||||
|
|
||||||
|
|
||||||
|
def _measure() -> str:
|
||||||
|
"""Tiltakets tekst, lest fra basen: den ER lang og står ordrett i domsnøkkelens kanoniske form,
|
||||||
|
så en kopi her ville vært en transkribering som kunne drive fra basen uten at noe ble rødt."""
|
||||||
|
return str(json.loads(_IR.read_text(encoding="utf-8"))["measure"])
|
||||||
|
|
||||||
|
|
||||||
|
def _baseline_file(path: Path, codes: dict[str, tuple[float, float]]) -> str:
|
||||||
|
"""En kostnadsgrunnlags-JSON på PROSJEKTETS form (`okf.load_cost_baseline_file`)."""
|
||||||
|
path.write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"project_id": _IR_PROJECT,
|
||||||
|
"items": {c: {"quantity": q, "unit_cost": u} for c, (q, u) in codes.items()},
|
||||||
|
}
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
return str(path)
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_proposal_from_outside_reproduces_the_bases_own_golden() -> None:
|
||||||
|
"""Fasiten er basens INNSJEKKEDE golden-suite — produsentens frosne utfall av
|
||||||
|
`validate_proposal` på nøyaktig denne IR-en, med seedet Monte Carlo. Den ble skrevet før
|
||||||
|
verktøykassen fantes, så den kan ikke ha blitt tilpasset døren."""
|
||||||
|
proc = _toolbox("validate-proposal", "--proposal", str(_IR))
|
||||||
|
assert proc.returncode == 0, proc.stderr
|
||||||
|
payload = json.loads(proc.stdout)
|
||||||
|
assert payload["decision"] == "validated"
|
||||||
|
assert payload["nominal_feasible"] == _GOLDEN["nominal_feasible"]
|
||||||
|
assert payload["p10"] == _GOLDEN["p10"]
|
||||||
|
assert payload["p50"] == _GOLDEN["p50"]
|
||||||
|
assert payload["p90"] == _GOLDEN["p90"]
|
||||||
|
assert payload["claimed_saving_nok"] == _GOLDEN["claimed_saving_nok"]
|
||||||
|
# Uten grunnlag kjørte stadium 0 ALDRI, og døren sier det framfor å la kalleren tro at
|
||||||
|
# tallene ble holdt mot prosjektets egne kostnadslinjer.
|
||||||
|
assert payload["cost_baseline_anchored"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_proposal_refuses_a_code_the_cost_baseline_never_priced(tmp_path: Path) -> None:
|
||||||
|
"""Stadium 0 gjennom døren, med den ORDRETTE setningen løpet ellers skriver — det er den
|
||||||
|
Steg 5 mater tilbake til neste forsøk, så en dør som omskrev den ville brutt sløyfa."""
|
||||||
|
ukjent = _baseline_file(tmp_path / "annen.json", {"VARME-TOTAL": (1.0, 2.0)})
|
||||||
|
proc = _toolbox("validate-proposal", "--proposal", str(_IR), "--cost-baseline", ukjent)
|
||||||
|
assert proc.returncode == 3, proc.stdout
|
||||||
|
payload = json.loads(proc.stdout)
|
||||||
|
assert payload["decision"] == "rejected"
|
||||||
|
assert payload["reason"] == (
|
||||||
|
f"unknown cost code '{_IR_CODE}': not in project {_IR_PROJECT}'s cost baseline "
|
||||||
|
f"(1 known codes: 'VARME-TOTAL')"
|
||||||
|
)
|
||||||
|
assert payload["stage"] == "stage0-baseline"
|
||||||
|
assert payload["cost_baseline_anchored"] is True
|
||||||
|
# rc-0-kontroll: SAMME forslag mot et grunnlag som priser koden. Uten den måler armen over
|
||||||
|
# ingenting — den ville vært grønn mot en dør som avviste alt.
|
||||||
|
priset = _baseline_file(tmp_path / "priset.json", {_IR_CODE: (_IR_QUANTITY, _IR_UNIT_COST)})
|
||||||
|
ok = _toolbox("validate-proposal", "--proposal", str(_IR), "--cost-baseline", priset)
|
||||||
|
assert ok.returncode == 0, ok.stdout
|
||||||
|
assert json.loads(ok.stdout)["decision"] == "validated"
|
||||||
|
assert json.loads(ok.stdout)["cost_baseline_anchored"] is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_validate_proposal_applies_the_method_cap_through_the_same_door(tmp_path: Path) -> None:
|
||||||
|
"""Metodetaket er DATA (F8), og døren rekker det: et strengere tak enn den generiske P90
|
||||||
|
avviser et forslag P90-stadiet slapp gjennom — samme dør, samme ordrette setning.
|
||||||
|
|
||||||
|
Taket regnes UAVHENGIG her: 0,05 × (300 000 × 1,0) = 15 000, transkribert fra basen."""
|
||||||
|
tak = tmp_path / "tak.json"
|
||||||
|
tak.write_text(json.dumps({_measure(): 0.05}), encoding="utf-8")
|
||||||
|
ventet_tak = 0.05 * _IR_QUANTITY * _IR_UNIT_COST
|
||||||
|
assert ventet_tak == 15000.0
|
||||||
|
proc = _toolbox("validate-proposal", "--proposal", str(_IR), "--method-caps", str(tak))
|
||||||
|
assert proc.returncode == 3, proc.stdout
|
||||||
|
payload = json.loads(proc.stdout)
|
||||||
|
assert payload["reason"] == (
|
||||||
|
f"claimed {_IR_CLAIMED:.0f} exceeds the {_measure()} method cap {ventet_tak:.0f} "
|
||||||
|
f"(stricter than the generic P90)"
|
||||||
|
)
|
||||||
|
assert payload["stage"] == "stage5-method-cap"
|
||||||
|
# rc-0-kontroll: et tak som IKKE er strengere enn kravet slipper samme forslag gjennom.
|
||||||
|
romslig = tmp_path / "romslig.json"
|
||||||
|
romslig.write_text(json.dumps({_measure(): 0.30}), encoding="utf-8")
|
||||||
|
ok = _toolbox("validate-proposal", "--proposal", str(_IR), "--method-caps", str(romslig))
|
||||||
|
assert ok.returncode == 0, ok.stdout
|
||||||
|
assert json.loads(ok.stdout)["decision"] == "validated"
|
||||||
|
|
||||||
|
|
||||||
|
def test_verdict_key_from_outside_is_the_key_the_learning_loop_mints(tmp_path: Path) -> None:
|
||||||
|
"""A5: én publik domsnøkkel. Fasiten er `verdicts.verdict_key` kalt på trekk bygget HER —
|
||||||
|
og armen skiller de to formene som ellers ser like ut: IR-en skriver `30000`, kjørestien
|
||||||
|
sender en pydantic-FLOAT videre, og `_mint_id` hasher den rå verdien. En dør som mintet fra
|
||||||
|
JSON-en slik den står ville gitt en ANNEN nøkkel enn løpet, på samme forslag."""
|
||||||
|
trekk = ProposalFeatures(
|
||||||
|
affected_codes=frozenset({_IR_CODE}),
|
||||||
|
measure_type=_measure(),
|
||||||
|
claimed_saving_nok=_IR_CLAIMED,
|
||||||
|
description=_measure(),
|
||||||
|
)
|
||||||
|
som_heltall = ProposalFeatures(
|
||||||
|
affected_codes=frozenset({_IR_CODE}),
|
||||||
|
measure_type=_measure(),
|
||||||
|
claimed_saving_nok=30000,
|
||||||
|
description=_measure(),
|
||||||
|
)
|
||||||
|
assert verdict_key(trekk) != verdict_key(som_heltall), "de to formene er ikke lenger ulike"
|
||||||
|
|
||||||
|
proc = _toolbox("verdict-key", "--proposal", str(_IR))
|
||||||
|
assert proc.returncode == 0, proc.stderr
|
||||||
|
payload = json.loads(proc.stdout)
|
||||||
|
assert payload["verdict_id"] == verdict_key(trekk)
|
||||||
|
assert payload["verdict_id"] != verdict_key(som_heltall)
|
||||||
|
assert payload["affected_codes"] == [_IR_CODE]
|
||||||
|
assert payload["claimed_saving_nok"] == _IR_CLAIMED
|
||||||
|
|
||||||
|
# Nøkkelen er ingen konstant: et forslag med et annet beløp nøkles annerledes.
|
||||||
|
annet = json.loads(_IR.read_text(encoding="utf-8"))
|
||||||
|
annet["claimed_saving_nok"] = 12345
|
||||||
|
sti = tmp_path / "annet.json"
|
||||||
|
sti.write_text(json.dumps(annet), encoding="utf-8")
|
||||||
|
andre = _toolbox("verdict-key", "--proposal", str(sti))
|
||||||
|
assert andre.returncode == 0, andre.stderr
|
||||||
|
assert json.loads(andre.stdout)["verdict_id"] != payload["verdict_id"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_capture_verdict_from_outside_is_the_verdict_the_store_would_hold() -> None:
|
||||||
|
"""Dommen fanget utenfra skal være NØYAKTIG den `capture_verdict` bygger i løpet — samme
|
||||||
|
felter, samme id, samme serialisering (`verdict_to_dict`, som er formen `write_verdict`
|
||||||
|
legger på disk)."""
|
||||||
|
grunn = "realiseringsgapet er dokumentert i basen; tallene holder"
|
||||||
|
ventet = verdict_to_dict(
|
||||||
|
capture_verdict(
|
||||||
|
ProposalFeatures(
|
||||||
|
affected_codes=frozenset({_IR_CODE}),
|
||||||
|
measure_type=_measure(),
|
||||||
|
claimed_saving_nok=_IR_CLAIMED,
|
||||||
|
description=_measure(),
|
||||||
|
),
|
||||||
|
"approved",
|
||||||
|
grunn,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
proc = _toolbox(
|
||||||
|
"capture-verdict",
|
||||||
|
"--proposal",
|
||||||
|
str(_IR),
|
||||||
|
"--decision",
|
||||||
|
"approved",
|
||||||
|
"--rationale",
|
||||||
|
grunn,
|
||||||
|
)
|
||||||
|
assert proc.returncode == 0, proc.stderr
|
||||||
|
assert json.loads(proc.stdout) == ventet
|
||||||
|
|
||||||
|
|
||||||
|
def test_capture_verdict_refuses_a_decision_the_feedback_contract_does_not_know() -> None:
|
||||||
|
"""Vokabularet er kontraktens (`FeedbackContract`), ikke dørens egen kopi: en halvgitt eller
|
||||||
|
oppfunnet dom avvises med FELTNAVNET, aldri fullført på fagpersonens vegne."""
|
||||||
|
proc = _toolbox(
|
||||||
|
"capture-verdict",
|
||||||
|
"--proposal",
|
||||||
|
str(_IR),
|
||||||
|
"--decision",
|
||||||
|
"kanskje",
|
||||||
|
"--rationale",
|
||||||
|
"x",
|
||||||
|
)
|
||||||
|
assert proc.returncode == 3, proc.stdout
|
||||||
|
melding = json.loads(proc.stdout)["error"]["message"]
|
||||||
|
assert "decision" in melding and "kanskje" in melding
|
||||||
|
# rc-0-kontroll: samme kall med en dom kontrakten KJENNER.
|
||||||
|
ok = _toolbox(
|
||||||
|
"capture-verdict",
|
||||||
|
"--proposal",
|
||||||
|
str(_IR),
|
||||||
|
"--decision",
|
||||||
|
"rejected",
|
||||||
|
"--rationale",
|
||||||
|
"x",
|
||||||
|
)
|
||||||
|
assert ok.returncode == 0, ok.stdout
|
||||||
|
assert json.loads(ok.stdout)["decision"] == "rejected"
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue