feat(row6): a proposal whose approach declared no requirement is unsupported

Stress round 6 validated three falsification arms, and every validated
approach rested only on run-level declarations nobody can attribute to one
approach. declare_requirement now takes a required approach_id (a mandate
id or own-proposal; an unknown id is refused naming the valid ones), and a
ValidatedProposal whose approach has neither a mandate requirement nor a
declaration under its own id becomes validator.Unsupported - a Rejection
subclass carrying the validator's own ruling, reported as `unsupported` in
coverage, the outcome artefact, the settlement and the judge, and never
counted or summed. The rule is active whenever the debate held the
declaration tool, the micro base included; the road and pre-pass paths are
untouched. Declaration quality is not judged, so the rule can be satisfied
by declaring any document the run read.

The v1 gate's row 6 probes pass; its artefact half reads IKKE MÅLT because
stress round 6 predates approach-addressed declarations, and IKKE MÅLT is
never green - it fails the exit code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-17 16:40:54 +02:00
commit 938a1ca30e
23 changed files with 718 additions and 115 deletions

View file

@ -3112,6 +3112,25 @@ Python ≥3.10. MAF (`agent-framework-core` 1.16.0, `-orchestrations` 1.1.1 —
`r761-2025` uten `index.md`) — `--bundle-root` peker da på en utpakket kopi; bevisene for type 1, 3 `r761-2025` uten `index.md`) — `--bundle-root` peker da på en utpakket kopi; bevisene for type 1, 3
og 7 er EKSISTERENDE tester registrert ved node-id, så en omdøping gjør typen rød til registeret og 7 er EKSISTERENDE tester registrert ved node-id, så en omdøping gjør typen rød til registeret
rettes (gatet av en egen arm). rettes (gatet av en egen arm).
- **Et forslag uten tilnærmingens EGEN erklæring kan ikke bære `validated` (rad 6, 17.09):** målt
på stressrunde 6 hadde alle 10 validerte tilnærmingene bare kjørings-erklæringer, som ingen kan
knytte til én tilnærming — og tre falsifiseringsarmer validerte. `declare_requirement` tar derfor
et PÅKREVD `approach_id` (mandatets id-er + `own-proposal`; ukjent id → `UnknownApproach`, en
returnert nekt som navngir de gyldige), og `DeclaredRequirement`/`requirement_payload` bærer det.
I `_evaluate_mandate` blir en `ValidatedProposal` hvis tilnærming verken har `requirement` i
mandatet eller en erklæring under sin egen id til `validator.Unsupported` — en `Rejection`-
SUBKLASSE med validatorens egen `ValidatedProposal` på seg, så hver `isinstance(...,
ValidatedProposal)` sier nei uten å røres, mens de som NAVNGIR statusen (coverage `unsupported`,
`outcome_payload` `outcome_type: "unsupported"` med persentilene, `settle` `UNSUPPORTED`,
`rejection_stage` `unsupported`, dommeren) sjekker klassen FØRST. `validator_decision` forblir
`validated` (den speiler kun validatoren). **Regelen er aktiv nøyaktig når debatten hadde
erklæringsverktøyet** — også på mikro-basen, som har 0 kravnumre (PM-rettelse: ingen
spesialbehandling); veg-stien og pre-pass-stien har ingen trapp og er urørt. Erklæringens
KVALITET dømmes ikke (P22 § 4), så regelen kan spilles ved å erklære et hvilket som helst lest
dokument — uttalt svakhet. Dommeren leser en erklæring under tilnærmingens id som `approach`, en
uten `approach_id` (eldre artefakter) som `run`; v1-gatens rad 6 sier da «IKKE MÅLT», og IKKE
MÅLT feller exit-koden (aldri grønn). Load-bearing MÅLT
(`tests/test_row6_declaration_rule_loadbearing.py` + rad 6-probene), ti mutasjoner alle røde.
- **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet. - **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet.
- Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase. - Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase.

View file

@ -495,9 +495,17 @@ when the seam is detached, so the loop cannot silently degrade into theater.
**Naming the requirement that binds a direction.** Whoever navigates a knowledge base — the **Naming the requirement that binds a direction.** Whoever navigates a knowledge base — the
exploration's hypothesiser and, since the debate started navigating, the proposer — is asked to exploration's hypothesiser and, since the debate started navigating, the proposer — is asked to
name the ONE requirement that binds the direction it commits to, and to declare it with name the ONE requirement that binds the direction it commits to, and to declare it with
`declare_requirement(bundle_id, path, ref)` after reading it. A declaration naming a document `declare_requirement(bundle_id, path, ref, approach_id)` after reading it. A declaration naming
the run never opened is **refused by name** (`RequirementNotRead`) and comes back as a turn the a document the run never opened is **refused by name** (`RequirementNotRead`) and comes back as a
model can correct by going and reading it; nothing is recorded until it has. A marked hypothesis turn the model can correct by going and reading it; nothing is recorded until it has.
`approach_id` names the approach the requirement binds — one of the mandate's ids, or
`own-proposal` — and an id the run was not commissioned with is refused with the valid ones named
(`UnknownApproach`). **A proposal whose approach declared nothing cannot be `validated`:** when
the debate held the declaration tool, a candidate whose numbers pass every stage but whose
approach has neither a requirement written into the mandate nor a declaration filed under its
own id is reported `unsupported` — not rejected (the numbers held), not validated (nothing was
named as its ground), never counted or summed. The declaration's *quality* is not judged; a run
can satisfy the rule by declaring any document it read, and that is a known weakness. A marked hypothesis
therefore carries `"requirement": {"path": ..., "ref": ...}`, or an explicit `null` together with therefore carries `"requirement": {"path": ..., "ref": ...}`, or an explicit `null` together with
`"why_none"` — a base that holds no requirement for a direction is a finding worth stating, and `"why_none"` — a base that holds no requirement for a direction is a finding worth stating, and
the field is never simply omitted. Where an approach carries one, the proposer's prompt names it the field is never simply omitted. Where an approach carries one, the proposer's prompt names it

View file

@ -106,11 +106,14 @@ Mandate outcome
Validated: 1 of 3 approaches — best 30000 NOK (the outcome this run carries) Validated: 1 of 3 approaches — best 30000 NOK (the outcome this run carries)
``` ```
De tre statusene betyr forskjellige ting, og forskjellen er poenget: De fire statusene betyr forskjellige ting, og forskjellen er poenget:
- **VALIDATED** — tallene holdt. Beløpet er det validatoren slapp gjennom. - **VALIDATED** — tallene holdt. Beløpet er det validatoren slapp gjennom.
- **REJECTED** — tilnærmingen ble vurdert, og falt på tallene. Grunnen står der; det er den du kan - **REJECTED** — tilnærmingen ble vurdert, og falt på tallene. Grunnen står der; det er den du kan
gjøre noe med (ofte er den «regnet mot feil grunnlag», ikke «dårlig idé»). gjøre noe med (ofte er den «regnet mot feil grunnlag», ikke «dårlig idé»).
- **UNSUPPORTED** — tallene holdt, men ingen krav i kunnskapsbasen ble erklært som bindende for
akkurat denne tilnærmingen. Den telles ikke som validert og summeres ikke. Skriv kravet inn i
bestillingen (`requirement` på tilnærmingen), eller la kjøringen erklære det.
- **NOT EVALUATED** — kjøringen rakk den aldri. Den er **ikke** avvist, og den er ikke glemt. - **NOT EVALUATED** — kjøringen rakk den aldri. Den er **ikke** avvist, og den er ikke glemt.
En tilnærming du bestilte forsvinner aldri i stillhet. Det er hele grunnen til at oppgjøret finnes. En tilnærming du bestilte forsvinner aldri i stillhet. Det er hele grunnen til at oppgjøret finnes.

View file

@ -42,6 +42,8 @@ _AI_LINE_MIN = 30
GREEN = "GRØNN" GREEN = "GRØNN"
RED = "RØD" RED = "RØD"
DIAGNOSIS = "DIAGNOSE" DIAGNOSIS = "DIAGNOSE"
#: A row whose evidence could not be read. Never green: on a failing row it fails the exit code.
NOT_MEASURED = "IKKE MÅLT"
ROUNDS_CONTRACT = """\ ROUNDS_CONTRACT = """\
Rundekatalogen (--rounds-dir) har fast form: Rundekatalogen (--rounds-dir) har fast form:
@ -498,6 +500,8 @@ class StressMeasure:
commissioned: int = 0 commissioned: int = 0
where: str = "" where: str = ""
missing: str = "" missing: str = ""
#: Declarations with no ``approach_id`` — written before the rule; the row cannot be measured.
unaddressed: int = 0
undeclared_ids: tuple[str, ...] = field(default=()) undeclared_ids: tuple[str, ...] = field(default=())
@ -538,6 +542,7 @@ def measure_stress(
approaches = [a for v in verdicts for a in v.approaches] approaches = [a for v in verdicts for a in v.approaches]
validated = [a for a in approaches if a.status == "validated"] validated = [a for a in approaches if a.status == "validated"]
undeclared = [a for a in validated if a.requirement_source != "approach"] undeclared = [a for a in validated if a.requirement_source != "approach"]
unaddressed = sum(v.unaddressed_declarations for v in verdicts)
commissioned = sum( commissioned = sum(
len(load_mandate(repo_root / c / "mandate.json").approaches) for c in contexts len(load_mandate(repo_root / c / "mandate.json").approaches) for c in contexts
) )
@ -549,6 +554,7 @@ def measure_stress(
rows=len(approaches), rows=len(approaches),
commissioned=commissioned, commissioned=commissioned,
where=str(stress_root), where=str(stress_root),
unaddressed=unaddressed,
undeclared_ids=tuple(a.approach_id for a in undeclared), undeclared_ids=tuple(a.approach_id for a in undeclared),
) )
@ -556,29 +562,54 @@ def measure_stress(
def score_undeclared( def score_undeclared(
probes: Sequence[str], outcomes: Mapping[str, str], m: StressMeasure, label: str probes: Sequence[str], outcomes: Mapping[str, str], m: StressMeasure, label: str
) -> Row: ) -> Row:
"""GREEN only when every probe passes AND the artefacts were measured with k = 0. Evidence that
could not be read is IKKE MÅLT never green, and it fails the exit code like red does."""
failing = [ failing = [
f"{n.split('::')[-1]}={outcomes.get(n, 'missing')}" f"{n.split('::')[-1]}={outcomes.get(n, 'missing')}"
for n in probes for n in probes
if outcomes.get(n) != "passed" if outcomes.get(n) != "passed"
] ]
if not probes:
failing.append("ingen probe registrert")
title = "6 validert UTEN erklært krav (tilnærmingens egen)" title = "6 validert UTEN erklært krav (tilnærmingens egen)"
exceptions = [f"probe {x}" for x in failing] exceptions = [f"probe {x}" for x in failing]
probe_state = "prober røde" if failing else "prober grønne"
k: int | None = None
n: int | None = None
diagnostics: tuple[str, ...] = ()
if m.missing: if m.missing:
reason = f"{label}: ikke målt, artefakter mangler ({m.missing})" reason = f"{probe_state}; {label}: ikke målt, artefakter mangler ({m.missing})"
k: int | None = None elif m.unaddressed:
n: int | None = None reason = (
f"{probe_state}; {label}: ikke målt: artefaktene er eldre enn regelen "
f"(approach_id mangler på {m.unaddressed} erklæring(er))"
)
diagnostics = (
f"før regelen: {m.undeclared} av {m.validated} validerte uten tilnærmingens egen "
"erklæring — regelen ville gjort dem unsupported, men modellen fikk aldri spørsmålet",
)
else: else:
k, n = m.undeclared, m.validated k, n = m.undeclared, m.validated
reason = ( reason = (
f"{label} ({m.where}): {k} av {n} validerte uten erklæring fra tilnærmingen; " f"{probe_state}; {label} ({m.where}): {k} av {n} validerte uten erklæring fra "
f"{m.undeclared_anywhere} uten noen erklæring i kjøringen" f"tilnærmingen; {m.undeclared_anywhere} uten noen erklæring i kjøringen"
) )
exceptions += [f"validert uten erklæring: {a}" for a in m.undeclared_ids] exceptions += [f"validert uten erklæring: {a}" for a in m.undeclared_ids]
red = bool(failing) or bool(k) if failing or k:
if not probes: status = RED
red, exceptions = True, [*exceptions, "ingen probe registrert"] elif k is None:
status = NOT_MEASURED
else:
status = GREEN
return Row( return Row(
"undeclared", title, k, n, RED if red else GREEN, reason, exceptions=tuple(exceptions) "undeclared",
title,
k,
n,
status,
reason,
exceptions=tuple(exceptions),
diagnostics=diagnostics,
) )
@ -596,7 +627,7 @@ def score_named(m: StressMeasure, label: str) -> Row:
title, title,
None, None,
None, None,
DIAGNOSIS, NOT_MEASURED,
f"{label}: ikke målt, artefakter mangler ({m.missing})", f"{label}: ikke målt, artefakter mangler ({m.missing})",
failing=False, failing=False,
diagnostics=(NAMED_WARNING,), diagnostics=(NAMED_WARNING,),

View file

@ -215,7 +215,8 @@ _INSTRUCTIONS: Final = {
"BINDS it: pass read_dir a 'filter' word taken from the approach's own label — " "BINDS it: pass read_dir a 'filter' word taken from the approach's own label — "
"filter='rundkjoring' finds the level's requirements about roundabouts, and one of them is " "filter='rundkjoring' finds the level's requirements about roundabouts, and one of them is "
"the 'Krav 4.1.2-1' you are looking for — read it with read_file, then call " "the 'Krav 4.1.2-1' you are looking for — read it with read_file, then call "
"declare_requirement with the base id, that path and the requirement's own number. The " "declare_requirement with the base id, that path, the requirement's own number and the "
"short label of the direction as approach_id. The "
"reply gives back the document's own title and number: if they are not about your measure, " "reply gives back the document's own title and number: if they are not about your measure, "
"you declared the wrong requirement and should filter again. A direction with no " "you declared the wrong requirement and should filter again. A direction with no "
"requirement behind it is a guess. " "requirement behind it is a guess. "
@ -371,6 +372,21 @@ class DeclaredRequirement:
bundle_id: str bundle_id: str
path: str path: str
ref: str ref: str
#: WHICH approach the declaration is for (row 6). A requirement bound at run level cannot be
#: attributed to one approach — the judge labelled such a declaration ``run`` — so the rule that
#: a validated proposal must rest on its own approach's declaration needs the address on the
#: record itself. In a commissioned run it is one of the mandate's ids or ``own-proposal``; the
#: exploration, which mints its directions after declaring, records the label verbatim.
approach_id: str
class UnknownApproach(ValueError):
"""A declaration named an approach this run was not commissioned with (row 6).
Returned as a refused TURN, never raised out of the run (the ``RequirementNotRead`` rule): the
refusal NAMES the valid ids, and naming them is the correction a declaration filed under an
id no approach carries would be recorded against nothing and could never satisfy the rule.
"""
class RequirementNotRead(ValueError): class RequirementNotRead(ValueError):
@ -1068,6 +1084,7 @@ def _index_excerpt(body: str) -> tuple[str, bool]:
_RETURNABLE_REFUSALS: Final = ( _RETURNABLE_REFUSALS: Final = (
ExplorationError, ExplorationError,
RequirementNotRead, RequirementNotRead,
UnknownApproach,
okf.BundleIdMismatch, okf.BundleIdMismatch,
okf.BundlePathNotFound, okf.BundlePathNotFound,
okf.DocumentPathRefused, okf.DocumentPathRefused,
@ -1186,6 +1203,7 @@ def navigator_tools(
opened: list[ToolCall] | None = None, opened: list[ToolCall] | None = None,
requirements: list[DeclaredRequirement] | None = None, requirements: list[DeclaredRequirement] | None = None,
labels: Sequence[str] = (), labels: Sequence[str] = (),
approach_ids: Sequence[str] | None = None,
) -> list[FunctionTool]: ) -> list[FunctionTool]:
"""The navigator's tools: survey the catalogue, open one base, read one document — and, when """The navigator's tools: survey the catalogue, open one base, read one document — and, when
the caller offers the two sinks, DECLARE the requirement that binds a direction. the caller offers the two sinks, DECLARE the requirement that binds a direction.
@ -1241,6 +1259,11 @@ def navigator_tools(
the run's own read trace and a second record of it would be free to disagree with the first. the run's own read trace and a second record of it would be free to disagree with the first.
Passing one without the other is refused at construction: a log that cannot see what was opened Passing one without the other is refused at construction: a log that cannot see what was opened
would accept every declaration, which is the vacuous-gate class. would accept every declaration, which is the vacuous-gate class.
**Every declaration names the approach it is for** (row 6). ``approach_ids`` is the set a
commissioned run can file under the mandate's ids plus ``own-proposal`` — and an id outside it
is refused with the valid ones named. ``None`` (the exploration, which has no ids until it mints
them) records any non-empty id verbatim.
""" """
if (opened is None) != (requirements is None): if (opened is None) != (requirements is None):
raise ExplorationError( raise ExplorationError(
@ -1462,20 +1485,36 @@ def navigator_tools(
"'Krav 4.1.2-1'), read_file to read it, then declare it. The reply gives back the " "'Krav 4.1.2-1'), read_file to read it, then declare it. The reply gives back the "
"document's own " "document's own "
"title and number, so you can see whether you declared the requirement you meant: a " "title and number, so you can see whether you declared the requirement you meant: a "
"declaration of a requirement that is not about the measure is worth nothing." "declaration of a requirement that is not about the measure is worth nothing. "
"approach_id names the approach the requirement binds: declare once for EACH approach "
"you propose for (the run's own proposal is 'own-proposal'). A proposal whose approach "
"declared no requirement is not validated, however good its numbers are."
), ),
) )
def declare_requirement(bundle_id: str, path: str, ref: str) -> dict[str, Any]: def declare_requirement(
bundle_id: str, path: str, ref: str, approach_id: str
) -> dict[str, Any]:
try: try:
return _declare_requirement(bundle_id, path, ref) return _declare_requirement(bundle_id, path, ref, approach_id)
except _RETURNABLE_REFUSALS as exc: except _RETURNABLE_REFUSALS as exc:
return _refused_mapping(exc) return _refused_mapping(exc)
def _declare_requirement(bundle_id: str, path: str, ref: str) -> dict[str, Any]: def _declare_requirement(
bundle_id: str, path: str, ref: str, approach_id: str
) -> dict[str, Any]:
assert opened is not None and requirements is not None # the constructor guard above assert opened is not None and requirements is not None # the constructor guard above
# The base is resolved by the SAME index every read rung uses, so an unknown base is # The base is resolved by the SAME index every read rung uses, so an unknown base is
# refused here exactly as it is there rather than being accepted into the record. # refused here exactly as it is there rather than being accepted into the record.
resolved_dir = _resolve_bundle(index, bundle_id) resolved_dir = _resolve_bundle(index, bundle_id)
# Row 6: the address is checked before anything is read, so a declaration filed under no
# approach is refused whatever else is right about it.
if approach_ids is not None and approach_id not in approach_ids:
raise UnknownApproach(
f"{approach_id!r}; this run's approaches are {', '.join(map(repr, approach_ids))}. "
"Declare the requirement under the id of the approach it binds"
)
if not approach_id.strip():
raise UnknownApproach("an empty approach_id; name the approach this requirement binds")
read_paths = [call.path for call in opened if call.name == "read_file" and call.path] read_paths = [call.path for call in opened if call.name == "read_file" and call.path]
if path not in read_paths: if path not in read_paths:
raise RequirementNotRead( raise RequirementNotRead(
@ -1518,7 +1557,9 @@ def navigator_tools(
"Use read_dir with a 'filter' word from the approach's own label to find the " "Use read_dir with a 'filter' word from the approach's own label to find the "
"candidates, then read_file the ones that could bind it" "candidates, then read_file the ones that could bind it"
) )
requirements.append(DeclaredRequirement(bundle_id=bundle_id, path=path, ref=ref)) requirements.append(
DeclaredRequirement(bundle_id=bundle_id, path=path, ref=ref, approach_id=approach_id)
)
# P20/A1: give back the DOCUMENT's own title and number, read off the base rather than # P20/A1: give back the DOCUMENT's own title and number, read off the base rather than
# echoed from the arguments. MEASURED (P19 round 3, P17b): 13 declarations over 5 runs and # echoed from the arguments. MEASURED (P19 round 3, P17b): 13 declarations over 5 runs and
# NOT ONE named a fasit concept — the tool answered ``{"declared": true, ...}`` to every # NOT ONE named a fasit concept — the tool answered ``{"declared": true, ...}`` to every
@ -1532,6 +1573,7 @@ def navigator_tools(
"bundle_id": bundle_id, "bundle_id": bundle_id,
"path": path, "path": path,
"ref": ref, "ref": ref,
"approach_id": approach_id,
"title": declared[0], "title": declared[0],
"req_number": declared[1], "req_number": declared[1],
"binds": ( "binds": (
@ -1577,7 +1619,10 @@ def requirement_payload(declared: Sequence[DeclaredRequirement]) -> list[dict[st
copies of "what a declaration looks like" would drift into two answers about one run, which is copies of "what a declaration looks like" would drift into two answers about one run, which is
the -(p) defect landing in exactly the files an operator reads after a paid run. the -(p) defect landing in exactly the files an operator reads after a paid run.
""" """
return [{"bundle_id": d.bundle_id, "path": d.path, "ref": d.ref} for d in declared] return [
{"bundle_id": d.bundle_id, "path": d.path, "ref": d.ref, "approach_id": d.approach_id}
for d in declared
]
def _refused( def _refused(

View file

@ -249,7 +249,7 @@ def _budget_payload(exc: BudgetExceeded) -> dict[str, Any]:
"""The exhausted-budget body: the ledger's own triple, plus the human line for the log. """The exhausted-budget body: the ledger's own triple, plus the human line for the log.
The ``budget_exhausted`` key's PRESENCE is the discriminator — it is not folded into The ``budget_exhausted`` key's PRESENCE is the discriminator — it is not folded into
``outcome_type`` (whose values, ``validated``/``rejected``, mean "the run concluded and ``outcome_type`` (whose values, ``validated``/``rejected``/``unsupported``, mean "the run concluded and
here is the verdict") for the same reason ``BudgetStop`` was given its own field instead of here is the verdict") for the same reason ``BudgetStop`` was given its own field instead of
widening ``stop_reason``. Nor could it be: ``outcome_payload`` is the ONE copy of that fork widening ``stop_reason``. Nor could it be: ``outcome_payload`` is the ONE copy of that fork
and takes a ``ValidatedProposal | Rejection``, neither of which an exhausted run has.""" and takes a ``ValidatedProposal | Rejection``, neither of which an exhausted run has."""

View file

@ -334,7 +334,9 @@ class ApproachOutcome:
id: str id: str
label: str label: str
status: Literal["validated", "rejected", "not_evaluated"] #: ``unsupported`` (row 6): the numbers held but the approach declared no binding requirement,
#: so it is neither a success nor a numeric rejection. Never counted as validated.
status: Literal["validated", "rejected", "unsupported", "not_evaluated"]
detail: str = "" detail: str = ""
saving_nok: float | None = None saving_nok: float | None = None
@ -471,6 +473,8 @@ def settle(
lines.append(f" {row.id:<20} VALIDATED {amount:>14} {row.label}") lines.append(f" {row.id:<20} VALIDATED {amount:>14} {row.label}")
elif row.status == "rejected": elif row.status == "rejected":
lines.append(f" {row.id:<20} REJECTED {row.detail}") lines.append(f" {row.id:<20} REJECTED {row.detail}")
elif row.status == "unsupported":
lines.append(f" {row.id:<20} UNSUPPORTED {row.detail}")
else: else:
lines.append(f" {row.id:<20} NOT EVALUATED {row.detail}") lines.append(f" {row.id:<20} NOT EVALUATED {row.detail}")

View file

@ -35,7 +35,7 @@ from collections.abc import Mapping, Sequence
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from portfolio_optimiser.validator import Rejection, ValidatedProposal from portfolio_optimiser.validator import Rejection, Unsupported, ValidatedProposal
if TYPE_CHECKING: # provenance imports agent_framework — keep it out of the runtime import graph if TYPE_CHECKING: # provenance imports agent_framework — keep it out of the runtime import graph
from portfolio_optimiser.provenance import ProvenanceStamp from portfolio_optimiser.provenance import ProvenanceStamp
@ -113,7 +113,21 @@ def outcome_payload(
validated/rejected branching, shared by ``write_outbox`` and the hosted invocations validated/rejected branching, shared by ``write_outbox`` and the hosted invocations
response (``hosting._response_payload``). Two copies of the branch would drift, and a response (``hosting._response_payload``). Two copies of the branch would drift, and a
drifted copy would let the HTTP surface describe an outcome the outbox never wrote drifted copy would let the HTTP surface describe an outcome the outbox never wrote
the ``to_ore`` single-source rule (-(p)) applied to a payload shape.""" the ``to_ore`` single-source rule (-(p)) applied to a payload shape.
``unsupported`` (row 6) is checked FIRST because it subclasses ``Rejection``: it carries the
validator's percentiles (the numbers held) AND the reason, so a reader sees both halves."""
if isinstance(outcome, Unsupported):
return {
"outcome_type": "unsupported",
"reason": outcome.reason,
"p10": outcome.validated.p10,
"p50": outcome.validated.p50,
"p90": outcome.validated.p90,
"nominal_feasible": outcome.validated.nominal_feasible,
"checker_verdict": checker_verdict,
"verdict_id": verdict_id,
}
if isinstance(outcome, ValidatedProposal): if isinstance(outcome, ValidatedProposal):
return { return {
"outcome_type": "validated", "outcome_type": "validated",

View file

@ -123,8 +123,10 @@ from portfolio_optimiser.provenance import ProvenanceStamp
from portfolio_optimiser.reference_domain import Project, load_reference_projects from portfolio_optimiser.reference_domain import Project, load_reference_projects
from portfolio_optimiser.tracing import TracingConfigError, configure_tracing, tracing_notice from portfolio_optimiser.tracing import TracingConfigError, configure_tracing, tracing_notice
from portfolio_optimiser.validator import ( from portfolio_optimiser.validator import (
UNSUPPORTED_REASON,
Grounding, Grounding,
Rejection, Rejection,
Unsupported,
ValidatedProposal, ValidatedProposal,
baseline_from_project, baseline_from_project,
classify_codes, classify_codes,
@ -410,7 +412,10 @@ def _coverage_row(
) -> ApproachOutcome: ) -> ApproachOutcome:
"""One coverage row from one evaluated approach. A rejection carries the validator's reason """One coverage row from one evaluated approach. A rejection carries the validator's reason
verbatim a bare status would tell the expert their approach failed without telling them why, verbatim a bare status would tell the expert their approach failed without telling them why,
which is the part they can actually act on.""" which is the part they can actually act on. An ``Unsupported`` outcome (row 6) is checked
first: it subclasses ``Rejection``, and its own status is the whole point of the class."""
if isinstance(outcome, Unsupported):
return ApproachOutcome(id=row_id, label=label, status="unsupported", detail=outcome.reason)
if isinstance(outcome, ValidatedProposal): if isinstance(outcome, ValidatedProposal):
return ApproachOutcome( return ApproachOutcome(
id=row_id, id=row_id,
@ -490,6 +495,15 @@ def evaluate_mandate_candidates(
return tuple(rows) return tuple(rows)
def _declarable_ids(mandate: Mandate | None) -> list[str]:
"""The approach ids a declaration may be filed under (row 6): the mandate's own, plus the run's
own proposal whenever the run makes one always without a mandate."""
ids = [a.id for a in mandate.approaches] if mandate is not None else []
if mandate is None or mandate.allow_own_proposals:
ids.append(OWN_PROPOSAL_ID)
return ids
def _select_outcome( def _select_outcome(
produced: list[tuple[int, ValidatedProposal | Rejection]], produced: list[tuple[int, ValidatedProposal | Rejection]],
) -> ValidatedProposal | Rejection: ) -> ValidatedProposal | Rejection:
@ -511,6 +525,7 @@ async def _evaluate_mandate(
mandate: Mandate, mandate: Mandate,
evaluate: Callable[[Approach | None], Awaitable[ValidatedProposal | Rejection]], evaluate: Callable[[Approach | None], Awaitable[ValidatedProposal | Rejection]],
budget_stops: list[str] | None = None, budget_stops: list[str] | None = None,
declared: Sequence[DeclaredRequirement] | None = None,
) -> tuple[ ) -> tuple[
ValidatedProposal | Rejection, ValidatedProposal | Rejection,
tuple[ApproachOutcome, ...], tuple[ApproachOutcome, ...],
@ -532,6 +547,14 @@ async def _evaluate_mandate(
become ``not_evaluated`` rows. But if the very first approach exhausts the budget there is become ``not_evaluated`` rows. But if the very first approach exhausts the budget there is
nothing honest to return, so ``BudgetExceeded`` propagates exactly as it did before a run nothing honest to return, so ``BudgetExceeded`` propagates exactly as it did before a run
that produced nothing must still fail loudly rather than hand back an empty report. that produced nothing must still fail loudly rather than hand back an empty report.
**Row 6: a validated outcome must rest on ITS approach's own declaration.** When ``declared`` is
given (the rule is active see ``run_project``), a ``ValidatedProposal`` whose approach carries
no ``requirement`` of its own and has no declaration filed under its id becomes
``Unsupported``: the numbers held, the ground was never named. A declaration made for another
approach does not stand in, and nothing about the declaration's QUALITY is judged — a
requirement the model declared is accepted whatever it says (P22 § 4: an overlap gate would
refuse legitimate proposals). ``declared`` is read at evaluation time, after the debate.
""" """
plan: list[tuple[str, str, Approach | None]] = [(a.id, a.label, a) for a in mandate.approaches] plan: list[tuple[str, str, Approach | None]] = [(a.id, a.label, a) for a in mandate.approaches]
if mandate.allow_own_proposals: if mandate.allow_own_proposals:
@ -563,6 +586,15 @@ async def _evaluate_mandate(
for rid, lbl, _ in plan[index:] for rid, lbl, _ in plan[index:]
) )
break break
if (
declared is not None
and isinstance(outcome, ValidatedProposal)
and not (approach is not None and approach.requirement is not None)
and row_id not in {d.approach_id for d in declared}
):
outcome = Unsupported(
proposal=outcome.proposal, reason=UNSUPPORTED_REASON, validated=outcome
)
produced.append((index, outcome)) produced.append((index, outcome))
evaluated.append((row_id, outcome)) evaluated.append((row_id, outcome))
rows.append(_coverage_row(row_id, label, outcome)) rows.append(_coverage_row(row_id, label, outcome))
@ -622,8 +654,10 @@ def _bundle_pointer(bundle: okf.Bundle, bundle_id: str, *, dimension: str | None
"requirement number or path contains that word, and reports 'total_matches'.\n" "requirement number or path contains that word, and reports 'total_matches'.\n"
"Before you settle on a measure, name the ONE requirement of this base that BINDS it: " "Before you settle on a measure, name the ONE requirement of this base that BINDS it: "
"find it with a filter, read it with read_file, then call " "find it with a filter, read it with read_file, then call "
f"declare_requirement({bundle_id!r}, path, ref) with the requirement's own number. A " f"declare_requirement({bundle_id!r}, path, ref, approach_id) with the requirement's own "
"declaration naming a document this run never opened is refused; reading it is the fix." "number and the id of the approach it binds (the run's own proposal is 'own-proposal'). "
"Declare once per approach: a proposal whose approach declared nothing is not validated. "
"A declaration naming a document this run never opened is refused; reading it is the fix."
) )
@ -1184,6 +1218,12 @@ async def run_project(
#: documents above, so the gate's vocabulary and the gate's text describe one reading of one #: documents above, so the gate's vocabulary and the gate's text describe one reading of one
#: base. Empty on the road path, which is what keeps the rule unable to fire there. #: base. Empty on the road path, which is what keeps the rule unable to fire there.
bundle_references: tuple[str, ...] = () bundle_references: tuple[str, ...] = ()
#: Row 6: whether the declaration rung was offered to the debate. The rule that a validated
#: proposal needs its approach's own declaration is active exactly when it was: a run that
#: could declare and did not is the silence the rule exists for. It is NOT relaxed for a base
#: without requirement numbers — any document the run read can be declared, and a base that
#: holds nothing worth declaring is a finding about the base, not a reason to validate.
requirement_rung = False
# S2c: a CALLER-OWNED sink for what the debate opens (the ``parse_failures``/``ExplorationTrace`` # S2c: a CALLER-OWNED sink for what the debate opens (the ``parse_failures``/``ExplorationTrace``
# shape). A returned value would be lost on exactly the run that most needs the evidence — a # shape). A returned value would be lost on exactly the run that most needs the evidence — a
# budget stop mid-debate raises out of ``debate.run`` and constructs no ``RunResult`` at all. # budget stop mid-debate raises out of ``debate.run`` and constructs no ``RunResult`` at all.
@ -1300,8 +1340,11 @@ async def run_project(
# names every direction the run carries rather than picking one it cannot # names every direction the run carries rather than picking one it cannot
# attribute. Without a mandate this is empty and the reply is unchanged. # attribute. Without a mandate this is empty and the reply is unchanged.
labels=[a.label for a in mandate.approaches] if mandate else (), labels=[a.label for a in mandate.approaches] if mandate else (),
# Row 6: the ids a declaration may be filed under.
approach_ids=_declarable_ids(mandate),
) )
) )
requirement_rung = True
# What the navigation could NOT reach, taken from the run's ONE walk. The road path below # What the navigation could NOT reach, taken from the run's ONE walk. The road path below
# navigates no bundle at all, so its empty tuple is literally true rather than a stand-in. # navigates no bundle at all, so its empty tuple is literally true rather than a stand-in.
skipped_links: tuple[okf.SkippedLink, ...] = bundle.skipped skipped_links: tuple[okf.SkippedLink, ...] = bundle.skipped
@ -1588,7 +1631,10 @@ async def run_project(
validator_outcome = await _evaluate(None) validator_outcome = await _evaluate(None)
else: else:
validator_outcome, coverage, evaluated = await _evaluate_mandate( validator_outcome, coverage, evaluated = await _evaluate_mandate(
mandate, _evaluate, budget_stops mandate,
_evaluate,
budget_stops,
declared=debate_requirements if requirement_rung else None,
) )
except BaseException as stop: except BaseException as stop:
# Recorded and re-raised UNTOUCHED. This arm decides nothing about the exception itself — # Recorded and re-raised UNTOUCHED. This arm decides nothing about the exception itself —
@ -1679,7 +1725,9 @@ async def run_project(
model=model, model=model,
role="proposer", role="proposer",
validator_decision=( validator_decision=(
"validated" if isinstance(validator_outcome, ValidatedProposal) else "rejected" "validated"
if isinstance(validator_outcome, (ValidatedProposal, Unsupported))
else "rejected"
), ),
token_usage=meter.tokens, token_usage=meter.tokens,
# Whether stage 0 of the deterministic gate had a baseline to reconcile against. Read off # Whether stage 0 of the deterministic gate had a baseline to reconcile against. Read off
@ -1799,7 +1847,7 @@ async def run_project(
update={ update={
"validator_decision": ( "validator_decision": (
"validated" "validated"
if isinstance(approach_outcome, ValidatedProposal) if isinstance(approach_outcome, (ValidatedProposal, Unsupported))
else "rejected" else "rejected"
), ),
# P20: ``code_forms`` must follow ITS OWN approach too, for # P20: ``code_forms`` must follow ITS OWN approach too, for

View file

@ -69,7 +69,7 @@ import argparse
import json import json
import os import os
import sys import sys
from collections.abc import Sequence from collections.abc import Mapping, Sequence
from dataclasses import asdict, dataclass from dataclasses import asdict, dataclass
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
@ -98,7 +98,7 @@ class ApproachVerdict:
approach_id: str approach_id: str
label: str label: str
status: str # "validated" | "rejected" | "not_evaluated" status: str # "validated" | "rejected" | "unsupported" | "not_evaluated"
#: (a) - grounded by an OPENED path, or by a citation under a NARROWED list. See the module #: (a) - grounded by an OPENED path, or by a citation under a NARROWED list. See the module
#: docstring: a whole-base citation list is stamped before any model call and grounds nothing. #: docstring: a whole-base citation list is stamped before any model call and grounds nothing.
grounded: bool grounded: bool
@ -197,6 +197,10 @@ class ContextSetVerdict:
approach_rows_seen: int approach_rows_seen: int
concepts_in_base: int concepts_in_base: int
ferdig: bool ferdig: bool
#: Row 6 - declarations with no ``approach_id``: written before declarations carried one. A run
#: with any cannot be measured against the rule that a validated proposal needs its own
#: approach's declaration, and the v1 gate says so instead of counting.
unaddressed_declarations: int = 0
def to_payload(self) -> dict[str, Any]: def to_payload(self) -> dict[str, Any]:
"""Byte-stable plain data: the ONE rendering, shared by the CLI's stdout and its file.""" """Byte-stable plain data: the ONE rendering, shared by the CLI's stdout and its file."""
@ -265,21 +269,29 @@ def _inside(base: Path, raw: str) -> Path | None:
return resolved if resolved == root or root in resolved.parents else None return resolved if resolved == root or root in resolved.parents else None
def _attributable(approach: Any, declared: Sequence[str]) -> tuple[tuple[str, ...], str]: def _attributable(
approach: Any, declared: Sequence[Mapping[str, Any]]
) -> tuple[tuple[str, ...], str]:
"""Which declared requirement paths this approach may be judged on, and where they came from. """Which declared requirement paths this approach may be judged on, and where they came from.
The approach's OWN requirement wins when it has one: the mandate carries it per approach, so The approach's OWN requirement wins when it has one: the mandate carries it per approach, so
it is unambiguous by construction. Otherwise the RUN's declarations are attributable - the it is unambiguous by construction. Next, a declaration filed under THIS approach's id (row 6)
debate declares once for the whole run, so the row says ``run`` rather than pretending the is the approach's own and says ``approach`` too. A declaration with no ``approach_id`` at all
declaration was made about it. ``absent`` is the third value and is not the same as "declared was written before declarations carried one; it can only be attributed to the whole run, and
nothing that matched": a run that declared nothing is a different finding from one that the row says ``run`` rather than pretending it was made about this approach. A declaration
declared the wrong document.""" filed under ANOTHER approach's id is not this one's, so a run whose declarations all name other
approaches reads ``absent`` here the same value as a run that declared nothing.
"""
own = getattr(approach, "requirement", None) own = getattr(approach, "requirement", None)
path = "" if own is None else str(getattr(own, "path", "") or "") path = "" if own is None else str(getattr(own, "path", "") or "")
if path: if path:
return (path,), "approach" return (path,), "approach"
if declared: addressed = tuple(str(r["path"]) for r in declared if r.get("approach_id") == approach.id)
return tuple(declared), "run" if addressed:
return addressed, "approach"
legacy = tuple(str(r["path"]) for r in declared if "approach_id" not in r)
if legacy:
return legacy, "run"
return (), "absent" return (), "absent"
@ -359,14 +371,13 @@ def score_context_set(
# what every stress round has been) writes no ``{run_id}-exploration.json`` at all - the # what every stress round has been) writes no ``{run_id}-exploration.json`` at all - the
# hypothesiser never runs - so the debate artefact is the only one that can carry them there. # hypothesiser never runs - so the debate artefact is the only one that can carry them there.
# Both are read, because an ``--explore`` run carries them in the other. # Both are read, because an ``--explore`` run carries them in the other.
declared_paths: list[str] = [] declared_records: list[dict[str, Any]] = []
for artefact in (debate, outbox / f"{run_id}-exploration.json"): for artefact in (debate, outbox / f"{run_id}-exploration.json"):
if artefact.is_file(): if artefact.is_file():
declared_paths += [ declared_records += [
str(r.get("path", "")) r for r in _read_json(artefact).get("requirements", []) if r.get("path")
for r in _read_json(artefact).get("requirements", [])
if r.get("path")
] ]
declared_paths = [str(r["path"]) for r in declared_records]
hallucinated_reads: list[str] = [] hallucinated_reads: list[str] = []
for call in tool_calls: for call in tool_calls:
raw = str(call.get("path", "")) raw = str(call.get("path", ""))
@ -417,9 +428,11 @@ def score_context_set(
named_in_measure=False, named_in_measure=False,
named_in_snippet=False, named_in_snippet=False,
hallucinations=(), hallucinations=(),
requirement_declared=_attributable(approach, declared_paths)[0], requirement_declared=_attributable(approach, declared_records)[0],
requirement_source=_attributable(approach, declared_paths)[1], requirement_source=_attributable(approach, declared_records)[1],
requirement_hit=bool(set(_attributable(approach, declared_paths)[0]) & wanted), requirement_hit=bool(
set(_attributable(approach, declared_records)[0]) & wanted
),
prose_codes=(), prose_codes=(),
priced=False, priced=False,
not_evaluated_reason=stop_reason or "absent", not_evaluated_reason=stop_reason or "absent",
@ -472,7 +485,7 @@ def score_context_set(
forms = payload.get("provenance", {}).get("code_forms") or classify_codes(codes) forms = payload.get("provenance", {}).get("code_forms") or classify_codes(codes)
prose_codes = tuple(sorted(c for c in codes if forms.get(c) == "prose")) prose_codes = tuple(sorted(c for c in codes if forms.get(c) == "prose"))
attributable, requirement_source = _attributable(approach, declared_paths) attributable, requirement_source = _attributable(approach, declared_records)
requirement_hit = bool(set(attributable) & wanted) requirement_hit = bool(set(attributable) & wanted)
halluc = [f"citation:{f}" for f in sorted(cited_files - concept_names)] halluc = [f"citation:{f}" for f in sorted(cited_files - concept_names)]
@ -563,6 +576,7 @@ def score_context_set(
must_refuse=tuple(refusals), must_refuse=tuple(refusals),
hallucinated_reads=tuple(hallucinated_reads), hallucinated_reads=tuple(hallucinated_reads),
requirements_declared=tuple(declared_paths), requirements_declared=tuple(declared_paths),
unaddressed_declarations=sum(1 for r in declared_records if "approach_id" not in r),
token_usage=token_usage, token_usage=token_usage,
stop_reason=stop_reason if coverage_seen else "absent", stop_reason=stop_reason if coverage_seen else "absent",
anchored=anchored, anchored=anchored,

View file

@ -102,6 +102,30 @@ class Rejection:
reason: str reason: str
#: The ONE sentence an unsupported outcome carries. ``rejection_stage`` keys on it, so the judge can
#: tell this falsifier from the numeric ones without a second copy of the wording.
UNSUPPORTED_REASON: Final = (
"no declared requirement for this approach: the numbers held, but no requirement of the "
"knowledge base was declared as binding it"
)
@dataclass(frozen=True)
class Unsupported(Rejection):
"""A proposal whose NUMBERS held but whose approach declared no binding requirement (row 6).
Neither ``validated`` (nothing in the knowledge base was said to support the direction) nor an
ordinary rejection (every numeric stage passed). It subclasses ``Rejection`` on purpose: every
consumer that asks "is this validated?" with ``isinstance(..., ValidatedProposal)`` answers no
without being touched, so it is never counted, summed or carried as a success. The consumers
that NAME the status coverage, the outcome artefact, the settlement, the judge check for
this class first. ``validated`` keeps the validator's own ruling, which is what
``provenance.validator_decision`` mirrors: the validator said yes, and the record says so.
"""
validated: ValidatedProposal
def _solve_max_feasible(items: list[AffectedItem], fraction: float) -> float: def _solve_max_feasible(items: list[AffectedItem], fraction: float) -> float:
"""Real CBC solve: maximize total saving subject to a per-item upper bound and a """Real CBC solve: maximize total saving subject to a per-item upper bound and a
global fraction cap. Raises ``CbcUnavailable`` if CBC is genuinely missing.""" global fraction cap. Raises ``CbcUnavailable`` if CBC is genuinely missing."""
@ -728,6 +752,7 @@ _REJECTION_STAGES: Final = (
("stage4-p90", ("exceeds P90 feasible",)), ("stage4-p90", ("exceeds P90 feasible",)),
("stage4b-nominal", ("exceeds the nominal feasible",)), ("stage4b-nominal", ("exceeds the nominal feasible",)),
("stage5-method-cap", ("method cap",)), ("stage5-method-cap", ("method cap",)),
("unsupported", ("no declared requirement for this approach",)),
) )

View file

@ -38,7 +38,7 @@ from collections.abc import Callable
from pathlib import Path from pathlib import Path
from portfolio_optimiser import hitl from portfolio_optimiser import hitl
from portfolio_optimiser.mandate import OWN_PROPOSAL_ID, Approach, Mandate from portfolio_optimiser.mandate import BindingRequirement, OWN_PROPOSAL_ID, Approach, Mandate
from portfolio_optimiser.run import run_project from portfolio_optimiser.run import run_project
from portfolio_optimiser.simulation import ScriptedChatClient from portfolio_optimiser.simulation import ScriptedChatClient
from portfolio_optimiser.verdicts import ( from portfolio_optimiser.verdicts import (
@ -66,8 +66,11 @@ def _reply(measure: str, claimed: int) -> str:
# Labels ABSENT from the bundle's own prose (the "LED-retrofit" trap: it appears in 6 bundle files, # Labels ABSENT from the bundle's own prose (the "LED-retrofit" trap: it appears in 6 bundle files,
# so a client keyed on it would match every prompt through the context and prove nothing). # so a client keyed on it would match every prompt through the context and prove nothing).
_LED = Approach(id="led-retrofit", label="Behovsstyrt belysning i fellesarealer") #: Row 6: a commissioned approach validates only on a requirement of its own. These tests are about
_HVAC = Approach(id="hvac-swap", label="Utskifting av ventilasjonsaggregat") #: selection and per-approach artefacts, not about declarations, so the commission names one.
_REQ = BindingRequirement(path="tiltak-led-retrofit.md", ref="Krav 1")
_LED = Approach(id="led-retrofit", label="Behovsstyrt belysning i fellesarealer", requirement=_REQ)
_HVAC = Approach(id="hvac-swap", label="Utskifting av ventilasjonsaggregat", requirement=_REQ)
_LED_MEASURE = "Behovsstyrt belysning i fellesarealer" _LED_MEASURE = "Behovsstyrt belysning i fellesarealer"
_HVAC_MEASURE = "Utskifting av ventilasjonsaggregat" _HVAC_MEASURE = "Utskifting av ventilasjonsaggregat"
@ -181,7 +184,8 @@ async def test_each_approach_outcome_is_written_with_its_own_status(tmp_path: Pa
assert by_approach["led-retrofit"]["outcome_type"] == "validated" assert by_approach["led-retrofit"]["outcome_type"] == "validated"
assert by_approach["hvac-swap"]["outcome_type"] == "rejected" assert by_approach["hvac-swap"]["outcome_type"] == "rejected"
assert by_approach["hvac-swap"]["reason"], "a rejected approach must carry the reason" assert by_approach["hvac-swap"]["reason"], "a rejected approach must carry the reason"
assert by_approach[OWN_PROPOSAL_ID]["outcome_type"] == "validated" # The run's own proposal names no requirement and none was declared for it (row 6).
assert by_approach[OWN_PROPOSAL_ID]["outcome_type"] == "unsupported"
async def test_each_artefact_stamps_its_own_validator_decision(tmp_path: Path) -> None: async def test_each_artefact_stamps_its_own_validator_decision(tmp_path: Path) -> None:

View file

@ -362,7 +362,7 @@ def test_the_requirement_gate_reads_the_second_bases_own_opened_list(tmp_path: P
opened_1.append(explore.ToolCall(name="read_file", bundle_id=bundle_1, path=doc_1)) opened_1.append(explore.ToolCall(name="read_file", bundle_id=bundle_1, path=doc_1))
assert not opened_2, "the two bases shared one opened sink" assert not opened_2, "the two bases shared one opened sink"
refusal = tools_2["declare_requirement"].func(bundle_2, doc_1, "Krav 1") refusal = tools_2["declare_requirement"].func(bundle_2, doc_1, "Krav 1", "a1")
assert refusal["refusal"] == "RequirementNotRead", refusal assert refusal["refusal"] == "RequirementNotRead", refusal
assert not reqs_2, "base 2 recorded a requirement it never read" assert not reqs_2, "base 2 recorded a requirement it never read"
@ -371,7 +371,7 @@ def test_the_requirement_gate_reads_the_second_bases_own_opened_list(tmp_path: P
for name in [f.name for f in okf.navigate_bundle(second).context_files][:3]: for name in [f.name for f in okf.navigate_bundle(second).context_files][:3]:
tools_2["read_file"].func(bundle_2, name) tools_2["read_file"].func(bundle_2, name)
opened_2.append(explore.ToolCall(name="read_file", bundle_id=bundle_2, path=name)) opened_2.append(explore.ToolCall(name="read_file", bundle_id=bundle_2, path=name))
accepted = tools_2["declare_requirement"].func(bundle_2, doc_2, "Krav 1") accepted = tools_2["declare_requirement"].func(bundle_2, doc_2, "Krav 1", "a1")
assert accepted.get("declared") is True, accepted assert accepted.get("declared") is True, accepted
assert [r.path for r in reqs_2] == [doc_2] assert [r.path for r in reqs_2] == [doc_2]
assert not reqs_1, "the two bases shared one requirements sink" assert not reqs_1, "the two bases shared one requirements sink"

View file

@ -102,7 +102,7 @@ def test_a_requirement_the_run_never_opened_is_refused_and_recorded_nowhere() ->
"""(a) The declaration's one falsifier is the run's own read trace.""" """(a) The declaration's one falsifier is the run's own read trace."""
tools, _opened, declared = _wired() tools, _opened, declared = _wired()
answer = tools["declare_requirement"].func( answer = tools["declare_requirement"].func(
bundle_id="tunnel-hauglia", path=_a_concept(), ref="12.1" approach_id="a1", bundle_id="tunnel-hauglia", path=_a_concept(), ref="12.1"
) )
assert answer["refusal"] == "RequirementNotRead" assert answer["refusal"] == "RequirementNotRead"
assert "0 document(s)" in answer["refused"] assert "0 document(s)" in answer["refused"]
@ -130,7 +130,7 @@ def test_the_correction_is_to_read_it_and_then_it_is_accepted() -> None:
opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=other)) opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=other))
opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=path)) opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=path))
answer = tools["declare_requirement"].func( answer = tools["declare_requirement"].func(
bundle_id="tunnel-hauglia", path=path, ref="Krav 12.1" approach_id="a1", bundle_id="tunnel-hauglia", path=path, ref="Krav 12.1"
) )
# P20/A1 widened the reply: the three arguments PLUS the document's own title and number and # P20/A1 widened the reply: the three arguments PLUS the document's own title and number and
# the sentence saying what the declaration binds. Asserted key by key rather than by equality, # the sentence saying what the declaration binds. Asserted key by key rather than by equality,
@ -142,15 +142,30 @@ def test_the_correction_is_to_read_it_and_then_it_is_accepted() -> None:
path, path,
"Krav 12.1", "Krav 12.1",
) )
assert set(answer) == {"declared", "bundle_id", "path", "ref", "title", "req_number", "binds"} assert set(answer) == {
assert declared == [DeclaredRequirement(bundle_id="tunnel-hauglia", path=path, ref="Krav 12.1")] "declared",
"bundle_id",
"path",
"ref",
"approach_id",
"title",
"req_number",
"binds",
}
assert declared == [
DeclaredRequirement(
bundle_id="tunnel-hauglia", path=path, ref="Krav 12.1", approach_id="a1"
)
]
def test_a_declaration_naming_an_unknown_base_is_refused() -> None: def test_a_declaration_naming_an_unknown_base_is_refused() -> None:
"""(c) The same base index every read rung uses, so no second, laxer resolution exists.""" """(c) The same base index every read rung uses, so no second, laxer resolution exists."""
tools, opened, declared = _wired() tools, opened, declared = _wired()
opened.append(ToolCall(name="read_file", bundle_id="nope", path="a.md")) opened.append(ToolCall(name="read_file", bundle_id="nope", path="a.md"))
answer = tools["declare_requirement"].func(bundle_id="nope", path="a.md", ref="1") answer = tools["declare_requirement"].func(
approach_id="a1", bundle_id="nope", path="a.md", ref="1"
)
assert answer["refusal"] == "ExplorationError" assert answer["refusal"] == "ExplorationError"
assert declared == [] assert declared == []
@ -252,14 +267,15 @@ def test_a_hit_is_counted_against_this_approachs_fasit_never_the_base() -> None:
label="L", label="L",
requirement=BindingRequirement(path="krav/12-1/a.md", ref="12.1"), requirement=BindingRequirement(path="krav/12-1/a.md", ref="12.1"),
) )
paths, source = stress._attributable(approach, ["krav/99-9/elsewhere.md"]) paths, source = stress._attributable(approach, [{"path": "krav/99-9/elsewhere.md"}])
assert (paths, source) == (("krav/12-1/a.md",), "approach") assert (paths, source) == (("krav/12-1/a.md",), "approach")
assert set(paths) & wanted assert set(paths) & wanted
# A run-level declaration that is NOT one of this approach's fasit concepts is reported and is # A run-level declaration (one written before declarations carried an ``approach_id``) that is
# not a hit — the base holds it, which is exactly what must not count. # NOT one of this approach's fasit concepts is reported and is not a hit — the base holds it,
# which is exactly what must not count.
bare = Approach(id="a2", label="M") bare = Approach(id="a2", label="M")
paths, source = stress._attributable(bare, ["krav/99-9/elsewhere.md"]) paths, source = stress._attributable(bare, [{"path": "krav/99-9/elsewhere.md"}])
assert (paths, source) == (("krav/99-9/elsewhere.md",), "run") assert (paths, source) == (("krav/99-9/elsewhere.md",), "run")
assert not set(paths) & wanted assert not set(paths) & wanted
@ -408,11 +424,13 @@ def test_the_declaration_reaches_the_debate_artefact(tmp_path: Path) -> None:
[ToolCall(name="read_file", bundle_id="b", path="k/a.md")] [ToolCall(name="read_file", bundle_id="b", path="k/a.md")]
), ),
requirements=explore.requirement_payload( requirements=explore.requirement_payload(
[DeclaredRequirement(bundle_id="b", path="k/a.md", ref="12.1")] [DeclaredRequirement(bundle_id="b", path="k/a.md", ref="12.1", approach_id="a1")]
), ),
) )
payload = json.loads((tmp_path / "r1-debate.json").read_text(encoding="utf-8")) payload = json.loads((tmp_path / "r1-debate.json").read_text(encoding="utf-8"))
assert payload["requirements"] == [{"bundle_id": "b", "path": "k/a.md", "ref": "12.1"}] assert payload["requirements"] == [
{"bundle_id": "b", "path": "k/a.md", "ref": "12.1", "approach_id": "a1"}
]
# An empty list is the honest positive statement every pre-P19 run makes. # An empty list is the honest positive statement every pre-P19 run makes.
outbox.write_debate_tools(str(tmp_path), "r2", tool_calls=[]) outbox.write_debate_tools(str(tmp_path), "r2", tool_calls=[])
assert ( assert (

View file

@ -70,7 +70,7 @@ def test_c1_a_declaration_after_one_document_is_refused_with_the_denominator() -
opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=path)) opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=path))
answer = tools["declare_requirement"].func( answer = tools["declare_requirement"].func(
bundle_id="tunnel-hauglia", path=path, ref="Krav 1.1—1" approach_id="a1", bundle_id="tunnel-hauglia", path=path, ref="Krav 1.1—1"
) )
assert answer["refusal"] == "RequirementNotRead" assert answer["refusal"] == "RequirementNotRead"
@ -93,7 +93,7 @@ def test_c1_the_same_declaration_after_three_documents_is_accepted() -> None:
opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=name)) opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=name))
answer = tools["declare_requirement"].func( answer = tools["declare_requirement"].func(
bundle_id="tunnel-hauglia", path=files[0], ref="Krav 1.1—1" approach_id="a1", bundle_id="tunnel-hauglia", path=files[0], ref="Krav 1.1—1"
) )
assert answer["declared"] is True assert answer["declared"] is True
@ -109,7 +109,7 @@ def test_c1_the_same_path_read_three_times_is_still_one_document() -> None:
opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=path)) opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=path))
answer = tools["declare_requirement"].func( answer = tools["declare_requirement"].func(
bundle_id="tunnel-hauglia", path=path, ref="Krav 1.1—1" approach_id="a1", bundle_id="tunnel-hauglia", path=path, ref="Krav 1.1—1"
) )
assert answer["refusal"] == "RequirementNotRead" assert answer["refusal"] == "RequirementNotRead"
@ -135,7 +135,9 @@ def test_c1_a_small_base_stays_declarable(tmp_path: Path) -> None:
for name in ("a.md", "b.md"): for name in ("a.md", "b.md"):
opened.append(ToolCall(name="read_file", bundle_id="mini", path=name)) opened.append(ToolCall(name="read_file", bundle_id="mini", path=name))
answer = tools["declare_requirement"].func(bundle_id="mini", path="a.md", ref="1") answer = tools["declare_requirement"].func(
approach_id="a1", bundle_id="mini", path="a.md", ref="1"
)
assert answer["declared"] is True assert answer["declared"] is True
assert len(declared) == 1 assert len(declared) == 1
@ -151,7 +153,7 @@ def test_c1_the_never_opened_refusal_still_fires_first() -> None:
opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=name)) opened.append(ToolCall(name="read_file", bundle_id="tunnel-hauglia", path=name))
answer = tools["declare_requirement"].func( answer = tools["declare_requirement"].func(
bundle_id="tunnel-hauglia", path=files[4], ref="Krav 1.1—1" approach_id="a1", bundle_id="tunnel-hauglia", path=files[4], ref="Krav 1.1—1"
) )
assert answer["refusal"] == "RequirementNotRead" assert answer["refusal"] == "RequirementNotRead"

View file

@ -634,7 +634,8 @@ async def test_a_hosted_exploration_shapes_the_mandate_the_run_evaluates(_hosted
) )
) )
assert body["outcome_type"] in {"validated", "rejected"} # ``unsupported`` (row 6) is an evaluated outcome too: the numbers were judged.
assert body["outcome_type"] in {"validated", "rejected", "unsupported"}
assert any(_LABEL in prompt for prompt in _hosted_backend), ( assert any(_LABEL in prompt for prompt in _hosted_backend), (
"the shaped approach never reached a prompt — the hosted door does not wire the mandate" "the shaped approach never reached a prompt — the hosted door does not wire the mandate"
) )

View file

@ -24,7 +24,7 @@ from pathlib import Path
import pytest import pytest
from portfolio_optimiser.mandate import OWN_PROPOSAL_ID, Approach, Mandate from portfolio_optimiser.mandate import BindingRequirement, OWN_PROPOSAL_ID, Approach, Mandate
from portfolio_optimiser.simulation import ScriptedChatClient from portfolio_optimiser.simulation import ScriptedChatClient
from portfolio_optimiser.run import run_project from portfolio_optimiser.run import run_project
from portfolio_optimiser.validator import Rejection, ValidatedProposal from portfolio_optimiser.validator import Rejection, ValidatedProposal
@ -48,8 +48,11 @@ def _reply(measure: str, claimed: int) -> str:
# Labels are chosen to be ABSENT from the bundle's own prose: "LED-retrofit" appears in 6 of the # Labels are chosen to be ABSENT from the bundle's own prose: "LED-retrofit" appears in 6 of the
# bundle's files, so a client keyed on it would match every prompt through the context and prove # bundle's files, so a client keyed on it would match every prompt through the context and prove
# nothing about which approach was bound to which call. # nothing about which approach was bound to which call.
_LED = Approach(id="led-retrofit", label="Behovsstyrt belysning i fellesarealer") #: Row 6: a commissioned approach validates only on a requirement of its own. These tests are about
_HVAC = Approach(id="hvac-swap", label="Utskifting av ventilasjonsaggregat") #: selection and per-approach artefacts, not about declarations, so the commission names one.
_REQ = BindingRequirement(path="tiltak-led-retrofit.md", ref="Krav 1")
_LED = Approach(id="led-retrofit", label="Behovsstyrt belysning i fellesarealer", requirement=_REQ)
_HVAC = Approach(id="hvac-swap", label="Utskifting av ventilasjonsaggregat", requirement=_REQ)
#: LED validates (30k <= cap); HVAC is above the cap -> the validator rejects it. #: LED validates (30k <= cap); HVAC is above the cap -> the validator rejects it.
_REPLY_BY_LABEL = { _REPLY_BY_LABEL = {
@ -160,8 +163,12 @@ async def test_outcome_is_the_best_validated_candidate_deterministically(big_fir
orderings pinned, ``produced[-1]`` fails one case and ``produced[0]`` fails the other. orderings pinned, ``produced[-1]`` fails one case and ``produced[0]`` fails the other.
""" """
sink: list[str] = [] sink: list[str] = []
big = Approach(id="big", label="Behovsstyrt belysning i fellesarealer") # 30k, validates big = Approach(
small = Approach(id="small", label="Nattsenking av temperatur") # default reply, 20k id="big", label="Behovsstyrt belysning i fellesarealer", requirement=_REQ
) # 30k, validates
small = Approach(
id="small", label="Nattsenking av temperatur", requirement=_REQ
) # default reply, 20k
mandate = Mandate( mandate = Mandate(
objective="Cut energy cost without rebuilding.", objective="Cut energy cost without rebuilding.",
approaches=(big, small) if big_first else (small, big), approaches=(big, small) if big_first else (small, big),

View file

@ -61,7 +61,9 @@ def _declare(
for name in [f.name for f in okf.navigate_bundle(str(_TUNNEL)).context_files][:3]: for name in [f.name for f in okf.navigate_bundle(str(_TUNNEL)).context_files][:3]:
opened.append(ToolCall(name="read_file", bundle_id=_BASE_ID, path=name)) opened.append(ToolCall(name="read_file", bundle_id=_BASE_ID, path=name))
opened.append(ToolCall(name="read_file", bundle_id=_BASE_ID, path=_DOC)) opened.append(ToolCall(name="read_file", bundle_id=_BASE_ID, path=_DOC))
answer = tools["declare_requirement"].func(bundle_id=_BASE_ID, path=_DOC, ref=ref) answer = tools["declare_requirement"].func(
approach_id="a1", bundle_id=_BASE_ID, path=_DOC, ref=ref
)
return answer, declared return answer, declared
@ -115,7 +117,8 @@ def test_the_comparison_never_reads_the_callers_own_ref() -> None:
def test_without_directions_the_reply_is_the_one_p20_shipped() -> None: def test_without_directions_the_reply_is_the_one_p20_shipped() -> None:
"""The exploration mints its own directions, so at declaration time it HAS none — and every """The exploration mints its own directions, so at declaration time it HAS none — and every
call site that passes no labels must be byte-identical to before. Absent keys, not empty ones: call site that passes no labels must carry no comparison keys (row 6 added ``approach_id`` to
every reply, the address the declaration was filed under). Absent keys, not empty ones:
"there was nothing to compare against" and "we compared and found nothing" are different "there was nothing to compare against" and "we compared and found nothing" are different
facts, and only one of them is true here.""" facts, and only one of them is true here."""
answer, _ = _declare(()) answer, _ = _declare(())
@ -124,6 +127,7 @@ def test_without_directions_the_reply_is_the_one_p20_shipped() -> None:
"bundle_id", "bundle_id",
"path", "path",
"ref", "ref",
"approach_id",
"title", "title",
"req_number", "req_number",
"binds", "binds",
@ -180,7 +184,12 @@ async def test_a_commissioned_run_reaches_the_tool_with_its_own_directions() ->
{"call": "read_file", "args": {"bundle_id": _BASE_ID, "path": _DOC}}, {"call": "read_file", "args": {"bundle_id": _BASE_ID, "path": _DOC}},
{ {
"call": "declare_requirement", "call": "declare_requirement",
"args": {"bundle_id": _BASE_ID, "path": _DOC, "ref": "Krav 1.1-1"}, "args": {
"bundle_id": _BASE_ID,
"path": _DOC,
"ref": "Krav 1.1-1",
"approach_id": "a1",
},
}, },
_VALID_REPLY, _VALID_REPLY,
_VALID_REPLY, _VALID_REPLY,

View file

@ -78,7 +78,9 @@ def _declare(
for other in also: for other in also:
opened.append(ToolCall(name="read_file", bundle_id=base, path=other)) opened.append(ToolCall(name="read_file", bundle_id=base, path=other))
opened.append(ToolCall(name="read_file", bundle_id=base, path=path)) opened.append(ToolCall(name="read_file", bundle_id=base, path=path))
return tools["declare_requirement"].func(bundle_id=base, path=path, ref="Krav 4.1.2-1") return tools["declare_requirement"].func(
approach_id="a1", bundle_id=base, path=path, ref="Krav 4.1.2-1"
)
def _base_with_a_requirement(root: Path) -> Path: def _base_with_a_requirement(root: Path) -> Path:
@ -114,7 +116,11 @@ def test_the_reply_carries_the_documents_own_title_and_number(tmp_path: Path) ->
assert answer["title"] == "Krav 4.1.2-1 Rundkjoring" assert answer["title"] == "Krav 4.1.2-1 Rundkjoring"
assert answer["req_number"] == "Krav 4.1.2-1" assert answer["req_number"] == "Krav 4.1.2-1"
assert "is the requirement the proposal rests on" in answer["binds"] assert "is the requirement the proposal rests on" in answer["binds"]
assert declared == [DeclaredRequirement(bundle_id="n-mini", path="krav.md", ref="Krav 4.1.2-1")] assert declared == [
DeclaredRequirement(
bundle_id="n-mini", path="krav.md", ref="Krav 4.1.2-1", approach_id="a1"
)
]
def test_a_path_that_is_no_concept_answers_with_empty_strings(tmp_path: Path) -> None: def test_a_path_that_is_no_concept_answers_with_empty_strings(tmp_path: Path) -> None:

View file

@ -0,0 +1,292 @@
"""Row 6 — a proposal whose approach declared no requirement cannot carry ``validated``.
Before this rule every numeric stage could pass and the run would stamp ``validated`` whether or
not anything in the knowledge base had been named as binding the direction. Measured on stress
round 6: three falsification arms validated, and every one of the ten validated approaches had only
run-level declarations which a judge cannot attribute to any one approach.
What each arm pins:
(a) on the shipped micro base, where the debate holds ``declare_requirement``, a silent approach
that used to validate is now ``unsupported`` and so is the run's own proposal;
(b) a declaration filed under the approach's id is what lets it validate, and ONLY that approach;
(c) a requirement written into the mandate counts as the approach's own declaration;
(d) the declaration's quality is not judged: any requirement the run read is accepted;
(e) an id no approach carries is refused with the valid ids named, and nothing is recorded;
(f) ``unsupported`` is never counted, summed or selected as a success, yet the validator's own
ruling is kept on the record (``provenance.validator_decision``) and in the artefact;
(g) with the rule inactive (no declaration rung offered) the old ruling stands;
(h) the judge reads the addressed declaration as the approach's own, a legacy one as ``run``.
"""
from __future__ import annotations
import json
from pathlib import Path
from typing import Any
import pytest
from portfolio_optimiser import okf, outbox, stress
from portfolio_optimiser.explore import DeclaredRequirement, navigator_tools, requirement_payload
from portfolio_optimiser.ir import AffectedItem, SavingsProposal
from portfolio_optimiser.mandate import Approach, BindingRequirement, Mandate, settle
from portfolio_optimiser.run import _evaluate_mandate, run_project
from portfolio_optimiser.simulation import scripted_factory
from portfolio_optimiser.validator import (
UNSUPPORTED_REASON,
Rejection,
Unsupported,
ValidatedProposal,
rejection_stage,
)
from portfolio_optimiser.verdicts import VerdictStore
_BUNDLE = Path(__file__).resolve().parents[1] / "shared" / "examples" / "bygg-energi-mikro"
_BASE_ID = "bygg-energi-mikro"
_PID = "BYGG-KONTOR-NORD"
_REPLY = (
'{"measure":"LED-retrofit av kontorbelysning","affected_items":'
'[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],"claimed_saving_nok":30000}'
)
_CHECKER = "Reasoning holds.\nVERDICT: APPROVE"
_CONCEPTS = [f.name for f in okf.navigate_bundle(str(_BUNDLE)).context_files]
def _mandate(**approach: Any) -> Mandate:
return Mandate(
objective="Kutt energikostnad",
approaches=(Approach(id="a1", label="LED-retrofit", description="reason", **approach),),
)
def _script(*declare_for: str, ref: str = "Krav 1.1-1") -> dict[str, Any]:
steps: list[Any] = []
if declare_for:
steps += [
{"call": "read_file", "args": {"bundle_id": _BASE_ID, "path": n}} for n in _CONCEPTS[:3]
]
steps += [
{
"call": "declare_requirement",
"args": {
"bundle_id": _BASE_ID,
"path": _CONCEPTS[0],
"ref": ref,
"approach_id": aid,
},
}
for aid in declare_for
]
return {"proposer": [*steps, _REPLY, _REPLY, _REPLY, _REPLY, _REPLY], "checker": _CHECKER}
async def _run(tmp_path: Path, script: dict[str, Any], mandate: Mandate) -> Any:
return await run_project(
_PID,
"local",
docs_dir=str(_BUNDLE),
bundle_dir=str(_BUNDLE),
store=VerdictStore(verdicts=[]),
client_factory=scripted_factory(script, []),
mandate=mandate,
outbox_dir=str(tmp_path),
run_id="r6",
)
def _statuses(result: Any) -> dict[str, str]:
return {row.id: row.status for row in result.coverage}
# ---------------------------------------------------------------------------------------------
# (a)-(d) the rule on the real run
# ---------------------------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_a_silent_approach_on_the_micro_base_is_unsupported(tmp_path: Path) -> None:
result = await _run(tmp_path, _script(), _mandate())
assert _statuses(result) == {"a1": "unsupported", "own-proposal": "unsupported"}
assert {row.detail for row in result.coverage} == {UNSUPPORTED_REASON}
@pytest.mark.asyncio
async def test_only_the_approach_that_declared_is_validated(tmp_path: Path) -> None:
result = await _run(tmp_path, _script("a1"), _mandate())
assert _statuses(result) == {"a1": "validated", "own-proposal": "unsupported"}
debate = json.loads((tmp_path / "r6-debate.json").read_text(encoding="utf-8"))
assert [r["approach_id"] for r in debate["requirements"]] == ["a1"]
@pytest.mark.asyncio
async def test_a_declaration_for_the_own_proposal_does_not_stand_in(tmp_path: Path) -> None:
result = await _run(tmp_path, _script("own-proposal"), _mandate())
assert _statuses(result) == {"a1": "unsupported", "own-proposal": "validated"}
@pytest.mark.asyncio
async def test_a_requirement_written_into_the_mandate_counts(tmp_path: Path) -> None:
requirement = BindingRequirement(path=_CONCEPTS[0], ref="Krav 1.1-1")
result = await _run(tmp_path, _script(), _mandate(requirement=requirement))
assert _statuses(result)["a1"] == "validated"
@pytest.mark.asyncio
async def test_the_declarations_quality_is_not_judged(tmp_path: Path) -> None:
result = await _run(tmp_path, _script("a1", ref="anything at all"), _mandate())
assert _statuses(result)["a1"] == "validated"
# ---------------------------------------------------------------------------------------------
# (e) the address
# ---------------------------------------------------------------------------------------------
def _tool(approach_ids: list[str] | None) -> tuple[Any, list[DeclaredRequirement]]:
opened: list[Any] = []
declared: list[DeclaredRequirement] = []
tools = navigator_tools(
[str(_BUNDLE)], opened=opened, requirements=declared, approach_ids=approach_ids
)
from portfolio_optimiser.explore import ToolCall
opened += [ToolCall(name="read_file", bundle_id=_BASE_ID, path=n) for n in _CONCEPTS[:3]]
return {t.name: t for t in tools}["declare_requirement"], declared
def test_an_unknown_approach_id_is_refused_naming_the_valid_ones() -> None:
tool, declared = _tool(["a1", "own-proposal"])
reply = tool.func(bundle_id=_BASE_ID, path=_CONCEPTS[0], ref="K", approach_id="a9")
assert reply["refusal"] == "UnknownApproach"
assert "'a1'" in reply["refused"] and "'own-proposal'" in reply["refused"]
assert declared == []
ok = tool.func(bundle_id=_BASE_ID, path=_CONCEPTS[0], ref="K", approach_id="a1")
assert ok["declared"] is True and ok["approach_id"] == "a1"
assert declared == [
DeclaredRequirement(bundle_id=_BASE_ID, path=_CONCEPTS[0], ref="K", approach_id="a1")
]
def test_without_a_commission_any_label_is_recorded_but_never_an_empty_one() -> None:
tool, declared = _tool(None)
assert tool.func(bundle_id=_BASE_ID, path=_CONCEPTS[0], ref="K", approach_id=" ")["refusal"]
assert declared == []
tool.func(bundle_id=_BASE_ID, path=_CONCEPTS[0], ref="K", approach_id="LED-retrofit")
assert requirement_payload(declared) == [
{"bundle_id": _BASE_ID, "path": _CONCEPTS[0], "ref": "K", "approach_id": "LED-retrofit"}
]
# ---------------------------------------------------------------------------------------------
# (f) never counted as a success; the validator's ruling kept
# ---------------------------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_unsupported_is_never_a_success_but_keeps_the_validators_ruling(
tmp_path: Path,
) -> None:
result = await _run(tmp_path, _script(), _mandate())
assert isinstance(result.outcome, Unsupported)
assert not isinstance(result.outcome, ValidatedProposal)
assert result.provenance.validator_decision == "validated"
text = settle(result.coverage)
assert "UNSUPPORTED" in text and "Validated: 0 of 2" in text
artefact = json.loads((tmp_path / "r6-a1-outcome.json").read_text(encoding="utf-8"))
assert artefact["outcome_type"] == "unsupported"
assert artefact["reason"] == UNSUPPORTED_REASON
assert artefact["p50"] > 0
proposal = json.loads((tmp_path / "r6-a1-proposal.json").read_text(encoding="utf-8"))
assert proposal["provenance"]["validator_decision"] == "validated"
def test_the_stage_label_names_the_new_falsifier() -> None:
assert rejection_stage(UNSUPPORTED_REASON) == "unsupported"
def _validated() -> ValidatedProposal:
proposal = SavingsProposal(
project_id=_PID,
measure="m",
affected_items=[AffectedItem(code="X", quantity=10.0, unit_cost=10.0)],
claimed_saving_nok=5.0,
)
return ValidatedProposal(proposal=proposal, p10=1.0, p50=2.0, p90=3.0, nominal_feasible=4.0)
def test_the_outcome_payload_carries_both_halves() -> None:
v = _validated()
payload = outbox.outcome_payload(
Unsupported(proposal=v.proposal, reason=UNSUPPORTED_REASON, validated=v),
checker_verdict="approve",
verdict_id="k",
)
assert (payload["outcome_type"], payload["p90"]) == ("unsupported", 3.0)
# ---------------------------------------------------------------------------------------------
# (g) inactive rule
# ---------------------------------------------------------------------------------------------
@pytest.mark.asyncio
@pytest.mark.parametrize(
("declared", "status"), [(None, "validated"), ([], "unsupported")], ids=["inactive", "active"]
)
async def test_the_rule_acts_only_when_a_declaration_rung_was_offered(
declared: list[DeclaredRequirement] | None, status: str
) -> None:
v = _validated()
async def evaluate(_approach: Approach | None) -> ValidatedProposal | Rejection:
return v
_, rows, _ = await _evaluate_mandate(
Mandate(objective="o", approaches=(Approach(id="a1", label="l"),)),
evaluate,
declared=declared,
)
assert {r.id: r.status for r in rows} == {"a1": status, "own-proposal": status}
# ---------------------------------------------------------------------------------------------
# (h) the judge
# ---------------------------------------------------------------------------------------------
def test_the_judge_attributes_addressed_declarations_and_labels_legacy_ones() -> None:
a1 = Approach(id="a1", label="l")
new = [{"path": "p1", "approach_id": "a1"}, {"path": "p2", "approach_id": "own-proposal"}]
assert stress._attributable(a1, new) == (("p1",), "approach")
assert stress._attributable(Approach(id="a2", label="l"), new) == ((), "absent")
assert stress._attributable(a1, [{"path": "p0"}]) == (("p0",), "run")
@pytest.mark.asyncio
async def test_a_run_offered_no_declaration_rung_keeps_the_validators_ruling(
tmp_path: Path,
) -> None:
"""The road path holds no knowledge base, so no ``declare_requirement`` exists there: nothing
could have been declared, and the rule stays out of it. Drives the REAL ``run_project``
the arm above only proves ``_evaluate_mandate`` honours ``declared=None``, not that the run
passes it."""
from portfolio_optimiser.reference_domain import load_reference_projects
from portfolio_optimiser.validator import proposal_for
project = load_reference_projects()[0]
reply = proposal_for(project, ["05.2", "03.1"], claimed_saving_nok=200_000).model_dump_json()
docs = tmp_path / "docs"
docs.mkdir()
(docs / "kilde.md").write_text("Cost saving measure candidates for the project.\n", "utf-8")
result = await run_project(
project.id,
"local",
docs_dir=str(docs),
store=VerdictStore(verdicts=[]),
client_factory=scripted_factory({"proposer": reply, "checker": _CHECKER}, []),
mandate=Mandate(
objective="o", approaches=(Approach(id="a1", label="l"),), allow_own_proposals=False
),
)
assert _statuses(result) == {"a1": "validated"}

View file

@ -355,7 +355,7 @@ async def test_a_commissioned_run_writes_the_coverage_artefact(tmp_path: Path) -
payload = json.loads((out / "r1-coverage.json").read_text(encoding="utf-8")) payload = json.loads((out / "r1-coverage.json").read_text(encoding="utf-8"))
assert payload["stop_reason"] == "", "nothing cut this run short" assert payload["stop_reason"] == "", "nothing cut this run short"
assert [r["id"] for r in payload["rows"]] == ["a1"] assert [r["id"] for r in payload["rows"]] == ["a1"]
assert payload["rows"][0]["status"] in {"validated", "rejected"} assert payload["rows"][0]["status"] in {"validated", "rejected", "unsupported"}
async def test_a_mandateless_run_leaves_the_outbox_byte_identical(tmp_path: Path) -> None: async def test_a_mandateless_run_leaves_the_outbox_byte_identical(tmp_path: Path) -> None:

View file

@ -328,13 +328,47 @@ def test_row6_green_needs_both_the_probes_and_zero_undeclared() -> None:
assert gate.score_undeclared(_PROBES, failing, _CLEAN, "s").status == gate.RED assert gate.score_undeclared(_PROBES, failing, _CLEAN, "s").status == gate.RED
def test_row6_missing_artefacts_are_never_zero() -> None: def test_row6_missing_artefacts_are_never_zero_and_never_green() -> None:
"""Probes passing and artefacts absent (clean clone, CI, a base mid-rebuild) is NOT a pass:
the row says IKKE MÅLT and fails the exit code."""
row = gate.score_undeclared( row = gate.score_undeclared(
_PROBES, _all_pass(_PROBES), gate.StressMeasure(missing="ut finnes ikke"), "s" _PROBES, _all_pass(_PROBES), gate.StressMeasure(missing="ut finnes ikke"), "s"
) )
assert row.k is None assert row.k is None
assert "ikke målt, artefakter mangler" in row.reason assert "ikke målt" in row.reason
assert " av " in row.line() assert " av " in row.line()
assert (row.status, row.failing) == (gate.NOT_MEASURED, True)
assert gate.exit_code([row]) == 1
def test_row6_artefacts_older_than_the_rule_are_not_measured() -> None:
old = gate.StressMeasure(validated=10, undeclared=10, unaddressed=12, where="x")
row = gate.score_undeclared(_PROBES, _all_pass(_PROBES), old, "s")
assert (row.k, row.status) == (None, gate.NOT_MEASURED)
assert "eldre enn regelen" in row.reason and "approach_id mangler" in row.reason
def test_row7_not_measured_is_not_green_either() -> None:
row = gate.score_named(gate.StressMeasure(missing="borte"), "s")
assert (row.k, row.status, row.failing) == (None, gate.NOT_MEASURED, False)
@pytest.mark.parametrize("missing", ["all", "outcome0", "kept"])
def test_rows_1_2_4_with_missing_files_are_red(tmp_path: Path, missing: str) -> None:
root = _green_rounds(tmp_path / "r")
if missing == "all":
root = tmp_path / "absent"
elif missing == "outcome0":
(root / "0" / "outcome.json").unlink()
else:
(root / "3" / "report.kept.md").unlink()
rows = [
gate.score_rounds(root, 3, _AI),
gate.score_changes(root, 3, _AI),
gate.score_kept(root, 0.8),
]
assert gate.exit_code(rows) == 1
assert gate.GREEN not in {r.status for r in rows} or missing != "all"
def test_row6_measures_the_stress_outboxes_when_they_exist(tmp_path: Path) -> None: def test_row6_measures_the_stress_outboxes_when_they_exist(tmp_path: Path) -> None:
@ -353,6 +387,9 @@ def test_row6_measures_the_stress_outboxes_when_they_exist(tmp_path: Path) -> No
assert m.validated == 0 assert m.validated == 0
pytest.skip(f"stress artefacts not judgeable right now: {m.missing}") pytest.skip(f"stress artefacts not judgeable right now: {m.missing}")
assert (m.validated, m.undeclared, m.named, m.commissioned) == (10, 10, 1, 20) assert (m.validated, m.undeclared, m.named, m.commissioned) == (10, 10, 1, 20)
assert m.unaddressed > 0 # stress round 6 predates approach-addressed declarations
row = gate.score_undeclared(_PROBES, _all_pass(_PROBES), m, "s")
assert (row.k, row.status) == (None, gate.NOT_MEASURED)
def test_row7_is_a_diagnosis_and_never_moves_the_exit_code() -> None: def test_row7_is_a_diagnosis_and_never_moves_the_exit_code() -> None:
@ -427,5 +464,7 @@ def test_the_command_is_red_today_with_every_row_in_its_output(tmp_path: Path) -
assert (rows["types"]["k"], rows["types"]["n"]) == (3, 8) assert (rows["types"]["k"], rows["types"]["n"]) == (3, 8)
assert rows["kept"]["status"] == gate.RED assert rows["kept"]["status"] == gate.RED
assert (rows["maf"]["k"], rows["maf"]["n"], rows["maf"]["status"]) == (3, 8, gate.RED) assert (rows["maf"]["k"], rows["maf"]["n"], rows["maf"]["status"]) == (3, 8, gate.RED)
assert rows["undeclared"]["status"] == gate.RED # Probes green since row 6; stress round 6 predates the rule (or is absent) -> never green.
assert rows["undeclared"]["status"] == gate.NOT_MEASURED
assert rows["undeclared"]["k"] is None
assert rows["named"]["failing"] is False assert rows["named"]["failing"] is False

View file

@ -14,10 +14,9 @@ action is exactly partial. Such a probe goes green only when it is rewritten to
and observe what it does. and observe what it does.
**Row 6 (a validated proposal whose approach declared no requirement).** Two probes against the **Row 6 (a validated proposal whose approach declared no requirement).** Two probes against the
real ``run_project``: no declaration anywhere, and a declaration made by the RUN (the debate) but real ``run_project`` on the shipped micro base, where the debate holds ``declare_requirement``:
not by the approach. The second is the reading the gate measures the stress outboxes with: a no declaration anywhere, and a declaration filed for ANOTHER approach (the run's own proposal).
run-level declaration cannot be attributed to one approach (the judge labels it ``run``), so it Neither may leave ``a1`` validated a declaration counts only under its own approach's id.
does not count as the approach having declared anything.
""" """
from __future__ import annotations from __future__ import annotations
@ -110,42 +109,34 @@ def _mandate() -> Mandate:
) )
async def _statuses(script: dict[str, Any], tmp_path: Path) -> dict[str, str]: async def _statuses(script: dict[str, Any], tmp_path: Path, base: Path) -> dict[str, str]:
result = await run_project( result = await run_project(
_PID, _PID,
"local", "local",
docs_dir=str(_BUNDLE), docs_dir=str(base),
bundle_dir=str(_BUNDLE), bundle_dir=str(base),
store=VerdictStore(verdicts=[]), store=VerdictStore(verdicts=[]),
client_factory=scripted_factory(script, []), client_factory=scripted_factory(script, []),
mandate=_mandate(), mandate=_mandate(),
outbox_dir=str(tmp_path), outbox_dir=str(tmp_path / "out"),
run_id="v1-row6", run_id="v1-row6",
) )
return {row.id: row.status for row in result.coverage} return {row.id: row.status for row in result.coverage}
@pytest.mark.xfail(strict=True, reason="row 6: no stage refuses a validation with no declaration") def _declaring_script(base: Path, approach_id: str) -> dict[str, Any]:
@pytest.mark.asyncio concepts = [f.name for f in okf.navigate_bundle(str(base)).context_files][:3]
async def test_row6_an_approach_that_declared_nothing_cannot_be_validated(tmp_path: Path) -> None: return {
statuses = await _statuses({"proposer": _VALID_REPLY, "checker": _CHECKER_REPLY}, tmp_path)
debate = json.loads((tmp_path / "v1-row6-debate.json").read_text(encoding="utf-8"))
assert debate["requirements"] == [] # precondition: nothing was declared anywhere
assert statuses["a1"] != "validated", "validated without any declared requirement"
@pytest.mark.xfail(strict=True, reason="row 6: a run-level declaration still stands in")
@pytest.mark.asyncio
async def test_row6_a_run_level_declaration_does_not_stand_in_for_the_approach(
tmp_path: Path,
) -> None:
concepts = [f.name for f in okf.navigate_bundle(str(_BUNDLE)).context_files][:3]
script = {
"proposer": [ "proposer": [
*({"call": "read_file", "args": {"bundle_id": _BASE_ID, "path": n}} for n in concepts), *({"call": "read_file", "args": {"bundle_id": _BASE_ID, "path": n}} for n in concepts),
{ {
"call": "declare_requirement", "call": "declare_requirement",
"args": {"bundle_id": _BASE_ID, "path": concepts[0], "ref": "probe"}, "args": {
"bundle_id": _BASE_ID,
"path": concepts[0],
"ref": "Krav 1.1-1",
"approach_id": approach_id,
},
}, },
_VALID_REPLY, _VALID_REPLY,
_VALID_REPLY, _VALID_REPLY,
@ -154,7 +145,30 @@ async def test_row6_a_run_level_declaration_does_not_stand_in_for_the_approach(
], ],
"checker": _CHECKER_REPLY, "checker": _CHECKER_REPLY,
} }
statuses = await _statuses(script, tmp_path)
debate = json.loads((tmp_path / "v1-row6-debate.json").read_text(encoding="utf-8"))
assert [r["path"] for r in debate["requirements"]] == [concepts[0]] # precondition def _declared(tmp_path: Path) -> list[dict[str, Any]]:
debate = json.loads((tmp_path / "out" / "v1-row6-debate.json").read_text(encoding="utf-8"))
return list(debate["requirements"])
@pytest.mark.asyncio
async def test_row6_an_approach_that_declared_nothing_cannot_be_validated(tmp_path: Path) -> None:
base = _BUNDLE
statuses = await _statuses(
{"proposer": _VALID_REPLY, "checker": _CHECKER_REPLY}, tmp_path, base
)
assert _declared(tmp_path) == [] # precondition: nothing was declared anywhere
assert statuses["a1"] != "validated", "validated without any declared requirement"
@pytest.mark.asyncio
async def test_row6_a_run_level_declaration_does_not_stand_in_for_the_approach(
tmp_path: Path,
) -> None:
"""The debate declares a requirement — but for the run's OWN proposal, not for ``a1``."""
base = _BUNDLE
statuses = await _statuses(_declaring_script(base, "own-proposal"), tmp_path, base)
declared = _declared(tmp_path)
assert [d.get("approach_id") for d in declared] == ["own-proposal"] # precondition
assert statuses["a1"] != "validated", "validated on a declaration the approach never made" assert statuses["a1"] != "validated", "validated on a declaration the approach never made"