feat(p22): declare_requirement answers with a COMPARISON, not a confirmation

P19 DEL A made a direction name the requirement that binds it; P20/A1 made the reply carry the
DOCUMENT's own title and number instead of echoing the caller's arguments. Re-measured at the head
of this session against the six round-5 debate traces: requirement_hit is 0 of 20 approach rows and
0 of 12 declarations -- the third round in a row at zero. P21/C1 made the runs LOOK first and it
worked on its own terms (distinct documents before a declaration went 1,1,1,2,5,13 -> 3,3,5,7,11,12)
and the hit did not move. The runs were made to read MORE, not righter.

The reply now compares: it names the directions the run was commissioned to pursue and says which
of their words appear in the declared document's own title and number, or that none do.

A REPORT, never a gate -- the declaration is recorded either way, because a requirement can bind a
measure without sharing a word with the name someone gave it, which is exactly how the alternative
rule the C1 measurement rejected failed one rung over. The words compared are the DOCUMENT's, never
`ref`: a comparison against the caller's own argument can only ever agree. Matching is generous in
both directions, and that failure direction is chosen -- a false "no overlap" pushes a model away
from a declaration that was right, a false "overlap" merely keeps the report quiet.

MEASURED BEFORE IT WAS BUILT, offline against the six traces as the order required (no paid calls
in DEL B): the rule speaks on 10 of 12 declarations and stays quiet on 2. A rule that spoke on 12
of 12, or on 0 of 12, could not tell the two classes apart.

`labels` defaults to empty, so every call site written before today is byte-identical and the three
keys are ABSENT rather than empty -- "there was nothing to compare against" and "we compared and
found nothing" are different facts. RUN-level, as the declaration itself is (P19 A4).

Also re-measured: the order cited requirement_hit as "0 of 12". The field is per APPROACH (0 of 20);
12 is the number of DECLARATIONS (7 distinct, 0 hits). Both zero, so the conclusion stands, but they
are two populations.

Load-bearing MEASURED (tests/test_requirement_comparison_loadbearing.py, 8 arms), eight mutations
all red against the WHOLE suite + green control 1881/5 (from 1873/5, superset, 0 removed) and
golden demo-transcript.stdout BYTE-UNCHANGED (shasum -a 1 of the CONTENT =
ea8c534773acdbe41ae68f2c55724d69aaf8be4f): B1 detach the run.py wiring (1 red, that arm alone) -
B2 always report an overlap (5) - B3 never report one (2) - B4 compare against the caller's ref
(1, that arm alone) - B5 make it a gate (5) - B6 emit the keys with no directions (2, one an OLDER
independent witness) - B7 exact token equality instead of substring (1) - B8 drop the minimum word
length (3).

Honesty limits, stated: no LIVE model has read the comparison yet (DEL D is the measurement); the
report cannot say a requirement IS right, only that it shares no word with the direction; and
finding 4 (`named` 1/20) is this same matter from the other side, so DEL D measures it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-16 00:04:47 +02:00
commit 9072359606
5 changed files with 360 additions and 1 deletions

View file

@ -3011,6 +3011,49 @@ Python ≥3.10. MAF (`agent-framework-core` 1.16.0, `-orchestrations` 1.1.1 —
uttalt:** ingen betalt kjoering bekrefter ennaa at nekten endrer utfallet levende (DEL D er uttalt:** ingen betalt kjoering bekrefter ennaa at nekten endrer utfallet levende (DEL D er
maalingen); vinduet er aldri naadd av noe maalt, saa avkortnings-grenen er oevd kun syntetisk; og maalingen); vinduet er aldri naadd av noe maalt, saa avkortnings-grenen er oevd kun syntetisk; og
skjemaet naar fortsatt ikke prompten, saa foerste forsoek gjetter som foer. skjemaet naar fortsatt ikke prompten, saa foerste forsoek gjetter som foer.
- **`declare_requirement` svarer med en SAMMENLIGNING, ikke en bekreftelse (P22 DEL B, 16.09):**
P19 DEL A gjorde at en retning MAA navngi kravet som binder den, og P20/A1 lot svaret baere
DOKUMENTETS egen tittel og nummer i stedet for aa ekko kallerens argumenter. MAALT paa nytt ved
starten av oekt 126 mot de seks runde-5-sporene: `requirement_hit` er **0 av 20** approach-rader
og **0 av 12** erklaeringer — tredje runde paa rad paa null. P21/C1 fikk kjoeringene til aa SE
seg om, og virket paa sine egne premisser (distinkte dokumenter foer en erklaering
`1·1·1·2·5·13``3·3·5·7·11·12`), men treffet rikket seg ikke: kjoeringene ble faatt til aa lese
MER, ikke RIKTIGERE. **Ordrens egen denominator var upresis, og det er maalt:** feltet
`requirement_hit` er per APPROACH (0 av 20), mens 12 er antallet ERKLAERINGER (7 distinkte, 0
treff) — begge null, saa konklusjonen staar, men de er to populasjoner.
`_label_overlap` er den ene sammenligningen: hvilke ord fra kommisjonens retninger som finnes i
det ERKLAERTE DOKUMENTETS egen tittel og nummer. **En RAPPORT, aldri en gate** — erklaeringen
registreres uansett (M B5 → 5 roede): et krav kan binde et tiltak uten aa dele ett ord med navnet
noen ga det, hvilket er NOEYAKTIG slik den alternative regelen P21/C1 maalte og forkastet feilet,
ett trinn over. **Ordene som sammenlignes er DOKUMENTETS, aldri `ref`** — kallerens eget argument
ekkoet tilbake, og en sammenligning mot kallerens input kan bare vaere enig (M B4 → 1 roed, den
armen ALENE; P20/A1s regel anvendt paa halvdelen P20 ikke naadde). **Sjenerøs i BEGGE retninger**
(delstreng hver vei, saa `rundkjoring` moeter `Rundkjoringer` og `senkekostnader` moeter `senke`),
og feilretningen er VALGT: rapporten sier ett av to, og bare ett av dem kan gjoere skade — en
falsk «ingen overlapp» skyver en modell BORT fra en erklaering som var riktig, mens en falsk
«overlapp» bare gjoer rapporten stille. Delstreng feiler mot stille (P18s `filter` valgte samme
retning av samme grunn; M B7 → 1 roed, M B2 → 5, M B3 → 2). `_LABEL_WORD_MIN = 4`, ellers deler
hver label «for»/«med»/«til» med et halvt korpus (M B8 → 3 roede).
**MAALT FOER DEN BLE BYGGET**, offline mot de seks sporene slik ordren krevde (ingen betalte kall
i DEL B): regelen TALER paa **10 av 12** erklaeringer og tier paa 2 (begge fv412, paa
`materialer`). En regel som talte paa 12 av 12 — eller paa 0 av 12 — kunne ikke skilt de to
klassene, samme proeve P21/C1s terskel maatte bestaa. **`labels` DEFAULTER til tom**, saa hvert
kallsted skrevet foer i dag er BYTE-IDENTISK og de tre noeklene UTELATES (fravaerende, ikke tomme:
«det fantes ingenting aa sammenligne mot» og «vi sammenlignet og fant ingenting» er ulike fakta,
og bare ett av dem er sant der; M B6 → 2 roede, hvorav ett ELDRE uavhengig vitne i
`test_binding_requirement`). Utforskningen mynter sine egne retninger, saa den HAR ingen ved
erklaerings-tid. **RUN-nivaa, som erklaeringen selv (P19 A4):** debatten erklaerer ÉN gang per
kjoering, saa svaret navngir HVER retning kjoeringen baerer i stedet for aa velge én den ikke kan
tilskrive. Wiringen i `run.py` maales ATFERDSMESSIG — en kilde-assert er en lint (oekt 77s funn),
saa armen driver den EKTE debatten med et steg-manus som erklaerer og leser sammenligningen ut av
verktoeyets eget svar (M B1 → 1 roed, den armen ALENE). Load-bearing MAALT
(`tests/test_requirement_comparison_loadbearing.py`, 8 armer), **aatte mutasjoner alle roede mot
HELE suiten** + groenn kontroll **1881/5** (fra 1873/5, supersett, 0 fjernet) og golden
BYTE-UENDRET (`shasum -a 1` av INNHOLDET = `ea8c534773acdbe41ae68f2c55724d69aaf8be4f`).
**AErlighets-grenser, uttalt:** ingen LEVENDE modell har lest sammenligningen
(structured-output-grensens klasse) — DEL D er maalingen; rapporten kan ikke si at et krav ER
riktig, bare at det ikke deler ett ord med retningen; og funn 4 (`named` 1/20) er denne saken
sett fra den andre siden, saa den maales av DEL D og ikke av en egen soem.
- **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet. - **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet.
- Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase. - Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase.

View file

@ -508,6 +508,15 @@ when the seam is detached, so the loop cannot silently degrade into theater.
requirement can be seen to be wrong. The route is named in the instruction as well as the requirement can be seen to be wrong. The route is named in the instruction as well as the
answer: pass `read_dir` a `filter` word from the approach's own label. answer: pass `read_dir` a `filter` word from the approach's own label.
On a commissioned run the reply goes one step further and **compares**: it names the directions
the run was commissioned to pursue and says which of their words appear in the declared
document's own title and number — or that none do. It is a report, never a gate: the declaration
is recorded either way, because a requirement can bind a measure without sharing a word with the
name someone gave it. (Measured over three stress rounds, not one declaration named a concept the
answer key asks for, and making the runs read more did not change that. The rule was replayed
against those traces before it was built: it speaks on 10 of 12 declarations and stays quiet on
2.)
**What the commissioner counts as success reaches the readers.** A mandate's `success_criteria` **What the commissioner counts as success reaches the readers.** A mandate's `success_criteria`
used to reach the announcement and nothing else. It is now restated verbatim in the debate's task used to reach the announcement and nothing else. It is now restated verbatim in the debate's task
message — the prompt where `declare_requirement` is available — through one renderer, and is message — the prompt where `declare_requirement` is available — through one renderer, and is

View file

@ -24,6 +24,7 @@ This module imports ``agent_framework.orchestrations`` and therefore may never b
from __future__ import annotations from __future__ import annotations
import json import json
import re
import sys import sys
from collections.abc import Awaitable, Callable, Mapping, Sequence from collections.abc import Awaitable, Callable, Mapping, Sequence
from dataclasses import dataclass, field from dataclasses import dataclass, field
@ -1126,12 +1127,62 @@ def _refused_text(exc: Exception) -> str:
return f"REFUSED ({_refusal_kind(exc)}): {exc}" return f"REFUSED ({_refusal_kind(exc)}): {exc}"
#: P22 DEL B - the shortest word of a direction's label that can carry meaning into a comparison.
#: Below this every label shares "for", "med", "ny" with half a corpus and the report would speak
#: of an overlap nobody meant.
_LABEL_WORD_MIN: Final = 4
def _label_overlap(labels: Sequence[str], *document_text: str) -> tuple[str, ...]:
"""Which words of the commission's directions appear in the DECLARED DOCUMENT's own text.
The measured defect (P21 funn 2, re-measured at the head of okt 126): ``requirement_hit`` is
**0 of 20** approach rows over round 5's six paid runs and **0 of 12** declarations - a third
round in a row - and P21/C1 moved the documents read before a declaration from 1,1,1,2,5,13 to
3,3,5,7,11,12 without moving the hit. The runs were made to read MORE, not righter. P20/A1
already gives back the document's own title and number; what nobody said was whether that
document has anything to do with the direction the run is committed to.
**A REPORT, never a gate.** The declaration is recorded either way. A gate on word overlap
would refuse legitimate declarations - a requirement can bind a measure without sharing a word
with the name someone gave it - which is the P21/C1 alternative rule's failure, one rung over.
**Generous in BOTH directions, and that is the failure direction chosen on purpose.** A token
matches when it is a substring of a document token or the document token is a substring of it,
so ``rundkjoring`` meets ``Rundkjoringer`` and ``asfaltdekke`` meets ``Asfalt``. The report
says one of two things, and only one of them can do harm: a false "no overlap" pushes a model
away from a declaration that was right, while a false "overlap" merely keeps the report quiet.
Substring matching fails towards quiet. (P18's ``filter`` chose the same direction for the same
reason: a substring fails towards showing MORE, which can be narrowed.)
MEASURED offline against the six round-5 debate traces before this was built: the rule speaks
on **10 of 12** declarations and stays quiet on 2 (both fv412, on ``materialer``). A rule that
spoke on 12 of 12 or on 0 of 12 could not tell the two classes apart, which is the same test
P21/C1's threshold had to pass.
"""
haystack = {
token
for text in document_text
for token in re.split(r"[\W_]+", text.lower())
if len(token) >= _LABEL_WORD_MIN
}
shared: set[str] = set()
for label in labels:
for token in re.split(r"[\W_]+", label.lower()):
if len(token) < _LABEL_WORD_MIN:
continue
if any(token in other or other in token for other in haystack):
shared.add(token)
return tuple(sorted(shared))
def navigator_tools( def navigator_tools(
bundle_dirs: Sequence[str], bundle_dirs: Sequence[str],
*, *,
dimension: str | None = None, dimension: str | None = None,
opened: list[ToolCall] | None = None, opened: list[ToolCall] | None = None,
requirements: list[DeclaredRequirement] | None = None, requirements: list[DeclaredRequirement] | None = None,
labels: Sequence[str] = (),
) -> list[FunctionTool]: ) -> list[FunctionTool]:
"""The navigator's tools: survey the catalogue, open one base, read one document — and, when """The navigator's tools: survey the catalogue, open one base, read one document — and, when
the caller offers the two sinks, DECLARE the requirement that binds a direction. the caller offers the two sinks, DECLARE the requirement that binds a direction.
@ -1465,7 +1516,7 @@ def navigator_tools(
# ``binds`` says out loud what the declaration is for: without it the reply is data with no # ``binds`` says out loud what the declaration is for: without it the reply is data with no
# instruction, and the instruction is the whole correction. # instruction, and the instruction is the whole correction.
declared = _declared_document(index, bundle_id, path) declared = _declared_document(index, bundle_id, path)
return { reply: dict[str, Any] = {
"declared": True, "declared": True,
"bundle_id": bundle_id, "bundle_id": bundle_id,
"path": path, "path": path,
@ -1477,6 +1528,30 @@ def navigator_tools(
"declaration of a requirement that is not about the measure is worth nothing." "declaration of a requirement that is not about the measure is worth nothing."
), ),
} }
# P22 DEL B: turn the reply into a COMPARISON against what this run was commissioned to
# do. P20/A1 made the reply carry the document's own words; measured, that was not enough
# on its own - three rounds of declarations and not one named a fasit concept. The words
# compared are the DOCUMENT's (read off the base), never ``ref``, which is the caller's own
# argument echoed back: a comparison against the caller's input can only ever agree.
# Absent labels (the exploration mints its own directions, so there are none at
# declaration time) the two keys are omitted and the reply is byte-identical to P20's.
if labels:
overlap = _label_overlap(labels, declared[0], declared[1])
listed = ", ".join(repr(label) for label in labels)
reply["directions"] = list(labels)
reply["overlap"] = list(overlap)
reply["compare"] = (
f"You declared {declared[0]!r} ({declared[1]!r}) for these directions: {listed}. "
+ (
f"Words they share: {', '.join(overlap)}."
if overlap
else "No word of any of them appears in the document's own title or number. "
"If this document is not about the measure you declared it for, it is the "
"wrong requirement: filter the level again with a word from the direction "
"itself and read the candidates that come back."
)
)
return reply
tools = [list_bundles, read_bundle, read_dir, read_file] tools = [list_bundles, read_bundle, read_dir, read_file]
if requirements is not None: if requirements is not None:

View file

@ -1294,6 +1294,12 @@ async def run_project(
dimension=dimension_id, dimension=dimension_id,
opened=debate_tool_calls, opened=debate_tool_calls,
requirements=debate_requirements, requirements=debate_requirements,
# P22 DEL B: the commission's own direction names, so the declaration rung can
# answer with a COMPARISON instead of a confirmation. RUN-level, exactly as
# the declaration is (P19 A4): the debate declares once per run, so the reply
# names every direction the run carries rather than picking one it cannot
# attribute. Without a mandate this is empty and the reply is unchanged.
labels=[a.label for a in mandate.approaches] if mandate else (),
) )
) )
# What the navigation could NOT reach, taken from the run's ONE walk. The road path below # What the navigation could NOT reach, taken from the run's ONE walk. The road path below

View file

@ -0,0 +1,226 @@
"""P22 DEL B — ``declare_requirement`` answers with a COMPARISON, not a confirmation.
MEASURED (P21 funn 2, re-measured at the head of okt 126 against the six round-5 debate traces):
``requirement_hit`` is **0 of 20** approach rows and **0 of 12** declarations the third round in
a row at zero. P21/C1 made the runs LOOK first, and it worked on its own terms: distinct documents
opened before a declaration went from 1,1,1,2,5,13 to 3,3,5,7,11,12. The hit did not move. The
runs were made to read more, not righter.
P20/A1 had already made the reply carry the document's OWN title and number instead of echoing the
caller's arguments. What nobody said was whether that document has anything to do with the
direction the run is committed to. This turns the reply into that comparison.
**A REPORT, never a gate.** The declaration is recorded either way, and arm (c) is what holds that
open. A gate on word overlap would refuse legitimate declarations a requirement can bind a
measure without sharing a word with the name someone gave it which is precisely how the
alternative rule P21/C1 measured and rejected failed, one rung over.
**Measured before it was built**, offline against the six traces: the rule speaks on **10 of 12**
declarations and stays quiet on 2 (both fv412, on ``materialer``). A rule that spoke on 12 of 12,
or on 0 of 12, could not tell the two classes apart.
"""
from __future__ import annotations
from pathlib import Path
from typing import Any
import pytest
from portfolio_optimiser.simulation import scripted_factory
from portfolio_optimiser import okf
from portfolio_optimiser.explore import ToolCall, navigator_tools
from portfolio_optimiser.mandate import Approach, Mandate
from portfolio_optimiser.run import run_project
_EXAMPLES = Path(__file__).resolve().parents[1] / "shared" / "examples"
_TUNNEL = _EXAMPLES / "tunnel-hauglia"
_BASE_ID = "tunnel-hauglia"
#: The fixture document the arms declare. Its title is ASCII-clean, which is what lets a label
#: share a word with it without a marker carrying a multibyte character into a scripted run.
_DOC = "tiltak-portalskjerming.md"
#: A direction whose words are IN that title ("Portalskjerming: senke L20 ...").
_MATCHING = "Billigere portalskjerming"
#: A direction that shares nothing with it. Checked against the document's own tokens, both ways.
_FOREIGN = "Asfaltdekke gjenbruk"
def _wired(labels: tuple[str, ...]) -> tuple[dict[str, Any], list[ToolCall], list[Any]]:
opened: list[ToolCall] = []
declared: list[Any] = []
tools = navigator_tools((str(_TUNNEL),), opened=opened, requirements=declared, labels=labels)
return {t.name: t for t in tools}, opened, declared
def _declare(
labels: tuple[str, ...], *, ref: str = "Krav 1.1-1"
) -> tuple[dict[str, Any], list[Any]]:
tools, opened, declared = _wired(labels)
for name in [f.name for f in okf.navigate_bundle(str(_TUNNEL)).context_files][:3]:
opened.append(ToolCall(name="read_file", bundle_id=_BASE_ID, path=name))
opened.append(ToolCall(name="read_file", bundle_id=_BASE_ID, path=_DOC))
answer = tools["declare_requirement"].func(bundle_id=_BASE_ID, path=_DOC, ref=ref)
return answer, declared
# --- (a)/(b) the discriminator: SAME document, SAME ref, different directions ---------------------
def test_a_direction_that_shares_a_word_is_told_which_one() -> None:
"""KNOWN-POSITIVE. The document's title carries the direction's own word, and the reply says
so the half that keeps the report from being one that only ever complains."""
answer, _ = _declare((_MATCHING,))
assert answer["overlap"] == ["portalskjerming"], answer["overlap"]
assert "portalskjerming" in answer["compare"]
def test_a_direction_that_shares_nothing_is_told_that_too() -> None:
"""KNOWN-NEGATIVE, and the measured case: the SAME document and the SAME ``ref`` as the arm
above the only difference is the direction. That is what makes this a comparison rather than
a confirmation, and it is exactly the 10-of-12 class the round-5 traces fall into."""
answer, _ = _declare((_FOREIGN,))
assert answer["overlap"] == []
assert "No word of any of them appears" in answer["compare"], answer["compare"]
assert repr(_FOREIGN) in answer["compare"], answer["compare"]
# --- (c) a REPORT, never a gate ------------------------------------------------------------------
def test_a_declaration_with_no_overlap_is_still_recorded() -> None:
"""The whole difference between this rung and P21/C1's. A requirement can bind a measure
without sharing a word with the name someone gave it, so refusing here would refuse
legitimate declarations the failure of the alternative rule the C1 measurement rejected."""
answer, declared = _declare((_FOREIGN,))
assert answer["declared"] is True
assert len(declared) == 1, "a reported mismatch must not swallow the declaration"
# --- (d) it reads the DOCUMENT, never the caller's own argument ----------------------------------
def test_the_comparison_never_reads_the_callers_own_ref() -> None:
"""``ref`` is the caller's argument echoed back, and a comparison against the caller's own
input can only ever agree P20/A1's rule (read off the base, never off the arguments)
applied to the half P20 did not reach. A ``ref`` stuffed with the direction's words must not
manufacture an overlap."""
answer, _ = _declare((_FOREIGN,), ref="Asfaltdekke gjenbruk krav")
assert answer["overlap"] == [], answer["overlap"]
# --- (e) no directions -> the P20 reply, unchanged ------------------------------------------------
def test_without_directions_the_reply_is_the_one_p20_shipped() -> None:
"""The exploration mints its own directions, so at declaration time it HAS none — and every
call site that passes no labels must be byte-identical to before. Absent keys, not empty ones:
"there was nothing to compare against" and "we compared and found nothing" are different
facts, and only one of them is true here."""
answer, _ = _declare(())
assert set(answer) == {
"declared",
"bundle_id",
"path",
"ref",
"title",
"req_number",
"binds",
}, sorted(answer)
# --- (f)/(g) the matching rule's two measured properties -----------------------------------------
def test_matching_is_generous_in_both_directions() -> None:
"""The failure direction chosen on purpose. A direction whose word is a PREFIX of the
document's own word counts, and so does the reverse — Norwegian inflects ("rundkjoring" vs
"Rundkjoringer") and a strict rule would report "no overlap" on a declaration that was right,
which is the only one of the two errors that can push a model away from a correct answer."""
# label word LONGER than the document's own ("portalskjerming" is a prefix of it)
assert _declare(("Portalskjermingen paa nordsiden",))[0]["overlap"] == ["portalskjermingen"]
# and SHORTER: the document says "senke", the direction "senkekostnader"
assert _declare(("Senkekostnader",))[0]["overlap"] == ["senkekostnader"]
def test_short_words_cannot_manufacture_an_overlap() -> None:
"""Below the minimum every label shares "for", "med", "til" with half a corpus, and the report
would speak of an overlap nobody meant. The control is the arm above: the same rule that
rejects these still finds the real word."""
answer, _ = _declare(("Ny og med til for",))
assert answer["overlap"] == [], answer["overlap"]
# --- (h) the commission's labels actually REACH the tool in a run --------------------------------
_PID = "TUNNEL-HAUGLIA"
_VERDICT_INPUT = {"decision": "approved", "rationale": "expert reviewed (test)"}
_VALID_REPLY = (
'{"measure":"LED-retrofit","affected_items":'
'[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],'
'"claimed_saving_nok":30000}'
)
_CHECKER_REPLY = "Reasoning holds.\nVERDICT: APPROVE"
@pytest.mark.asyncio
async def test_a_commissioned_run_reaches_the_tool_with_its_own_directions() -> None:
"""The WIRING, measured behaviourally rather than by reading ``run.py``. A source assertion is
a lint; this drives the real debate with a step manuscript that declares, and reads the
comparison back out of the tool's own answer. Without ``labels=`` in ``run.py`` the reply
carries no ``compare`` at all and this arm falls."""
concepts = [f.name for f in okf.navigate_bundle(str(_TUNNEL)).context_files]
script = {
"proposer": [
*(
{"call": "read_file", "args": {"bundle_id": _BASE_ID, "path": name}}
for name in concepts[:3]
),
{"call": "read_file", "args": {"bundle_id": _BASE_ID, "path": _DOC}},
{
"call": "declare_requirement",
"args": {"bundle_id": _BASE_ID, "path": _DOC, "ref": "Krav 1.1-1"},
},
_VALID_REPLY,
_VALID_REPLY,
_VALID_REPLY,
_VALID_REPLY,
],
"checker": _CHECKER_REPLY,
}
seen: list[str] = []
def factory(role: str) -> Any:
client = scripted_factory(script, [])(role)
original = client._inner_get_response
def recording(*, messages, options, stream=False, **kwargs): # type: ignore[no-untyped-def]
for message in messages:
for content in getattr(message, "contents", ()) or ():
value = getattr(content, "result", None)
if value:
seen.append(str(value))
return original(messages=messages, options=options, stream=stream, **kwargs)
client._inner_get_response = recording
return client
await run_project(
_PID,
"local",
docs_dir=str(_TUNNEL),
bundle_dir=str(_TUNNEL),
verdict_input=_VERDICT_INPUT,
client_factory=factory,
mandate=Mandate(
objective="Find savings",
approaches=(Approach(id="a1", label=_FOREIGN, description="expert's reason"),),
),
)
blob = "\n".join(seen)
assert "No word of any of them appears" in blob, (
"the commission's directions never reached the declaration rung: " + blob[-600:]
)
assert _FOREIGN in blob