feat(s2c): debatten navigerer basen i stedet for aa faa den utlevert [skip-docs]
MAJOR-3/S7a-3 gjorde utforskningen billig og lot pipelinen staa. Maalt paa K2
(630 konsepter, S7bs eget instrument, kjent-positiv-kontrollen reprodusert
eksakt FOER bruk): okf.bundle_context er 648 962 o200k-tokens og rir i TRE
kopier = 1 947 342 = 99,1 % av en kjoerings prompt-tokens.
Et premiss i maaledokumentet ble presisert foerst: de tre kopiene er tre
DEBATT-turer (proposer x2, checker x1), mens genererings-prompten er 156
tokens, fordi gen_context = debate_output or context. Det avgjorde formen -
generering trengte ingen egen soem, for aa binde `context` binder
siste-utvei-fallbacken ved konstruksjon.
run_project sender naa en PEKER (fast tekst + erklaert bundle_id + antall
konseptdokumenter i scope + stigen, O(1) i korpuset) og gir debatten de SAMME
fire verktoeyene utforskningen bruker - explore.navigator_tools gjenbrukt,
aldri en andre kopi av policyen.
Etter: 753 tokens like-for-like (samme manus, samme fire prompter, -99,96 %)
og 8 942 med en debatt som faktisk gaar stigen (-99,5 %), mot operatoerens
terskel 195 000 = 4,6 % av taket. Validert besparelse og validatorens dom er
UENDRET (850 000 NOK av 3 852 500, 2 av 5 felt paa stage 4 og 5, samme
dom-noekkel), og utforskningens 18 355 er uendret til tokenet.
§4.1a maatte flytte, ikke forsvinne: dimensjonsfilteret bodde i renderingen og
bor naa i VERKTOEYENE, paa begge trinn - en listing som skjuler et fremmed
dokument mens read_file serverer det paa sti er et filter i navnet alene.
okf.in_dimension er eneste predikat.
Sporet er kaller-eid (ExplorationToolRecorder -> RunResult.debate_tool_calls ->
{run_id}-debate.json fra en finally) og skrives ogsaa TOMT: en debatt som
navigerer ingenting ER S2c-regresjonen, saa den maa kunne leses.
Load-bearing MAALT: aatte mutasjoner roede mot HELE suiten, groenn kontroll
1306/5 (fra 1295/5), golden demo-transcript.stdout BYTE-UENDRET
(shasum -a 1 av innholdet = ea8c534773acdbe41ae68f2c55724d69aaf8be4f).
M7 falsifiserte seg selv, ikke gaten - staar som maalt.
Maaling: docs/2026-09-04-s2c-debatt-k2.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
5d8844fef5
commit
da5f10f140
13 changed files with 1044 additions and 98 deletions
|
|
@ -214,6 +214,17 @@ class ExplorationError(RuntimeError):
|
|||
"""The exploration cannot be honoured as configured, or produced something unreadable."""
|
||||
|
||||
|
||||
class DimensionScopeRefused(ValueError):
|
||||
"""A navigator asked for a document belonging to ANOTHER dimension than the run is scoped to.
|
||||
|
||||
A ``ValueError``, the ``BundlePathNotFound``/``BundleIdMismatch`` precedent: the caller is a
|
||||
model choosing a path, so the refusal must land on the CLI's refusal tuple and hosting's 400
|
||||
arm rather than the crash channel. It is deliberately NOT an ``ExplorationError``
|
||||
(a ``RuntimeError``): this is a refused read inside a run that is otherwise fine, not an
|
||||
exploration that cannot be honoured as configured.
|
||||
"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LedgerEntry:
|
||||
"""One progress-ledger round, reduced to the five fields the manager steers on (C.2).
|
||||
|
|
@ -375,6 +386,20 @@ class ExplorationTrace:
|
|||
tokens_spent: int = 0
|
||||
|
||||
|
||||
def tool_call_payload(calls: Sequence[ToolCall]) -> list[dict[str, Any]]:
|
||||
"""The ONE rendering of a tool trace into plain data, in CALL ORDER.
|
||||
|
||||
Two surfaces now write one: ``trace_payload`` for ``{run_id}-exploration.json`` and
|
||||
``run_project`` for ``{run_id}-debate.json`` (S2c). Two copies of "what a recorded call looks
|
||||
like" would drift into two answers about the same fact, which is the kø-(p) defect — and here
|
||||
the drift would land in the artefacts an operator reads to find out what a paid run opened.
|
||||
|
||||
Plain mappings only, so the RAW output layer stays MAF-free (``outbox.py`` may not import
|
||||
this module).
|
||||
"""
|
||||
return [{"name": call.name, "bundle_id": call.bundle_id, "path": call.path} for call in calls]
|
||||
|
||||
|
||||
def trace_payload(
|
||||
trace: ExplorationTrace, *, stop: str | None, completed: bool, mandate: Mandate | None
|
||||
) -> dict[str, Any]:
|
||||
|
|
@ -442,10 +467,7 @@ def trace_payload(
|
|||
}
|
||||
for call in trace.quick_validations
|
||||
],
|
||||
"tool_calls": [
|
||||
{"name": call.name, "bundle_id": call.bundle_id, "path": call.path}
|
||||
for call in trace.tool_calls
|
||||
],
|
||||
"tool_calls": tool_call_payload(trace.tool_calls),
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -859,7 +881,9 @@ def _index_excerpt(body: str) -> tuple[str, bool]:
|
|||
return (body[:cut] if cut > 0 else body[:_CATALOGUE_EXCERPT_CHARS]), True
|
||||
|
||||
|
||||
def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
|
||||
def navigator_tools(
|
||||
bundle_dirs: Sequence[str], *, dimension: str | None = None
|
||||
) -> list[FunctionTool]:
|
||||
"""The navigator's three tools: survey the catalogue, open one base, read one document.
|
||||
|
||||
Progressive disclosure, not stuffing (målbilde §2/§4): ``list_bundles`` never returns content,
|
||||
|
|
@ -898,6 +922,13 @@ def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
|
|||
The listing is built from ``Bundle.context_files``, which EXCLUDES the ``type: verdict`` layer
|
||||
by construction — prior verdicts reach a hypothesis only through the gated ExpeL fold inside
|
||||
``run_project``, never by being read as context here.
|
||||
|
||||
**``dimension`` scopes BOTH rungs, and both halves are the promise** (§4.1a, carried over in
|
||||
S2c when the DEBATE started navigating instead of being handed ``bundle_context``). The filter
|
||||
used to live in the rendering; with navigation it has to live in the tools, and a listing that
|
||||
hides a foreign-dimension document while ``read_file`` still serves it by path is a filter in
|
||||
name only — a model-chosen path is untrusted input, so the gate belongs where the bytes leave.
|
||||
``None`` (the exploration's own call) admits everything, byte-identical to before.
|
||||
"""
|
||||
index = _bundle_index(bundle_dirs)
|
||||
|
||||
|
|
@ -921,7 +952,9 @@ def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
|
|||
"id": bundle_id,
|
||||
"index_excerpt": excerpt,
|
||||
"index_truncated": truncated,
|
||||
"documents": len(bundle.context_files),
|
||||
"documents": sum(
|
||||
1 for f in bundle.context_files if okf.in_dimension(f, dimension)
|
||||
),
|
||||
"verdict_count": len(bundle.verdicts),
|
||||
# Tolerant on CONTENT, fail-fast on the PATH: an operator's bad directory is
|
||||
# refused by navigate_bundle above, while a navigable base that simply has no
|
||||
|
|
@ -953,7 +986,7 @@ def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
|
|||
# ONE renderer for both rungs (kø-(p)): this tool and ``read_dir`` differ only in WHICH
|
||||
# level they ask for, and two copies of a listing rule would drift into two answers about
|
||||
# one bundle. ``okf`` owns it, so the context seam stays framework-neutral.
|
||||
return okf.directory_listing(bundle)
|
||||
return okf.directory_listing(bundle, dimension=dimension)
|
||||
|
||||
@tool(
|
||||
name="read_dir",
|
||||
|
|
@ -968,7 +1001,7 @@ def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
|
|||
bundle_dir = _resolve_bundle(index, bundle_id)
|
||||
bundle = okf.navigate_bundle(bundle_dir)
|
||||
okf.assert_declared_ids_agree(bundle)
|
||||
return okf.directory_listing(bundle, path)
|
||||
return okf.directory_listing(bundle, path, dimension=dimension)
|
||||
|
||||
@tool(
|
||||
name="read_file",
|
||||
|
|
@ -979,7 +1012,30 @@ def navigator_tools(bundle_dirs: Sequence[str]) -> list[FunctionTool]:
|
|||
# safe_resolve is the ONE in-/out-of-bundle test in this repo, and it is fail-closed. A
|
||||
# model-chosen path is untrusted input by definition, so it goes through the same gate the
|
||||
# navigation walk uses rather than a second, laxer check.
|
||||
return Path(safe_resolve(bundle_dir, path)).read_text(encoding="utf-8")
|
||||
resolved = Path(safe_resolve(bundle_dir, path))
|
||||
if dimension is not None:
|
||||
# The SECOND half of the scope (§4.1a). A listing that hides a document while this rung
|
||||
# still serves it by path is a filter in name only, and the caller here is a model that
|
||||
# can name a path no listing gave it. Only a NAVIGATED concept file is judged: the walk
|
||||
# is what knows a file's declared dimension, and a path outside it is already refused —
|
||||
# or, for ``index.md``, is navigation rather than scoped knowledge.
|
||||
bundle = okf.navigate_bundle(bundle_dir)
|
||||
foreign = next(
|
||||
(
|
||||
f
|
||||
for f in bundle.context_files
|
||||
if Path(safe_resolve(bundle_dir, f.name)) == resolved
|
||||
and not okf.in_dimension(f, dimension)
|
||||
),
|
||||
None,
|
||||
)
|
||||
if foreign is not None:
|
||||
raise DimensionScopeRefused(
|
||||
f"document {path!r} in knowledge base {bundle_id!r} declares dimension "
|
||||
f"{foreign.frontmatter.get('dimension')!r}; this run is scoped to "
|
||||
f"{dimension!r} and reads only knowledge in scope"
|
||||
)
|
||||
return resolved.read_text(encoding="utf-8")
|
||||
|
||||
return [list_bundles, read_bundle, read_dir, read_file]
|
||||
|
||||
|
|
|
|||
|
|
@ -1015,6 +1015,25 @@ def assert_declared_ids_agree(bundle: Bundle) -> None:
|
|||
)
|
||||
|
||||
|
||||
def in_dimension(file: BundleFile, dimension: str | None) -> bool:
|
||||
"""Whether a concept file belongs to a run scoped to ``dimension`` (§4.1a).
|
||||
|
||||
ONE copy of the rule (kø-(p)), because two renderings now answer it: ``bundle_context`` renders
|
||||
the matched bodies, and ``directory_listing`` lists the matched documents for a navigator that
|
||||
reads them one at a time. Two copies of "which knowledge is in scope" would be free to disagree
|
||||
about the same base, and the disagreement would show up as an agent being shown a document it
|
||||
is then refused.
|
||||
|
||||
``dimension=None`` admits everything (byte-identical to the unscoped rendering), and a file
|
||||
carrying NO ``dimension`` is never dropped: un-scoped knowledge belongs to every scope, which
|
||||
is what makes a method or a cost reference usable across dimensions.
|
||||
"""
|
||||
if dimension is None:
|
||||
return True
|
||||
declared = file.frontmatter.get("dimension")
|
||||
return declared is None or declared == dimension
|
||||
|
||||
|
||||
class BundlePathNotFound(ValueError):
|
||||
"""A listing was asked for a directory the navigated bundle does not have.
|
||||
|
||||
|
|
@ -1025,7 +1044,9 @@ class BundlePathNotFound(ValueError):
|
|||
"""
|
||||
|
||||
|
||||
def directory_listing(bundle: Bundle, path: str = "") -> dict[str, Any]:
|
||||
def directory_listing(
|
||||
bundle: Bundle, path: str = "", *, dimension: str | None = None
|
||||
) -> dict[str, Any]:
|
||||
"""One LEVEL of a navigated bundle: the subdirectories under ``path`` with what each holds, and
|
||||
the concept documents that sit directly in it.
|
||||
|
||||
|
|
@ -1049,6 +1070,12 @@ def directory_listing(bundle: Bundle, path: str = "") -> dict[str, Any]:
|
|||
relative name would have to be composed by the caller, and the caller is a model: a path that
|
||||
never existed is worse than no path (``_index_excerpt``'s rule, one rung up).
|
||||
|
||||
``dimension`` scopes the listing exactly as it scopes ``bundle_context`` — ONE predicate
|
||||
(``in_dimension``) serves both, so a navigator is never shown a document the run would then
|
||||
refuse to open. Under a scope a directory holding only foreign-dimension documents raises
|
||||
``BundlePathNotFound`` like any other unknown path: within this run it holds nothing, and
|
||||
answering with an empty listing is the very confusion the refusal exists to prevent.
|
||||
|
||||
``documents`` on a directory entry is the count of concept documents in its whole SUBTREE — what
|
||||
the subtree holds, not what one ``read_dir`` on it returns. It is the price signal a navigator
|
||||
chooses against, and the tool description says which of the two it is rather than leaving the
|
||||
|
|
@ -1060,7 +1087,7 @@ def directory_listing(bundle: Bundle, path: str = "") -> dict[str, Any]:
|
|||
directories: dict[str, int] = {}
|
||||
documents: list[dict[str, Any]] = []
|
||||
for f in bundle.context_files:
|
||||
if not f.name.startswith(prefix):
|
||||
if not f.name.startswith(prefix) or not in_dimension(f, dimension):
|
||||
continue
|
||||
rest = f.name[len(prefix) :]
|
||||
head, sep, _ = rest.partition("/")
|
||||
|
|
@ -1113,10 +1140,8 @@ def bundle_context(bundle: Bundle, *, dimension: str | None = None) -> str:
|
|||
against — see ``tests/test_okf.py`` nav-golden gates."""
|
||||
sections = [bundle.index_summary.strip("\n")]
|
||||
for f in bundle.context_files:
|
||||
if dimension is not None:
|
||||
file_dim = f.frontmatter.get("dimension")
|
||||
if file_dim is not None and file_dim != dimension:
|
||||
continue
|
||||
if not in_dimension(f, dimension):
|
||||
continue
|
||||
title = unquote_scalar(f.frontmatter.get("title", f.name))
|
||||
body = f.body.strip("\n")
|
||||
sections.append(f"## {f.type or 'document'}: {title}\n\n{body}")
|
||||
|
|
|
|||
|
|
@ -189,6 +189,36 @@ def write_exploration(
|
|||
return path
|
||||
|
||||
|
||||
def write_debate_tools(
|
||||
outbox_dir: str,
|
||||
run_id: str,
|
||||
*,
|
||||
tool_calls: Sequence[Mapping[str, Any]],
|
||||
) -> Path:
|
||||
"""Write ``{run_id}-debate.json`` — WHICH documents the debate opened, in call order (S2c).
|
||||
|
||||
The sibling of ``write_exploration`` one phase over. Since S2c the debate navigates the
|
||||
knowledge base instead of being handed it whole, so "what did this run actually read" is a
|
||||
question about the debate too, and over a 630-concept corpus it is not answerable from a
|
||||
prompt log: the whole point is that the prompts no longer carry the base.
|
||||
|
||||
Written on EVERY run that has an outbox, including one whose agents opened nothing — unlike
|
||||
``write_parse_failures``, whose presence IS its signal. Here the empty case is the S2c
|
||||
regression itself (a debate that navigates nothing looks exactly like a cheap one), so it must
|
||||
be readable off the artefact rather than inferred from a file that is not there.
|
||||
|
||||
Takes already-rendered plain mappings (``explore.tool_call_payload``) so the RAW output layer
|
||||
stays MAF-free — ``write_exploration``'s own rule, same reason."""
|
||||
directory = Path(outbox_dir)
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
path = directory / f"{run_id}-debate.json"
|
||||
path.write_text(
|
||||
_dump({"run_id": run_id, "tool_calls": [dict(call) for call in tool_calls]}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
return path
|
||||
|
||||
|
||||
def write_plan_review(
|
||||
outbox_dir: str,
|
||||
run_id: str,
|
||||
|
|
|
|||
|
|
@ -60,18 +60,22 @@ from portfolio_optimiser.explore import (
|
|||
NAVIGATOR_ROLE,
|
||||
ExplorationContract,
|
||||
ExplorationResult,
|
||||
ExplorationToolRecorder,
|
||||
ExplorationTrace,
|
||||
ParkedStateError,
|
||||
PlanReviewDecision,
|
||||
PlanReviewParked,
|
||||
ToolCall,
|
||||
explore,
|
||||
exploration_notice,
|
||||
load_exploration_contract,
|
||||
load_parked,
|
||||
parked_notice,
|
||||
parked_payload,
|
||||
navigator_tools,
|
||||
resume_exploration,
|
||||
terminal_plan_reviewer,
|
||||
tool_call_payload,
|
||||
trace_payload,
|
||||
)
|
||||
from portfolio_optimiser.generate import ParseFailure, generate_via_llm
|
||||
|
|
@ -199,6 +203,16 @@ class RunResult:
|
|||
#: ever report zero — unlike ``cost_baseline_anchored`` and ``skipped_links``, both resolved
|
||||
#: above that cut.
|
||||
unkeyed_verdicts: int = 0
|
||||
#: Which documents the DEBATE opened, in call order (S2c). Since the debate navigates the base
|
||||
#: instead of being handed ``bundle_context``, "what did this run read" is no longer answerable
|
||||
#: from the prompts — that is the whole saving — so the run carries the trace itself.
|
||||
#:
|
||||
#: EMPTY is an honest POSITIVE statement ("the debate opened nothing"), which is why it
|
||||
#: defaults, exactly as ``skipped_links`` does; and it is also the S2c regression signal, which
|
||||
#: is why the outbox artefact is written even when it is empty rather than only on activity.
|
||||
#: The RESULT of each call is deliberately absent — that is the base's content, i.e. the very
|
||||
#: thing too big to ride along (``ToolCall``'s own rule, MAJOR-1).
|
||||
debate_tool_calls: tuple[ToolCall, ...] = ()
|
||||
|
||||
@property
|
||||
def verdict_key(self) -> str:
|
||||
|
|
@ -500,6 +514,38 @@ def _authored_texts(result: Any, name: str) -> list[str]:
|
|||
return texts
|
||||
|
||||
|
||||
#: The pointer's shape is fixed text plus the base id, its document count and (when scoped) the
|
||||
#: dimension — O(1) in the corpus by construction. The ceiling that guards it lives in the TEST
|
||||
#: (``test_debate_navigation_cost_loadbearing``), for ``_CATALOGUE_EXCERPT_CHARS``' reason: a bound
|
||||
#: imported from the implementation moves with it, and widening this is the regression the gate
|
||||
#: exists to catch.
|
||||
def _bundle_pointer(bundle: okf.Bundle, bundle_id: str, *, dimension: str | None = None) -> str:
|
||||
"""What the debate is told about the knowledge base INSTEAD of being given it (S2c).
|
||||
|
||||
It must do exactly two things: NAME the base by the id the tools take — a bounded prompt that
|
||||
omits it is a debate that cannot make a single call, which is this seam's vacuous form — and
|
||||
say the ladder exists. It carries no content: the whole point is that the corpus is read on
|
||||
demand and each result rides only from the call that asked for it.
|
||||
|
||||
The document COUNT is the price signal a proposer chooses against (``directory_listing``'s own
|
||||
``chars`` rule, one rung up), and it is the count IN SCOPE: under a dimension, advertising
|
||||
documents the tools will then refuse would be a number that describes a different run.
|
||||
"""
|
||||
documents = sum(1 for f in bundle.context_files if okf.in_dimension(f, dimension))
|
||||
scope = (
|
||||
f" Scope: dimension {dimension!r} — only knowledge in that scope is readable."
|
||||
if dimension
|
||||
else ""
|
||||
)
|
||||
return (
|
||||
f"Knowledge base: {bundle_id} ({documents} concept documents)."
|
||||
f"{scope}\n"
|
||||
"It is NOT included here — read it with your tools: list_bundles() for the bases, "
|
||||
f"read_bundle({bundle_id!r}) for its top level, read_dir({bundle_id!r}, path) for one "
|
||||
f"directory, read_file({bundle_id!r}, path) for one document. Open what you need."
|
||||
)
|
||||
|
||||
|
||||
def _debate_text(result: Any) -> str:
|
||||
"""The PROPOSER's converged output (fed into generation, F1). With ``output_from=agents`` both
|
||||
participants surface, so we select proposer-authored outputs specifically — taking the last of
|
||||
|
|
@ -844,8 +890,10 @@ async def run_project(
|
|||
# 2-3. Project + agent read-context + first-class citations. A bundle run derives ALL THREE from
|
||||
# the navigated OKF bundle via progressive disclosure (verdict layer EXCLUDED — målbilde §2/§4),
|
||||
# NOT keyword chunk-stuffing; the road path keeps the chunk-retrieval data source. ``debate_tools``
|
||||
# is the query-time retrieval surface — empty on the bundle path (navigation already placed the
|
||||
# curated context in the prompt, and a docs_dir==bundle_dir tool would re-leak the verdict layer).
|
||||
# is the query-time retrieval surface: since S2c the bundle path carries the four NAVIGATOR
|
||||
# tools there (progressive disclosure taken to its conclusion — the agents open what they need
|
||||
# instead of being handed the base), and a ``docs_dir==bundle_dir`` chunk tool is still refused,
|
||||
# because that one would re-leak the verdict layer the navigation excludes by construction.
|
||||
# S4.0 (F3): the run path SETS the validator's cost baseline, so the deterministic gate is
|
||||
# anchored to the project's real cost lines instead of the ones the proposal asserts.
|
||||
# * road path: the reference project's own ``cost_items`` ARE the baseline -> always anchored.
|
||||
|
|
@ -858,7 +906,8 @@ async def run_project(
|
|||
# the mount is carried alongside, so a base delivered under a directory name of its own is
|
||||
# opened rather than refused. What is still refused, before a single model call: a base
|
||||
# whose concepts declare two different corpora.
|
||||
resolved_bundle_id: okf.ResolvedBundleId | None = okf.reconcile_bundle_id(bundle_dir)
|
||||
resolved = okf.reconcile_bundle_id(bundle_dir)
|
||||
resolved_bundle_id: okf.ResolvedBundleId | None = resolved
|
||||
okf.assert_declared_ids_agree(bundle)
|
||||
project = _project_from_bundle(bundle_dir, project_id, bundle=bundle)
|
||||
# The THIRD projection into ``CostBaseline`` (MAJOR-4), behind an EXPLICIT commission and
|
||||
|
|
@ -872,14 +921,22 @@ async def run_project(
|
|||
if derive_cost_baseline
|
||||
else okf.load_optional_cost_baseline(bundle_dir)
|
||||
)
|
||||
# §4.1a context-scope: agents read ONLY dimension-scoped bundle knowledge (Step-3 filter);
|
||||
# dimension=None keeps the full context, byte-identical to before.
|
||||
context = okf.bundle_context(bundle, dimension=dimension.id if dimension else None)
|
||||
# S2c: the debate NAVIGATES the base; it is never handed the whole of it. Measured on K2
|
||||
# (630 concepts, docs/2026-09-04-syretest-s7b-k2.md § 3.4) the rendered context was 648 962
|
||||
# o200k tokens riding in THREE prompts — 99,1 % of a run's prompt cost, none of it asked
|
||||
# for twice. The task message now carries a POINTER, and the agents get the SAME four
|
||||
# tools the exploration uses; ``gen_context = debate_output or context`` below means the
|
||||
# generation fallback is bounded by the same change rather than by a second policy.
|
||||
dimension_id = dimension.id if dimension else None
|
||||
context = _bundle_pointer(bundle, resolved.id, dimension=dimension_id)
|
||||
citations = bundle_citations(bundle)
|
||||
# What the navigation could NOT reach, taken from the run's ONE walk. The road path below
|
||||
# navigates no bundle at all, so its empty tuple is literally true rather than a stand-in.
|
||||
skipped_links: tuple[okf.SkippedLink, ...] = bundle.skipped
|
||||
debate_tools: list[Any] = []
|
||||
# §4.1a context-scope, carried over: the agents read ONLY dimension-matched knowledge. The
|
||||
# filter used to live in the rendering; with navigation it lives in the TOOLS, on both
|
||||
# rungs (``navigator_tools``' own gate), because that is now where the bytes leave.
|
||||
debate_tools: list[Any] = list(navigator_tools([bundle_dir], dimension=dimension_id))
|
||||
else:
|
||||
project = _project_by_id(project_id)
|
||||
baseline = baseline_from_project(project)
|
||||
|
|
@ -915,12 +972,23 @@ async def run_project(
|
|||
# called. Attached only when servers are configured — with none there is nothing to attribute a
|
||||
# call to, and the middleware list stays exactly what it was before Trekk B.
|
||||
call_recorder = ToolCallRecorder(tool_server_index(mcp_servers)) if mcp_servers else None
|
||||
# S2c: a CALLER-OWNED sink for what the debate opens (the ``parse_failures``/``ExplorationTrace``
|
||||
# shape). A returned value would be lost on exactly the run that most needs the evidence — a
|
||||
# budget stop mid-debate raises out of ``debate.run`` and constructs no ``RunResult`` at all.
|
||||
# ``ExplorationToolRecorder`` is REUSED rather than re-implemented: it is already the recorder
|
||||
# for in-process navigator calls, ordered and un-deduplicated, which is exactly the question
|
||||
# here too ("did this run open anything, and in what sequence"). Its sibling
|
||||
# ``mcp_tools.ToolCallRecorder`` stays what it is — a sorted, de-duplicated EGRESS claim.
|
||||
debate_tool_calls: list[ToolCall] = []
|
||||
debate_middleware: list[Any] = [budget_mw, ExplorationToolRecorder(debate_tool_calls)]
|
||||
if call_recorder is not None:
|
||||
debate_middleware.append(call_recorder)
|
||||
debate = fresh_workflow(
|
||||
factory,
|
||||
max_rounds=max_rounds,
|
||||
enable_layer1_hitl=enable_layer1_hitl,
|
||||
tools=debate_tools,
|
||||
middleware=[budget_mw] if call_recorder is None else [budget_mw, call_recorder],
|
||||
middleware=debate_middleware,
|
||||
)
|
||||
# S4.2 cut (comparison protocol §4 pkt 2/3): everything above is offline — contracts, budget, and
|
||||
# the EAGER client build (fresh_workflow constructs the proposer+checker clients, workflow.py:64).
|
||||
|
|
@ -957,12 +1025,23 @@ async def run_project(
|
|||
# opens a connection — its promise to stop before the first call covers egress too. Constructed
|
||||
# tools that are never entered expose nothing, and ones never exited leave the process hanging,
|
||||
# so the stack owns both halves.
|
||||
async with AsyncExitStack() as mcp_stack:
|
||||
for live_tool in live_mcp_tools:
|
||||
await mcp_stack.enter_async_context(live_tool)
|
||||
result = await debate.run(
|
||||
f"Find a cost-saving measure for {project.id}.\nContext:\n{context}"
|
||||
)
|
||||
try:
|
||||
async with AsyncExitStack() as mcp_stack:
|
||||
for live_tool in live_mcp_tools:
|
||||
await mcp_stack.enter_async_context(live_tool)
|
||||
result = await debate.run(
|
||||
f"Find a cost-saving measure for {project.id}.\nContext:\n{context}"
|
||||
)
|
||||
finally:
|
||||
# ``finally``, the ``write_parse_failures`` precedent: any exception leaving the debate —
|
||||
# a budget stop is today's known one — destroys the same evidence, and a list of exception
|
||||
# types is a list that goes stale. Written even when EMPTY: "the debate opened nothing" is
|
||||
# the S2c regression itself, so it must be readable rather than inferred from an absence.
|
||||
if outbox_dir is not None:
|
||||
assert run_id is not None # narrowed by the step-0 guard (no wall-clock default)
|
||||
outbox.write_debate_tools(
|
||||
outbox_dir, run_id, tool_calls=tool_call_payload(debate_tool_calls)
|
||||
)
|
||||
# F1: the candidate must derive from the DEBATE. Feed the proposer's converged output into
|
||||
# generation (retrieval context is the last-resort fallback only). The checker's verdict
|
||||
# (Step 3/4) is parsed from the SAME debate result and gates the outcome below.
|
||||
|
|
@ -1219,6 +1298,7 @@ async def run_project(
|
|||
refinements=tuple(refinements),
|
||||
skipped_links=skipped_links,
|
||||
unkeyed_verdicts=unkeyed_verdicts,
|
||||
debate_tool_calls=tuple(debate_tool_calls),
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -1940,12 +2020,25 @@ def _load_scripted_replies(
|
|||
``scripted_factory``'s lookup, mid-run, long after the run appeared to start cleanly (MAJOR-2:
|
||||
measured for the three ``explore()`` adds on top of the debate's own two).
|
||||
|
||||
**An EXPLORATION role may also be given a step LIST** (MAJOR-1 b), because a single constant
|
||||
string can never emit a ``function_call`` — measured: 0 tool calls / 0 approaches / 1 round on
|
||||
4/4 bases, an offline rehearsal that was vacuous by construction. The list form is refused for
|
||||
the debate's roles BY NAME rather than accepted and ignored: the proposer answers
|
||||
``generate``'s own call, not an agent loop that would invoke a tool between turns, so a script
|
||||
of calls there describes a rehearsal that cannot happen."""
|
||||
**ANY role may be given a step LIST** (MAJOR-1 b), because a single constant string can never
|
||||
emit a ``function_call`` — measured: 0 tool calls / 0 approaches / 1 round on 4/4 bases, an
|
||||
offline rehearsal that was vacuous by construction.
|
||||
|
||||
Until S2c the list form was REFUSED for the debate's two roles, on the stated ground that "the
|
||||
proposer answers ``generate``'s own call, not an agent loop that would invoke a tool between
|
||||
turns, so a script of calls there describes a rehearsal that cannot happen". That ground is now
|
||||
measurably false: the debate's proposer and checker are ``Agent``s in a GroupChat and they hold
|
||||
the four navigator tools, so a constant-string rehearsal proves the debate RUNS while proving
|
||||
nothing about whether it OPENS the base — which is the identical vacuity MAJOR-1 closed one
|
||||
surface over. Keeping the refusal would have made the free half of the measurement ladder
|
||||
unable to reach the very seam S2c builds.
|
||||
|
||||
**Honesty limit, stated rather than encoded.** Scripts are per-CLIENT and each client gets its
|
||||
own copy, so a ``proposer`` script is consumed once by the DEBATE client and again, from the
|
||||
start, by the fresh client ``generate_via_llm`` builds. A script whose first step is a tool
|
||||
call therefore answers the generation call with a ``function_call`` too, which will not parse.
|
||||
That is the operator's to write correctly: guessing which steps were "meant for" which call
|
||||
site would be repair, and this loader validates."""
|
||||
try:
|
||||
raw = json.loads(Path(path).read_text(encoding="utf-8"))
|
||||
except FileNotFoundError as exc:
|
||||
|
|
@ -1963,12 +2056,6 @@ def _load_scripted_replies(
|
|||
for role in required_roles:
|
||||
if isinstance(raw[role], str):
|
||||
continue
|
||||
if role not in _EXPLORATION_SCRIPTED_ROLES:
|
||||
raise ValueError(
|
||||
f"--scripted-replies[{role!r}] is a step list, but that form is the exploration's: "
|
||||
f"only {', '.join(_EXPLORATION_SCRIPTED_ROLES)} run inside an agent loop that can "
|
||||
f"invoke a tool between turns ({path})"
|
||||
)
|
||||
_validate_script(role, raw[role], path)
|
||||
return {role: raw[role] for role in required_roles}
|
||||
|
||||
|
|
|
|||
|
|
@ -175,7 +175,8 @@ def scripted_proposer(
|
|||
|
||||
**Why the project id and not the cost code or the measure name** (measured, not assumed — the
|
||||
demo-week plan §6 flagged this as unverified): two prompt shapes reach this selector. The debate
|
||||
prompt (``run.py``) carries the whole bundle context; the generation prompt
|
||||
prompt (``run.py``) carries the task line plus a POINTER to the knowledge base — since S2c it no
|
||||
longer carries the base itself, which only sharpens the argument; the generation prompt
|
||||
(``generate._build_messages``) carries ``Project: {id} - {name}`` plus, as its context, the
|
||||
*debate output* — which is this selector's own earlier reply. So the cost code and measure name
|
||||
are present in the generation prompt only because the script put them there; keying on them
|
||||
|
|
@ -186,8 +187,8 @@ def scripted_proposer(
|
|||
|
||||
Anything other than exactly one match raises ``ScriptedCandidateError``. Validation, never
|
||||
repair: a default reply would let an unregistered project be answered with another project's
|
||||
numbers, which on screen is indistinguishable from a correct run. An ambiguous blob (a bundle
|
||||
context that names a sibling project) is a DATA problem, and it must surface at the rehearsal
|
||||
numbers, which on screen is indistinguishable from a correct run. An ambiguous blob (a tool
|
||||
result that names a sibling project) is a DATA problem, and it must surface at the rehearsal
|
||||
rather than be silently decided by registry order.
|
||||
"""
|
||||
|
||||
|
|
@ -806,9 +807,12 @@ async def simulate_exploration(
|
|||
shutil.copytree(bundle_dir, copy)
|
||||
copy_s = str(copy)
|
||||
|
||||
# The vacuity guard. A label the base already states would reach the hypothesis prompt as
|
||||
# navigated context whether or not the exploration ran, so the scenario's own assertion would
|
||||
# hold against an implementation that never wired the mandate at all.
|
||||
# The vacuity guard. A label the base already states is a label the walkthrough cannot
|
||||
# attribute to the exploration: since S2c the debate reads the base through its own tools, so
|
||||
# such a label could reach the hypothesis prompt as an ordinary tool result whether or not the
|
||||
# exploration ran, and the scenario's assertion would hold against an implementation that never
|
||||
# wired the mandate at all. ``bundle_context`` is used here as "everything the base states",
|
||||
# which is what it still renders — not as a claim about what any prompt carries.
|
||||
context = okf.bundle_context(okf.navigate_bundle(copy_s))
|
||||
if label in context:
|
||||
raise ValueError(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue