open/portfolio-optimiser
1
0
Fork 0
Code Issues Releases 3 Activity

feat(run,generate): a run says what its delivered input can ground, before it spends an attempt [skip-docs]

Browse source 
P7 is right and landed, but re-measuring it exposed a consequence no row stated: with the gate
live, 29 of 29 cost codes in 13 of 13 delivered proposals fall across the three free recordings
(PM's denominator; 14 of 14 in 8 proposals on the PARSEABLE one -- the five blobs that separate
the numbers are refused by pydantic's `claimed <= total` and never reach stage 0b). All 29 were
invented, so the gate is right; but a gate that always refuses is as useless as one that never
does.

The cause is that the PROMPT asks for something the input cannot supply. `_build_messages`
requires each affected_item to "restate a cost line as the project's price schedule already
carries it", while K2's delivered input carries 9 occurrences / 2 distinct code-shaped tokens --
`SHA-01`/`SHA-10`, both document numbers off a page footer -- and `derive_cost_baseline` refuses
the base outright. There is no cost line in it to restate.

`GroundingOffer(chars, identifiers, cost_lines)` reports it. The PAIR is the diagnosis: "50
identifiers, 0 cost lines" says what neither number says alone. A REPORT, never a gate -- it
blocks nothing, because a blocking requirement IS `--require-cost-baseline` (F4/D-3, opt-in,
untouched), and `_ground_against_input` is untouched.

The callsite is MEASURED, not chosen: `generate.py` composes the grounding per attempt, after
`await _fetch_parsed`, so a report there could only speak once an attempt had been paid for;
`run.py` binds both halves above the `--live-dry-run` cut and before the first `debate.run`, so
the FREE trip says it. `delivered` is bound ONCE and the same variable feeds the report and
`_evaluate`; the report composes THROUGH `_grounding_text`, the gate's own composer.

A pattern is admissible here and not in the gate, and that is the difference between a report and
a falsifier: an unknown form is a token left uncounted -- an under-count, never a false rejection.
The forms are transcribed from the measurement; bare numbers are excluded with the number
(46 394 / 2 117 in K2). `grounding_offer_notice` is the ONE renderer and is silent when the run
CAN anchor -- omission, never an empty row.

Load-bearing MEASURED (tests/test_grounding_offer_loadbearing.py, 12 arms), nine mutations all
red against the WHOLE suite + green control 1570/5 (from 1558/5, strict superset, 0 removed) and
the golden byte-unchanged (shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f).

Measurement: docs/2026-09-09-p8-forankringstilbudet.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 17:50:02 +02:00
commit 455d611660
5 changed files with 767 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

74
src/portfolio_optimiser/generate.py
View file

@ -22,6 +22,7 @@ Two entry points, because the LLM call is async while ``validator.self_repair``
from __future__ import annotations
import json
import re
from collections.abc import Callable, Mapping
from dataclasses import dataclass, field, replace
from typing import Any
@ -456,6 +457,79 @@ def _grounding_text(project: Project, baseline: CostBaseline | None, delivered:
)
#: The identifier forms the DELIVERED corpora actually carry, TRANSCRIBED from the measurement
#: (``docs/2026-09-09-p8-forankringstilbudet.md`` § 2) rather than chosen: over K2's two delivered
#: bundles the first form finds 50 distinct tokens and the second none, while over the three
#: N payloads the second finds 269-981 distinct and the first at most 3.
#:
#: **Why a pattern is admissible HERE and not in ``_ground_against_input``.** That is a GATE, and a
#: pattern there would be a rule about shapes that can wrongly REFUSE a real code. This is a
#: REPORT: a form it does not know is a token it fails to count, so it errs toward saying the input
#: offers LESS than it does — an under-count is a quiet report, never a false rejection.
#:
#: Bare numbers are deliberately EXCLUDED, with the number: K2 carries 46 394 occurrences over
#: 2 117 distinct values (P7 § 2), so counting them would make every report positive and the
#: measurement inert — the repo's cardinal class, a gate that can only come out green.
_IDENTIFIER_FORMS = (
# ``SHA-01``, ``RIM-02``, ``B-20-00-00``, ``FOR-2011-12-06-1357`` (K2's 50).
re.compile(r"\b[A-ZÆØÅ]{1,8}[-_]\d{1,4}(?:[-_]\d{1,4})*\b"),
# ``Krav 3.3.1—13`` (the N corpora's dominant form). EM-DASH U+2014 AND the hyphen, because the
# binding known positive is the em-dash spelling and only the em-dash spelling scores 6 of 6.
re.compile(r"Krav\s+\d+(?:\.\d+)*\s*[\u2014-]\s*\d+"),
)
@dataclass(frozen=True)
class GroundingOffer:
"""P8: what the DELIVERED input of one run can lawfully ground an ``affected_item`` code in.
A REPORT, never a gate. It blocks nothing — a run whose offer is null still runs — because a
blocking requirement is exactly ``--require-cost-baseline``, which F4 settled as opt-in.
The PAIR is the whole diagnosis, and neither number says it alone. MEASURED on K2: the
delivered text offers 50 citable identifiers and ZERO cost lines, while the proposer prompt
asks each ``affected_item`` to "restate a cost line as the project's price schedule already
carries it". An operator reading "identifiers: 50, cost lines: 0" learns that no attempt could
have succeeded; reading either number by itself, they learn nothing of the sort.
``chars`` is the size of the exact text P7's gate will measure against — carried so the report
and the gate can be seen to be talking about the same input, which is the only defence against
a second rendering free to disagree with the one that was sent.
"""
#: Length of ``_grounding_text``'s output — the text the gate itself will search.
chars: int
#: Distinct tokens of any ``_IDENTIFIER_FORMS`` shape the text carries.
identifiers: int
#: Cost lines this run can anchor one of them AS. 0 when the run is un-anchored, which is the
#: state every free K2 arm measured. Read off the SAME ``baseline`` binding ``_grounding_text``
#: takes as its third source, so the count and the anchoring can never disagree.
cost_lines: int
def grounding_offer(
project: Project, baseline: CostBaseline | None, delivered: str
) -> GroundingOffer:
"""Measure what ``delivered`` can ground, on the EXACT text the gate will see.
Composed THROUGH ``_grounding_text`` — the one composer ``generate_via_llm`` passes to
``validate_proposal`` — never re-assembled here. A second composition would be free to drift
from the one that was actually sent, and a report about a text nobody was given is worse than
no report: it reads as evidence.
Deterministic and free: no model call, no network, and no second walk of the bundle.
"""
text = _grounding_text(project, baseline, delivered)
found: set[str] = set()
for form in _IDENTIFIER_FORMS:
found |= set(form.findall(text))
return GroundingOffer(
chars=len(text),
identifiers=len(found),
cost_lines=0 if baseline is None else len(baseline.items),
)
async def generate_via_llm(
chat_client: BaseChatClient,
project: Project,

83
src/portfolio_optimiser/run.py
View file

@ -80,7 +80,12 @@ from portfolio_optimiser.explore import (
tool_call_payload,
trace_payload,
)
from portfolio_optimiser.generate import ParseFailure, generate_via_llm
from portfolio_optimiser.generate import (
GroundingOffer,
ParseFailure,
generate_via_llm,
grounding_offer,
)
from portfolio_optimiser.ir import SavingsProposal
from portfolio_optimiser.mandate import (
OWN_PROPOSAL_ID,
@ -248,6 +253,15 @@ class RunResult:
#: Carried HERE and not on ``ProvenanceStamp``: the stamp describes the gate that judged ONE
#: candidate, while this is a RUN-level fact about what the run was allowed to read at all.
prepass: prepass.PrepassDeclaration | None = None
#: What this run's DELIVERED input could ground an ``affected_item`` code in (P8): the size of
#: the text P7's gate searched, how many distinct identifiers of a measured form it carries,
#: and how many cost lines the run can anchor one of them AS.
#:
#: A RUN-level fact settled ONCE, before any candidate exists, which is why it is here and not
#: on ``ProvenanceStamp`` (that describes the gate that judged ONE candidate) — ``skipped_links``'
#: own placement rule. DEFAULTED, ``prepass``' half of the rule: ``None`` is the true statement
#: "no measurement was made", and there is exactly one place that makes it.
grounding_offer: GroundingOffer | None = None
@property
def verdict_key(self) -> str:
@ -313,6 +327,12 @@ class DryRunReport:
#: resolves a payload, so a dry run can honestly report what it would have read — whereas a
#: field resolved above that cut could only ever report zero.
prepass: prepass.PrepassDeclaration | None = None
#: What the delivered input of a REAL run of this configuration could ground an
#: ``affected_item`` code in (P8). Carried here for ``cost_baseline_anchored``'s reason and,
#: more sharply, because this is the surface on which "before it spends its three attempts"
#: is provable at all: the dry-run cut returns before the first model call, so a dry run that
#: reports a null offer has said the run cannot succeed WITHOUT paying to find out.
grounding_offer: GroundingOffer | None = None
@dataclass(frozen=True)
@ -781,6 +801,37 @@ def cost_baseline_notice(anchored: bool) -> str | None:
return None if anchored else _UNANCHORED_NOTICE
def grounding_offer_notice(offer: GroundingOffer | None) -> str | None:
"""Render the one line that says what this run's delivered input can ground, or ``None`` when
there is nothing to warn about (P8).
ONE renderer with N callsites, never N copies of the wording (kø-(p)), taking the
ALREADY-MEASURED value rather than a text: a renderer that re-composed the grounding would be
a second resolution of the same rule, free to drift from the run it describes
(``cost_baseline_notice``'s rule).
``None`` when the run CAN anchor a cost line — omission, never an empty row
(``mandate.announce``'s rule) — and ``None`` on ``None``, which is the honest reading of "no
measurement was made". This is deliberately NOT ``proposal_review_notice``'s deviation: there,
silence on zero was ambiguous; here, a run that can anchor its lines has nothing to report that
the outcome does not already say.
BOTH numbers, because the pair is the diagnosis. "0 cost lines" alone reads as a restatement of
``cost_baseline_notice``; "50 identifiers" alone reads as good news. Together they say the
thing P8 measured: the input offers plenty to cite and nothing to cost, while the proposer
prompt asks for a cost line.
English, like every other line this CLI prints."""
if offer is None or offer.cost_lines > 0:
return None
return (
f" Grounding offer: {offer.identifiers} distinct identifier(s) and "
f"{offer.cost_lines} cost line(s) in the {offer.chars} characters this run was given — "
"the proposer is asked to restate a cost line this input does not carry, so every "
"candidate it invents will be refused as ungrounded"
)
def bundle_id_notice(resolved: okf.ResolvedBundleId | None) -> str | None:
"""Render the one line that says a base was mounted under a name it does not answer to, or
``None`` when there is nothing to say.
@ -1166,6 +1217,17 @@ async def run_project(
"--derive-cost-baseline when the base carries a priced schedule"
)
# P8: what this run was GIVEN, composed ONCE. Bound HERE and not inside ``_evaluate`` below,
# and that placement is the measurement this seam rests on: this is the first point at which
# both halves exist AND it is above the ``--live-dry-run`` cut, so the offer can be reported
# on the FREE trip — before the first model call at ``debate.run``, let alone the three
# generation attempts. ``generate.py``'s own composition happens per attempt, AFTER
# ``_fetch_parsed`` has returned, so a report from there could only ever speak once an attempt
# had been paid for. ONE binding feeding both the report and the gate: two compositions of one
# text are free to disagree, which is exactly what a report must not be able to do (kø-(p)).
delivered = "\n".join([context, bundle_grounding])
offer = grounding_offer(project, baseline, delivered)
# Trekk B2 (krav 3): configured MCP servers become tools the AGENTS can call during the debate.
# Appended to BOTH paths — on the bundle path they are the first tools that path has ever had.
# Constructed here but NOT connected: an ``MCPTool`` is an async context manager, so the run
@ -1239,6 +1301,9 @@ async def run_project(
bundle_id_source=resolved_bundle_id,
skipped_links=skipped_links,
prepass=prepass_declaration,
# P8, and this surface is the point: the offer is measured ABOVE this cut, so a dry
# run reports it having made no model call at all.
grounding_offer=offer,
)
# The MCP lifecycle (Trekk B2): entered HERE, after the dry-run cut above, so a dry run never
# opens a connection — its promise to stop before the first call covers egress too. Constructed
@ -1378,7 +1443,7 @@ async def run_project(
# ``context`` is the DELIVERED rendering (pre-pass cut / bundle pointer / retrieved
# chunks) — never ``gen_context``, which on the debate path is the model's own
# summary and would let a code the debate invented ground the proposal repeating it.
grounding="\n".join([context, bundle_grounding]),
grounding=delivered,
)
refinements.extend(generated.refinements)
return generated.outcome
@ -1587,6 +1652,9 @@ async def run_project(
debate_tool_calls=tuple(debate_tool_calls),
expert_revisions=tuple(expert_reviews),
prepass=prepass_declaration,
# P8: read off the SAME single measurement the gate's own grounding descends from, so the
# record and the refusals cannot describe different inputs.
grounding_offer=offer,
)
@ -3836,6 +3904,12 @@ def main(argv: list[str] | None = None) -> int:
notice = cost_baseline_notice(report.cost_baseline_anchored)
if notice is not None:
print(notice)
# P8, printed next to the line it qualifies: "stage 0 is skipped" says the gate lost a
# falsifier; this says what the input could have offered it instead. On the FREE trip, so
# an operator learns a run cannot be grounded without paying three attempts to find out.
offer_notice = grounding_offer_notice(report.grounding_offer)
if offer_notice is not None:
print(offer_notice)
# The second measured silence on this surface: a bundle with an unfollowable cross-link
# dry-ran to rc 0 with nothing said, so a half-read base looked exactly like a small one.
nav_notice = skipped_links_notice(report.skipped_links)
@ -3925,6 +3999,11 @@ def main(argv: list[str] | None = None) -> int:
notice = cost_baseline_notice(result.provenance.cost_baseline_anchored)
if notice is not None:
print(notice)
# Same renderer on the full run, read off the run's OWN measurement: a run that spent every
# attempt being refused as ungrounded is exactly where the input-side fact costs the most.
offer_notice = grounding_offer_notice(result.grounding_offer)
if offer_notice is not None:
print(offer_notice)
# Same renderer on the full run, and deliberately so: a run that PRODUCED a proposal from a
# half-read base is where the silence cost the most — the dry run at least produced nothing.
nav_notice = skipped_links_notice(result.skipped_links)