feat(run,generate): a run says what its delivered input can ground, before it spends an attempt [skip-docs]
P7 is right and landed, but re-measuring it exposed a consequence no row stated: with the gate live, 29 of 29 cost codes in 13 of 13 delivered proposals fall across the three free recordings (PM's denominator; 14 of 14 in 8 proposals on the PARSEABLE one -- the five blobs that separate the numbers are refused by pydantic's `claimed <= total` and never reach stage 0b). All 29 were invented, so the gate is right; but a gate that always refuses is as useless as one that never does. The cause is that the PROMPT asks for something the input cannot supply. `_build_messages` requires each affected_item to "restate a cost line as the project's price schedule already carries it", while K2's delivered input carries 9 occurrences / 2 distinct code-shaped tokens -- `SHA-01`/`SHA-10`, both document numbers off a page footer -- and `derive_cost_baseline` refuses the base outright. There is no cost line in it to restate. `GroundingOffer(chars, identifiers, cost_lines)` reports it. The PAIR is the diagnosis: "50 identifiers, 0 cost lines" says what neither number says alone. A REPORT, never a gate -- it blocks nothing, because a blocking requirement IS `--require-cost-baseline` (F4/D-3, opt-in, untouched), and `_ground_against_input` is untouched. The callsite is MEASURED, not chosen: `generate.py` composes the grounding per attempt, after `await _fetch_parsed`, so a report there could only speak once an attempt had been paid for; `run.py` binds both halves above the `--live-dry-run` cut and before the first `debate.run`, so the FREE trip says it. `delivered` is bound ONCE and the same variable feeds the report and `_evaluate`; the report composes THROUGH `_grounding_text`, the gate's own composer. A pattern is admissible here and not in the gate, and that is the difference between a report and a falsifier: an unknown form is a token left uncounted -- an under-count, never a false rejection. The forms are transcribed from the measurement; bare numbers are excluded with the number (46 394 / 2 117 in K2). `grounding_offer_notice` is the ONE renderer and is silent when the run CAN anchor -- omission, never an empty row. Load-bearing MEASURED (tests/test_grounding_offer_loadbearing.py, 12 arms), nine mutations all red against the WHOLE suite + green control 1570/5 (from 1558/5, strict superset, 0 removed) and the golden byte-unchanged (shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f). Measurement: docs/2026-09-09-p8-forankringstilbudet.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
277bb95777
commit
455d611660
5 changed files with 767 additions and 2 deletions
|
|
@ -22,6 +22,7 @@ Two entry points, because the LLM call is async while ``validator.self_repair``
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass, field, replace
|
||||
from typing import Any
|
||||
|
|
@ -456,6 +457,79 @@ def _grounding_text(project: Project, baseline: CostBaseline | None, delivered:
|
|||
)
|
||||
|
||||
|
||||
#: The identifier forms the DELIVERED corpora actually carry, TRANSCRIBED from the measurement
|
||||
#: (``docs/2026-09-09-p8-forankringstilbudet.md`` § 2) rather than chosen: over K2's two delivered
|
||||
#: bundles the first form finds 50 distinct tokens and the second none, while over the three
|
||||
#: N payloads the second finds 269-981 distinct and the first at most 3.
|
||||
#:
|
||||
#: **Why a pattern is admissible HERE and not in ``_ground_against_input``.** That is a GATE, and a
|
||||
#: pattern there would be a rule about shapes that can wrongly REFUSE a real code. This is a
|
||||
#: REPORT: a form it does not know is a token it fails to count, so it errs toward saying the input
|
||||
#: offers LESS than it does — an under-count is a quiet report, never a false rejection.
|
||||
#:
|
||||
#: Bare numbers are deliberately EXCLUDED, with the number: K2 carries 46 394 occurrences over
|
||||
#: 2 117 distinct values (P7 § 2), so counting them would make every report positive and the
|
||||
#: measurement inert — the repo's cardinal class, a gate that can only come out green.
|
||||
_IDENTIFIER_FORMS = (
|
||||
# ``SHA-01``, ``RIM-02``, ``B-20-00-00``, ``FOR-2011-12-06-1357`` (K2's 50).
|
||||
re.compile(r"\b[A-ZÆØÅ]{1,8}[-_]\d{1,4}(?:[-_]\d{1,4})*\b"),
|
||||
# ``Krav 3.3.1—13`` (the N corpora's dominant form). EM-DASH U+2014 AND the hyphen, because the
|
||||
# binding known positive is the em-dash spelling and only the em-dash spelling scores 6 of 6.
|
||||
re.compile(r"Krav\s+\d+(?:\.\d+)*\s*[\u2014-]\s*\d+"),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GroundingOffer:
|
||||
"""P8: what the DELIVERED input of one run can lawfully ground an ``affected_item`` code in.
|
||||
|
||||
A REPORT, never a gate. It blocks nothing — a run whose offer is null still runs — because a
|
||||
blocking requirement is exactly ``--require-cost-baseline``, which F4 settled as opt-in.
|
||||
|
||||
The PAIR is the whole diagnosis, and neither number says it alone. MEASURED on K2: the
|
||||
delivered text offers 50 citable identifiers and ZERO cost lines, while the proposer prompt
|
||||
asks each ``affected_item`` to "restate a cost line as the project's price schedule already
|
||||
carries it". An operator reading "identifiers: 50, cost lines: 0" learns that no attempt could
|
||||
have succeeded; reading either number by itself, they learn nothing of the sort.
|
||||
|
||||
``chars`` is the size of the exact text P7's gate will measure against — carried so the report
|
||||
and the gate can be seen to be talking about the same input, which is the only defence against
|
||||
a second rendering free to disagree with the one that was sent.
|
||||
"""
|
||||
|
||||
#: Length of ``_grounding_text``'s output — the text the gate itself will search.
|
||||
chars: int
|
||||
#: Distinct tokens of any ``_IDENTIFIER_FORMS`` shape the text carries.
|
||||
identifiers: int
|
||||
#: Cost lines this run can anchor one of them AS. 0 when the run is un-anchored, which is the
|
||||
#: state every free K2 arm measured. Read off the SAME ``baseline`` binding ``_grounding_text``
|
||||
#: takes as its third source, so the count and the anchoring can never disagree.
|
||||
cost_lines: int
|
||||
|
||||
|
||||
def grounding_offer(
|
||||
project: Project, baseline: CostBaseline | None, delivered: str
|
||||
) -> GroundingOffer:
|
||||
"""Measure what ``delivered`` can ground, on the EXACT text the gate will see.
|
||||
|
||||
Composed THROUGH ``_grounding_text`` — the one composer ``generate_via_llm`` passes to
|
||||
``validate_proposal`` — never re-assembled here. A second composition would be free to drift
|
||||
from the one that was actually sent, and a report about a text nobody was given is worse than
|
||||
no report: it reads as evidence.
|
||||
|
||||
Deterministic and free: no model call, no network, and no second walk of the bundle.
|
||||
"""
|
||||
text = _grounding_text(project, baseline, delivered)
|
||||
found: set[str] = set()
|
||||
for form in _IDENTIFIER_FORMS:
|
||||
found |= set(form.findall(text))
|
||||
return GroundingOffer(
|
||||
chars=len(text),
|
||||
identifiers=len(found),
|
||||
cost_lines=0 if baseline is None else len(baseline.items),
|
||||
)
|
||||
|
||||
|
||||
async def generate_via_llm(
|
||||
chat_client: BaseChatClient,
|
||||
project: Project,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue