feat(run,generate): a run says what its delivered input can ground, before it spends an attempt [skip-docs]
P7 is right and landed, but re-measuring it exposed a consequence no row stated: with the gate live, 29 of 29 cost codes in 13 of 13 delivered proposals fall across the three free recordings (PM's denominator; 14 of 14 in 8 proposals on the PARSEABLE one -- the five blobs that separate the numbers are refused by pydantic's `claimed <= total` and never reach stage 0b). All 29 were invented, so the gate is right; but a gate that always refuses is as useless as one that never does. The cause is that the PROMPT asks for something the input cannot supply. `_build_messages` requires each affected_item to "restate a cost line as the project's price schedule already carries it", while K2's delivered input carries 9 occurrences / 2 distinct code-shaped tokens -- `SHA-01`/`SHA-10`, both document numbers off a page footer -- and `derive_cost_baseline` refuses the base outright. There is no cost line in it to restate. `GroundingOffer(chars, identifiers, cost_lines)` reports it. The PAIR is the diagnosis: "50 identifiers, 0 cost lines" says what neither number says alone. A REPORT, never a gate -- it blocks nothing, because a blocking requirement IS `--require-cost-baseline` (F4/D-3, opt-in, untouched), and `_ground_against_input` is untouched. The callsite is MEASURED, not chosen: `generate.py` composes the grounding per attempt, after `await _fetch_parsed`, so a report there could only speak once an attempt had been paid for; `run.py` binds both halves above the `--live-dry-run` cut and before the first `debate.run`, so the FREE trip says it. `delivered` is bound ONCE and the same variable feeds the report and `_evaluate`; the report composes THROUGH `_grounding_text`, the gate's own composer. A pattern is admissible here and not in the gate, and that is the difference between a report and a falsifier: an unknown form is a token left uncounted -- an under-count, never a false rejection. The forms are transcribed from the measurement; bare numbers are excluded with the number (46 394 / 2 117 in K2). `grounding_offer_notice` is the ONE renderer and is silent when the run CAN anchor -- omission, never an empty row. Load-bearing MEASURED (tests/test_grounding_offer_loadbearing.py, 12 arms), nine mutations all red against the WHOLE suite + green control 1570/5 (from 1558/5, strict superset, 0 removed) and the golden byte-unchanged (shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f). Measurement: docs/2026-09-09-p8-forankringstilbudet.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
277bb95777
commit
455d611660
5 changed files with 767 additions and 2 deletions
43
CLAUDE.md
43
CLAUDE.md
|
|
@ -2316,6 +2316,49 @@ Python ≥3.10. MAF (`agent-framework-core` 1.16.0, `-orchestrations` 1.1.1 —
|
||||||
den hostede flaten er urørt (feltet er i ingen av hostings tre sett); og
|
den hostede flaten er urørt (feltet er i ingen av hostings tre sett); og
|
||||||
`--require-cost-baseline` er IKKE gjort til default (F4/D-3). Måling:
|
`--require-cost-baseline` er IKKE gjort til default (F4/D-3). Måling:
|
||||||
`docs/2026-09-09-p7-forankrede-identifikatorer.md`.
|
`docs/2026-09-09-p7-forankrede-identifikatorer.md`.
|
||||||
|
- **En kjøring SIER hva den leverte inputen kan forankre, før den bruker et forsøk på et forslag
|
||||||
|
som ikke kan bli forankret (P8, 09.09):** P7 er riktig og landet — men re-målingen avdekket en
|
||||||
|
konsekvens ingen rad uttalte: med gaten live faller **29 av 29 koder i 13 av 13 leverte forslag**
|
||||||
|
over de tre gratis-opptakene (PMs nevner; på den PARSEBARE nevneren 14 av 14 i 8 forslag — de
|
||||||
|
fem blobene som skiller tallene avvises av pydantics `claimed <= total` og når aldri stadium 0b).
|
||||||
|
Alle 29 VAR oppdiktet, så gaten er riktig; men en gate som alltid nekter er like ubrukelig som en
|
||||||
|
som aldri gjør det. **Årsaken er at PROMPTEN ber om noe inputen ikke kan levere:**
|
||||||
|
`_build_messages` krever at hver `affected_item` «restate a cost line as the project's price
|
||||||
|
schedule already carries it», mens K2s LEVERTE input bærer **9 forekomster / 2 distinkte**
|
||||||
|
kodeformede tokens — `SHA-01`/`SHA-10`, begge dokumentnumre fra en sidefot — og
|
||||||
|
`derive_cost_baseline` nekter basen: det finnes ingen kostlinje å gjengi. `GroundingOffer(chars,
|
||||||
|
identifiers, cost_lines)` er svaret, og **PARET er diagnosen**: «50 identifikatorer, 0
|
||||||
|
kostlinjer» sier det ingen av tallene sier alene. **En RAPPORT, aldri en gate** — den blokkerer
|
||||||
|
ikke, fordi et blokkerende krav ER `--require-cost-baseline` (F4/D-3, opt-in, urørt), og
|
||||||
|
`_ground_against_input` er URØRT. **Kallstedet er MÅLT, ikke valgt:** `generate.py` komponerer
|
||||||
|
grunnlaget PER FORSØK, etter `await _fetch_parsed`, så en rapport derfra kan først tale når ett
|
||||||
|
forsøk er betalt; `run.py` binder begge halvdeler over `--live-dry-run`-kuttet og før første
|
||||||
|
`debate.run`, så den FRIE turen sier det. `delivered` bindes ÉN gang og gir samme variabel til
|
||||||
|
rapporten og til `_evaluate` (to komposisjoner av én tekst er kø-(p)), og rapporten komponerer
|
||||||
|
GJENNOM `_grounding_text` — samme funksjon gaten bruker. **Et mønster er tillatt HER og ikke i
|
||||||
|
gaten**, og det er skillet mellom rapport og gate: en ukjent form er et token som ikke telles,
|
||||||
|
altså under-telling, aldri falsk avvisning. Formene er TRANSKRIBERT fra målingen (K2: 50
|
||||||
|
distinkte kodeformede, 0 kravnumre; N-korpusene: 269–981 distinkte kravnumre, ≤3 kodeformede);
|
||||||
|
**rene tall er BEVISST utelatt med tallet** (46 394 / 2 117 i K2 — å telle dem gjør hver rapport
|
||||||
|
positiv og målingen inert). `grounding_offer_notice` er ENESTE renderer og tier når kjøringen KAN
|
||||||
|
forankre — omisjon, aldri tom rad. Load-bearing MÅLT
|
||||||
|
(`tests/test_grounding_offer_loadbearing.py`, 12 armer), **ni mutasjoner alle røde mot HELE
|
||||||
|
suiten** + grønn kontroll **1570/5** (fra 1558/5, supersett, 0 fjernet) og golden BYTE-UENDRET
|
||||||
|
(`shasum -a 1` av INNHOLDET = `ea8c534773acdbe41ae68f2c55724d69aaf8be4f`): M1 teller alltid null
|
||||||
|
(2, kjent-positiven blant dem) · M2 teller alltid positivt (7, K2-kontrollen) · M3 bygg fra
|
||||||
|
`delivered` alene (1) · M4 når aldri `RunResult` (3) · M5 når aldri `DryRunReport` (3) ·
|
||||||
|
M6 rendereren skriver alltid linja (2) · M7 detach dry-run-utskriften (1) · M8 detach
|
||||||
|
fullkjørings-utskriften (1) · M9 rene tall telles (2). **Ærlighets-grenser, uttalt:** ingen betalt
|
||||||
|
kjøring bekrefter at rapporten endrer noe levende (tre opptak, ÉN modell, ETT deployment); den
|
||||||
|
BLOKKERER ikke, så en kjøring med null tilbud kan fortsatt brenne tre forsøk — VALGT, ikke
|
||||||
|
oversett; `cost_lines` er `len(baseline.items)`, altså forankringen som ANTALL, og står her fordi
|
||||||
|
paret er diagnosen; testfixturene er P7s SPORede prompter, ikke de leverte kuttene (10 kB / 96 kB
|
||||||
|
verbatim anbuds- og standardtekst inn i et repo publisert på `open/` er operatørens
|
||||||
|
publiseringsbeslutning, ikke denne ordrens) — 8-/435-distinkt-tallene er MÅLT og står i
|
||||||
|
dokumentet; portefølje-armen og den hostede flaten er BEVISST urørt; og tilbudsmålingen er gjort
|
||||||
|
på okf sin GAMLE default-bundle (`K2-bundle-20260903`), så en re-måling på den nye (436 konsepter
|
||||||
|
/ 832 filer) er en senere, separat ordre. Måling:
|
||||||
|
`docs/2026-09-09-p8-forankringstilbudet.md`.
|
||||||
- **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet.
|
- **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet.
|
||||||
- Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase.
|
- Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase.
|
||||||
|
|
||||||
|
|
|
||||||
227
docs/2026-09-09-p8-forankringstilbudet.md
Normal file
227
docs/2026-09-09-p8-forankringstilbudet.md
Normal file
|
|
@ -0,0 +1,227 @@
|
||||||
|
# P8 — den leverte inputen bærer ingen kostlinjer, så ingen forankret proposal kan oppstå
|
||||||
|
|
||||||
|
Ordre `20260909T142951Z-365478643-from-.claude`, økt 110. HEAD ved start: `277bb95`.
|
||||||
|
Alt i dette dokumentet er målt **gratis** — null modellkall, **NOK 0,00**.
|
||||||
|
|
||||||
|
## § 0 Hva som ER målt og hva som IKKE er det
|
||||||
|
|
||||||
|
**Målt:** hva de tre gratis opptakene faktisk leverte og hvor mange av kodene P7-gaten feller;
|
||||||
|
hva den LEVERTE inputen i hvert korpus po har liggende kan tilby som lovlig `affected_item`-kode;
|
||||||
|
og at rapporten som nå bygges sier null der tilbudet er null og positivt der det er positivt.
|
||||||
|
|
||||||
|
**Ikke målt:** at rapporten endrer noe LEVENDE. Ingen betalt kjøring er gjort, og ingen er
|
||||||
|
bestilt. At en modell som får se linja oppfører seg annerledes er ikke vist — samme klasse som
|
||||||
|
structured-output-grensen. Rapporten er dessuten skrevet på grunnlag av tre opptak fra **ÉN**
|
||||||
|
modell og **ETT** deployment.
|
||||||
|
|
||||||
|
**Ikke bygget, med vilje:** prosa-skanningen (operatørbeslutning A) og
|
||||||
|
`--require-cost-baseline` som default (operatørbeslutning, F4/D-3). Begge står uendret.
|
||||||
|
|
||||||
|
## § 1 Premissene (i)–(viii) — hver verifisert selv
|
||||||
|
|
||||||
|
| # | Premiss (PM) | Mitt utfall |
|
||||||
|
|---|---|---|
|
||||||
|
| (i) | HEAD `277bb95`, `git ls-remote origin main` = `eb41374`, upushet 2 | **BEKREFTET** ordrett |
|
||||||
|
| (ii) | To utrackede: presentasjons-HTML (parallell økt) + `scratchpad/` | **BEKREFTET**, intet tredje |
|
||||||
|
| (iii) | 1558 passed / 5 skipped, ruff rent, mypy rent (37 filer), golden `ea8c534…` | **BEKREFTET** alle fire |
|
||||||
|
| (iv) | 13 forslag / 29 koder / 29 ugrunnet | **BEKREFTET på PMs nevner, med et avvik — se § 2** |
|
||||||
|
| (v) | K2 leverer 9/2 distinkte kodeformede (`SHA-01`/`SHA-10`, dokumentnumre); N100 0 kodeformede, 17/8 kravnumre | **BEKREFTET ordrett** |
|
||||||
|
| (vi) | Ved uttømte forsøk returneres siste `Rejection`, ingen krasj | **BEKREFTET**: `generate.py` returnerer `GenerationResult(outcome=last_ruling, …)` etter forsøksløkka |
|
||||||
|
| (vii) | `--require-cost-baseline` er opt-in og skal ikke bli default | **RESPEKTERT**, urørt |
|
||||||
|
| (viii) | (A) og (B) er operatørens | **RESPEKTERT**, ikke avgjort her |
|
||||||
|
|
||||||
|
## § 2 Konsekvensen — og AVVIKET mot PMs tall
|
||||||
|
|
||||||
|
PMs tall er reprodusert nøyaktig **på PMs nevner**, men den nevneren teller blobs som aldri
|
||||||
|
NÅR P7-gaten. Begge tall er sanne om hver sin ting, og begge gir 100 %:
|
||||||
|
|
||||||
|
| Opptak | Rå JSON-blobs | Koder | Ugrunnet | Parsebare til IR | Koder | Ugrunnet |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| `p6-Aopen-records.json` | 2 | 4 | **4** | 2 | 4 | **4** |
|
||||||
|
| `Aopen-records.json` | 8 | 22 | **22** | 3 | 7 | **7** |
|
||||||
|
| `p4-n100-free-records.json` | 3 | 3 | **3** | 3 | 3 | **3** |
|
||||||
|
| **Totalt** | **13** | **29** | **29 (100 %)** | **8** | **14** | **14 (100 %)** |
|
||||||
|
|
||||||
|
De fem blobene som skiller tallene avvises av en **annen og TIDLIGERE falsifiserer** —
|
||||||
|
pydantics `claimed_saving_nok <= affected items' total` — og når derfor aldri stadium 0b.
|
||||||
|
Det er ikke en svakhet i PMs måling; det er to nevnere om to ulike hendelser. Grunnlaget er
|
||||||
|
det **mest sjenerøse** som finnes: unionen av ALLE prompter i opptaket. Selv der er ingen kode
|
||||||
|
grunnet.
|
||||||
|
|
||||||
|
Et instrument-forbehold, skrevet ned fordi det først ga feil svar: en rå
|
||||||
|
`SavingsProposal.model_validate_json` avviser **alle 13**, fordi svarene bærer WIRE-formen av
|
||||||
|
`assumptions` (et array) som `_normalise_assumptions` folder tilbake. En måling som stoppet der
|
||||||
|
ville rapportert 0 forslag — null fordi instrumentet ikke kunne lese formen, ikke fordi formen
|
||||||
|
manglet.
|
||||||
|
|
||||||
|
## § 3 Årsaken — prompten ber om noe inputen ikke kan levere
|
||||||
|
|
||||||
|
`generate._build_messages` sier ordrett: «Each entry in `affected_items` must restate a cost line
|
||||||
|
as the project's price schedule already carries it». Målt på record 0 (turen som bar bundelen):
|
||||||
|
|
||||||
|
* **K2** (P6: 96 567 tegn; S7c: 145 281 tegn) → **9 forekomster / 2 distinkte** kodeformede, og
|
||||||
|
begge er `SHA-01` / `SHA-10`. Konteksten er entydig: `Oppdragsnr.: 52308329 Dokumentnr.: SHA-01
|
||||||
|
Versjon: 01` — en sidefot i en SHA-plan — og `Dokumentnavn: SHA-10 - … Restrisikorapport`.
|
||||||
|
**Dokumentnumre, ikke kostlinjer.**
|
||||||
|
* **N100** (10 569 tegn) → **0** kodeformede, men **17 forekomster / 8 distinkte** kravnumre på
|
||||||
|
`Krav X.Y.Z—N`-form (em-dash U+2014).
|
||||||
|
|
||||||
|
Modellen ble bedt om å gjengi en kostlinje som ikke finnes, og fabrikkerte den. Dette rimer med
|
||||||
|
funn 4 (økt 107): K2s prisskjema bærer ingen mengdefortegnelse, 0 av 92 rader navngir
|
||||||
|
kode/mengde/enhetspris.
|
||||||
|
|
||||||
|
## § 4 Tilbudet per korpus — målt med den SHIPPEDE funksjonen
|
||||||
|
|
||||||
|
Teksten er komponert nøyaktig som `run_project` komponerer P7s grunnlag
|
||||||
|
(`"\n".join([context, bundle_grounding])`, deretter gjennom `generate._grounding_text`).
|
||||||
|
«Kostlinje» er po sin EGEN definisjon (`okf.derive_cost_baseline`, MAJOR-4) — ikke en ny
|
||||||
|
heuristikk oppfunnet her.
|
||||||
|
|
||||||
|
| Korpus | Konsepter | Grunnlag (tegn) | Distinkte identifikatorer | Kostlinjer | `derive_cost_baseline` |
|
||||||
|
|---|---|---|---|---|---|
|
||||||
|
| K2 s7c + `payload-open` | 630 | 1 991 597 | 50 | **0** | REFUSED |
|
||||||
|
| K2 s7c + `payload-default` | 630 | 1 970 415 | 50 | **0** | REFUSED |
|
||||||
|
| K2 s7a2 (peker-armen, P6) | 629 | 1 894 500 | 50 | **0** | REFUSED |
|
||||||
|
| N100 `n100-2023` | 446 | 462 041 | 435 | **0** | REFUSED |
|
||||||
|
| N200 `n200-2024` | 1 133 | 1 500 962 | 982 | **0** | REFUSED |
|
||||||
|
| N500 `n500-2024` | 270 | 408 220 | 272 | **0** | REFUSED |
|
||||||
|
| `shared/veglys-fv-soer` | 5 | 30 038 | 3 | **1** | REFUSED (men fila finnes) |
|
||||||
|
| `shared/tunnel-hauglia` | 5 | 36 546 | 2 | **1** | REFUSED (men fila finnes) |
|
||||||
|
| `shared/bygg-energi-mikro` | 4 | 11 019 | 2 | **0** | REFUSED |
|
||||||
|
|
||||||
|
**Klassene, målt.** K2s 50 distinkte kodeformede er tegningsnumre (`B-20-00-00`, `F-20-00-01`,
|
||||||
|
`V-73-20-01-01`), dokumentnumre (`SHA-01`, `RIM-02`, `RIA-01`, `NOT-01`), stoff- og
|
||||||
|
standardreferanser (`PCB-7`, `PAH-16`, `DALI-2`) og forskriftsnumre
|
||||||
|
(`FOR-2011-12-06-1357`). **Ingen av dem er en kostkode.** N-korpusene bærer kravnumre og UUID-er
|
||||||
|
(N200: 2 290 distinkte UUID-er), ingen kostkoder.
|
||||||
|
|
||||||
|
**Er et forankret forslag i det hele tatt MULIG?** For **K2: NEI**, og tallet er 0 —
|
||||||
|
`derive_cost_baseline` nekter basen, og ingen av de 50 identifikatorene er en kostlinje. Bare
|
||||||
|
**2 av 50** når i det hele tatt prompten. For **N-korpusene: NEI** på kostlinje — en vegnormal
|
||||||
|
bærer ingen — men **JA** på identifikator: 435 / 982 / 272 distinkte kravnumre er sitérbare, og
|
||||||
|
8 av dem sto i N100-prompten.
|
||||||
|
|
||||||
|
**Rene tall er farligst og bæres videre uendret:** 46 394 forekomster / 2 117 distinkte i K2
|
||||||
|
(P7 § 2). Regelen er inert mot dem og feiler ÅPENT. Ikke bygget om på.
|
||||||
|
|
||||||
|
**Gammel default.** Alt over er målt på `K2-bundle-20260903`. okf melder (innboks
|
||||||
|
`20260909T134141Z`, `reply-expected: no`, lukket med `coord-done`) at deres default-bygg sluttet å
|
||||||
|
emittere den 2026-09-08, at gjeldende default er 436 konsepter / 832 filer, og at de leser
|
||||||
|
formfunnet som fortsatt stående fordi endringen treffer FORMEN, ikke innholdet. Ingen ny bundle er
|
||||||
|
bygget eller konsumert her — deres endring er ikke pushet, og en re-måling på den nye defaulten er
|
||||||
|
en senere, separat ordre.
|
||||||
|
|
||||||
|
## § 5 Hva som er bygget — en RAPPORT, ikke en gate
|
||||||
|
|
||||||
|
`generate.GroundingOffer(chars, identifiers, cost_lines)` + `generate.grounding_offer(...)`,
|
||||||
|
kalt fra `run.py`, rendret av `run.grounding_offer_notice`.
|
||||||
|
|
||||||
|
**Valget av kallsted, med grunnen (ordrens eget krav).** Begge kandidater ble lest først.
|
||||||
|
`generate.py` komponerer grunnlaget **per forsøk, ETTER `await _fetch_parsed(messages)`** — en
|
||||||
|
rapport derfra kan først tale når ett forsøk allerede er betalt, altså nøyaktig det ordren ber
|
||||||
|
den om å komme foran. `run.py` binder begge halvdeler ved `run.py:1228`, **over
|
||||||
|
`--live-dry-run`-kuttet og før første `debate.run`**. Valgt: **`run.py`**. Tellefunksjonen bor
|
||||||
|
likevel i `generate.py`, ved siden av `_grounding_text` den måler — å skille dem ville gitt to
|
||||||
|
steder å bli uenige på.
|
||||||
|
|
||||||
|
**De fire kravene:**
|
||||||
|
|
||||||
|
1. **BLOKKERER ikke.** En kjøring med null tilbud kjører som før. Et blokkerende krav ER
|
||||||
|
`--require-cost-baseline`, som premiss (vii) fredet.
|
||||||
|
2. **Måler den EKSAKTE teksten.** `grounding_offer` komponerer GJENNOM `_grounding_text` —
|
||||||
|
samme funksjon P7s gate bruker — og `run.py` binder `delivered` **én gang** og gir samme
|
||||||
|
variabel til både rapporten og `_evaluate`. To komposisjoner av én tekst er fri til å være
|
||||||
|
uenige (kø-(p)); her er de identiske ved konstruksjon.
|
||||||
|
3. **Når utfallet operatøren leser.** `RunResult.grounding_offer`, `DryRunReport.grounding_offer`
|
||||||
|
og én linje på stdout i begge CLI-armene.
|
||||||
|
4. **Gjenbruker P7s sømmer.** Ingen ny domstype ved siden av `Rejection`/`ValidatedProposal`.
|
||||||
|
`_ground_against_input` er **URØRT**.
|
||||||
|
|
||||||
|
**Identifikator-formene er TRANSKRIBERT fra målingen i § 4**, ikke valgt: `[A-ZÆØÅ]{1,8}[-_]\d…`
|
||||||
|
(K2s 50) og `Krav X.Y.Z—N` (N-korpusenes dominerende form). At et mønster er tillatt HER og ikke i
|
||||||
|
`_ground_against_input` er selve skillet mellom en rapport og en gate: en form rapporten ikke
|
||||||
|
kjenner er et token den unnlater å telle, altså en **under-telling** — aldri en falsk avvisning.
|
||||||
|
**Rene tall er BEVISST utelatt, med tallet** (46 394 / 2 117): å telle dem ville gjort hver rapport
|
||||||
|
positiv og målingen inert — repoets kardinalklasse, en gate som bare kan bli grønn.
|
||||||
|
|
||||||
|
Rendereren er **ÉN**, og den tier når kjøringen KAN forankre en kostlinje — omisjon, aldri en tom
|
||||||
|
rad (`cost_baseline_notice`s regel). Den bærer **begge tall**, fordi paret er diagnosen:
|
||||||
|
«0 kostlinjer» alene leses som en gjentakelse av `cost_baseline_notice`, «50 identifikatorer» alene
|
||||||
|
leses som gode nyheter.
|
||||||
|
|
||||||
|
## § 6 Kjent-positiv og kontroll
|
||||||
|
|
||||||
|
**Kjent-positiven (bindende):** N100s input bærer `Krav 3.3.1—13` (em-dash U+2014; P7 premiss
|
||||||
|
(vii): 6 av 6 prompter). Rapporten sier **positivt tilbud** der, og armen er paret med en kontroll
|
||||||
|
på at strengen faktisk STÅR i fixturen — en rapport som fant null fordi den lette etter ingenting
|
||||||
|
ville ellers bestått.
|
||||||
|
|
||||||
|
**Kontrollen på samme materiale:** K2s to genererings-prompter, der tilbudet er **null i begge
|
||||||
|
tall**. Uten den beviser en grønn kjent-positiv ingenting.
|
||||||
|
|
||||||
|
**Fixturvalget, uttalt.** Testene leser P7s egne SPORede fixturer under
|
||||||
|
`tests/fixtures/p7-grounding/`. De leverte KUTTENE (10 kB N100, 96 kB K2) er ikke sporet: å
|
||||||
|
committe verbatim anbudstekst og standardtekst inn i et repo som publiseres på `open/` er en
|
||||||
|
publiseringsbeslutning som tilhører operatøren, ikke denne ordren. 8-/435-distinkt-tallene står
|
||||||
|
derfor i § 4 som MÅLINGER; det testene asserterer er EGENSKAPEN, på tekst repoet allerede
|
||||||
|
shipper.
|
||||||
|
|
||||||
|
## § 7 Mutasjonstabellen
|
||||||
|
|
||||||
|
Grønn kontroll **1570 passed / 5 skipped** (fra 1558/5; **+12 node-ider, 0 fjernet**, målt med
|
||||||
|
`comm` mot en liste bygget fra HEAD). Golden `demo-transcript.stdout` BYTE-UENDRET,
|
||||||
|
`shasum -a 1` av INNHOLDET = `ea8c534773acdbe41ae68f2c55724d69aaf8be4f` (ALDRI git-blob-id-en).
|
||||||
|
Hver mutasjon kjørt mot HELE suiten, én per kall, restaurert fra `scratchpad/p8/mut-backup/`
|
||||||
|
med `shasum -c`, aldri `git checkout`.
|
||||||
|
|
||||||
|
| # | Mutasjon | Røde |
|
||||||
|
|---|---|---|
|
||||||
|
| M1 | rapporten teller alltid null | **2** (kjent-positiven blant dem) |
|
||||||
|
| M2 | rapporten teller alltid positivt | **7** (K2-kontrollen først) |
|
||||||
|
| M3 | rapporten bygges fra `delivered` alene, ikke gjennom `_grounding_text` | **1** (krav (2)-armen alene) |
|
||||||
|
| M4 | rapporten når aldri `RunResult` | **3** |
|
||||||
|
| M5 | rapporten når aldri `DryRunReport` | **3** |
|
||||||
|
| M6 | rendereren skriver alltid linja | **2** (omisjonen er selv gatet, på begge flater) |
|
||||||
|
| M7 | detach dry-run-utskriften | **1** |
|
||||||
|
| M8 | detach fullkjørings-utskriften | **1** |
|
||||||
|
| M9 | rene tall telles som tilbud | **2** |
|
||||||
|
|
||||||
|
Ni mutasjoner, **alle røde**. Ingen grønn mutasjon, altså ingen uvitnet søm i denne leveransen.
|
||||||
|
|
||||||
|
## § 8 Honesty limits
|
||||||
|
|
||||||
|
* **Alt er målt OFFLINE** på tre opptak fra **ÉN modell** og **ETT deployment**. Ingen betalt
|
||||||
|
kjøring bekrefter at rapporten endrer noe levende, og at en modell som ser linja velger
|
||||||
|
annerledes er **ikke vist** (structured-output-grensens klasse).
|
||||||
|
* **Rapporten BLOKKERER ikke.** En kjøring med null tilbud kan fortsatt brenne tre forsøk på et
|
||||||
|
forslag som ikke kan bli forankret. Det er **VALGT**, ikke oversett: et blokkerende krav er
|
||||||
|
`--require-cost-baseline`, og F4/D-3 la den beslutningen hos operatøren.
|
||||||
|
* **Identifikator-formene er et mønster**, og et mønster kan mangle en form. Retningen er
|
||||||
|
under-telling, aldri falsk avvisning — men et korpus med en tredje form vil rapportere lavere
|
||||||
|
enn det tilbyr, til noen måler den formen.
|
||||||
|
* **`cost_lines` er `len(baseline.items)`**, lest av SAMME `baseline`-binding `_grounding_text`
|
||||||
|
tar som sin tredje kilde. Den gjentar altså forankringen som et ANTALL. Den står her fordi
|
||||||
|
paret er diagnosen, ikke fordi antallet er en ny kjensgjerning.
|
||||||
|
* **Portefølje-armen er BEVISST ikke wiret.** `run_portfolio` skriver ingen slik linje;
|
||||||
|
`bundle_id_notice`s avgjørelse, ikke `cost_baseline_notice`s. Asymmetrien står her fordi
|
||||||
|
stillhet om den er det eneste gale svaret.
|
||||||
|
* **Den hostede flaten er urørt.** Feltet er i ingen av hostings tre sett, så Fase 4es to
|
||||||
|
halvdeler står uendret.
|
||||||
|
* **Tilbudsmålingen er gjort på okf sin GAMLE default-bundle** (`K2-bundle-20260903`). En
|
||||||
|
re-måling på den nye (436 konsepter / 832 filer) er en senere, separat ordre.
|
||||||
|
* **(A) prosa-skanningen og (B) blindsone-valget forblir OPERATØRENS.** Målingen her informerer
|
||||||
|
(B) — den handler om **TILBUDET** i inputen, ikke om hvilket utdrag modellen **VELGER** — men
|
||||||
|
den avgjør den ikke.
|
||||||
|
|
||||||
|
## § 9 Reproduksjon
|
||||||
|
|
||||||
|
uv run pytest -q tests/test_grounding_offer_loadbearing.py
|
||||||
|
uv run python -m portfolio_optimiser.run BYGG-KONTOR-NORD \
|
||||||
|
--docs-dir shared/examples/bygg-energi-mikro \
|
||||||
|
--bundle-dir shared/examples/bygg-energi-mikro --live-dry-run
|
||||||
|
uv run python -m portfolio_optimiser.run VEGLYS-FV-SOER \
|
||||||
|
--docs-dir shared/examples/veglys-fv-soer \
|
||||||
|
--bundle-dir shared/examples/veglys-fv-soer --live-dry-run
|
||||||
|
|
||||||
|
Den første basen har ingen kostbaseline og skriver tilbudslinja; den andre har én og tier.
|
||||||
|
Måleskriptene ligger i `scratchpad/p8/` (utracket).
|
||||||
|
|
@ -22,6 +22,7 @@ Two entry points, because the LLM call is async while ``validator.self_repair``
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
import re
|
||||||
from collections.abc import Callable, Mapping
|
from collections.abc import Callable, Mapping
|
||||||
from dataclasses import dataclass, field, replace
|
from dataclasses import dataclass, field, replace
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
@ -456,6 +457,79 @@ def _grounding_text(project: Project, baseline: CostBaseline | None, delivered:
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
#: The identifier forms the DELIVERED corpora actually carry, TRANSCRIBED from the measurement
|
||||||
|
#: (``docs/2026-09-09-p8-forankringstilbudet.md`` § 2) rather than chosen: over K2's two delivered
|
||||||
|
#: bundles the first form finds 50 distinct tokens and the second none, while over the three
|
||||||
|
#: N payloads the second finds 269-981 distinct and the first at most 3.
|
||||||
|
#:
|
||||||
|
#: **Why a pattern is admissible HERE and not in ``_ground_against_input``.** That is a GATE, and a
|
||||||
|
#: pattern there would be a rule about shapes that can wrongly REFUSE a real code. This is a
|
||||||
|
#: REPORT: a form it does not know is a token it fails to count, so it errs toward saying the input
|
||||||
|
#: offers LESS than it does — an under-count is a quiet report, never a false rejection.
|
||||||
|
#:
|
||||||
|
#: Bare numbers are deliberately EXCLUDED, with the number: K2 carries 46 394 occurrences over
|
||||||
|
#: 2 117 distinct values (P7 § 2), so counting them would make every report positive and the
|
||||||
|
#: measurement inert — the repo's cardinal class, a gate that can only come out green.
|
||||||
|
_IDENTIFIER_FORMS = (
|
||||||
|
# ``SHA-01``, ``RIM-02``, ``B-20-00-00``, ``FOR-2011-12-06-1357`` (K2's 50).
|
||||||
|
re.compile(r"\b[A-ZÆØÅ]{1,8}[-_]\d{1,4}(?:[-_]\d{1,4})*\b"),
|
||||||
|
# ``Krav 3.3.1—13`` (the N corpora's dominant form). EM-DASH U+2014 AND the hyphen, because the
|
||||||
|
# binding known positive is the em-dash spelling and only the em-dash spelling scores 6 of 6.
|
||||||
|
re.compile(r"Krav\s+\d+(?:\.\d+)*\s*[\u2014-]\s*\d+"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class GroundingOffer:
|
||||||
|
"""P8: what the DELIVERED input of one run can lawfully ground an ``affected_item`` code in.
|
||||||
|
|
||||||
|
A REPORT, never a gate. It blocks nothing — a run whose offer is null still runs — because a
|
||||||
|
blocking requirement is exactly ``--require-cost-baseline``, which F4 settled as opt-in.
|
||||||
|
|
||||||
|
The PAIR is the whole diagnosis, and neither number says it alone. MEASURED on K2: the
|
||||||
|
delivered text offers 50 citable identifiers and ZERO cost lines, while the proposer prompt
|
||||||
|
asks each ``affected_item`` to "restate a cost line as the project's price schedule already
|
||||||
|
carries it". An operator reading "identifiers: 50, cost lines: 0" learns that no attempt could
|
||||||
|
have succeeded; reading either number by itself, they learn nothing of the sort.
|
||||||
|
|
||||||
|
``chars`` is the size of the exact text P7's gate will measure against — carried so the report
|
||||||
|
and the gate can be seen to be talking about the same input, which is the only defence against
|
||||||
|
a second rendering free to disagree with the one that was sent.
|
||||||
|
"""
|
||||||
|
|
||||||
|
#: Length of ``_grounding_text``'s output — the text the gate itself will search.
|
||||||
|
chars: int
|
||||||
|
#: Distinct tokens of any ``_IDENTIFIER_FORMS`` shape the text carries.
|
||||||
|
identifiers: int
|
||||||
|
#: Cost lines this run can anchor one of them AS. 0 when the run is un-anchored, which is the
|
||||||
|
#: state every free K2 arm measured. Read off the SAME ``baseline`` binding ``_grounding_text``
|
||||||
|
#: takes as its third source, so the count and the anchoring can never disagree.
|
||||||
|
cost_lines: int
|
||||||
|
|
||||||
|
|
||||||
|
def grounding_offer(
|
||||||
|
project: Project, baseline: CostBaseline | None, delivered: str
|
||||||
|
) -> GroundingOffer:
|
||||||
|
"""Measure what ``delivered`` can ground, on the EXACT text the gate will see.
|
||||||
|
|
||||||
|
Composed THROUGH ``_grounding_text`` — the one composer ``generate_via_llm`` passes to
|
||||||
|
``validate_proposal`` — never re-assembled here. A second composition would be free to drift
|
||||||
|
from the one that was actually sent, and a report about a text nobody was given is worse than
|
||||||
|
no report: it reads as evidence.
|
||||||
|
|
||||||
|
Deterministic and free: no model call, no network, and no second walk of the bundle.
|
||||||
|
"""
|
||||||
|
text = _grounding_text(project, baseline, delivered)
|
||||||
|
found: set[str] = set()
|
||||||
|
for form in _IDENTIFIER_FORMS:
|
||||||
|
found |= set(form.findall(text))
|
||||||
|
return GroundingOffer(
|
||||||
|
chars=len(text),
|
||||||
|
identifiers=len(found),
|
||||||
|
cost_lines=0 if baseline is None else len(baseline.items),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
async def generate_via_llm(
|
async def generate_via_llm(
|
||||||
chat_client: BaseChatClient,
|
chat_client: BaseChatClient,
|
||||||
project: Project,
|
project: Project,
|
||||||
|
|
|
||||||
|
|
@ -80,7 +80,12 @@ from portfolio_optimiser.explore import (
|
||||||
tool_call_payload,
|
tool_call_payload,
|
||||||
trace_payload,
|
trace_payload,
|
||||||
)
|
)
|
||||||
from portfolio_optimiser.generate import ParseFailure, generate_via_llm
|
from portfolio_optimiser.generate import (
|
||||||
|
GroundingOffer,
|
||||||
|
ParseFailure,
|
||||||
|
generate_via_llm,
|
||||||
|
grounding_offer,
|
||||||
|
)
|
||||||
from portfolio_optimiser.ir import SavingsProposal
|
from portfolio_optimiser.ir import SavingsProposal
|
||||||
from portfolio_optimiser.mandate import (
|
from portfolio_optimiser.mandate import (
|
||||||
OWN_PROPOSAL_ID,
|
OWN_PROPOSAL_ID,
|
||||||
|
|
@ -248,6 +253,15 @@ class RunResult:
|
||||||
#: Carried HERE and not on ``ProvenanceStamp``: the stamp describes the gate that judged ONE
|
#: Carried HERE and not on ``ProvenanceStamp``: the stamp describes the gate that judged ONE
|
||||||
#: candidate, while this is a RUN-level fact about what the run was allowed to read at all.
|
#: candidate, while this is a RUN-level fact about what the run was allowed to read at all.
|
||||||
prepass: prepass.PrepassDeclaration | None = None
|
prepass: prepass.PrepassDeclaration | None = None
|
||||||
|
#: What this run's DELIVERED input could ground an ``affected_item`` code in (P8): the size of
|
||||||
|
#: the text P7's gate searched, how many distinct identifiers of a measured form it carries,
|
||||||
|
#: and how many cost lines the run can anchor one of them AS.
|
||||||
|
#:
|
||||||
|
#: A RUN-level fact settled ONCE, before any candidate exists, which is why it is here and not
|
||||||
|
#: on ``ProvenanceStamp`` (that describes the gate that judged ONE candidate) — ``skipped_links``'
|
||||||
|
#: own placement rule. DEFAULTED, ``prepass``' half of the rule: ``None`` is the true statement
|
||||||
|
#: "no measurement was made", and there is exactly one place that makes it.
|
||||||
|
grounding_offer: GroundingOffer | None = None
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def verdict_key(self) -> str:
|
def verdict_key(self) -> str:
|
||||||
|
|
@ -313,6 +327,12 @@ class DryRunReport:
|
||||||
#: resolves a payload, so a dry run can honestly report what it would have read — whereas a
|
#: resolves a payload, so a dry run can honestly report what it would have read — whereas a
|
||||||
#: field resolved above that cut could only ever report zero.
|
#: field resolved above that cut could only ever report zero.
|
||||||
prepass: prepass.PrepassDeclaration | None = None
|
prepass: prepass.PrepassDeclaration | None = None
|
||||||
|
#: What the delivered input of a REAL run of this configuration could ground an
|
||||||
|
#: ``affected_item`` code in (P8). Carried here for ``cost_baseline_anchored``'s reason and,
|
||||||
|
#: more sharply, because this is the surface on which "before it spends its three attempts"
|
||||||
|
#: is provable at all: the dry-run cut returns before the first model call, so a dry run that
|
||||||
|
#: reports a null offer has said the run cannot succeed WITHOUT paying to find out.
|
||||||
|
grounding_offer: GroundingOffer | None = None
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
|
|
@ -781,6 +801,37 @@ def cost_baseline_notice(anchored: bool) -> str | None:
|
||||||
return None if anchored else _UNANCHORED_NOTICE
|
return None if anchored else _UNANCHORED_NOTICE
|
||||||
|
|
||||||
|
|
||||||
|
def grounding_offer_notice(offer: GroundingOffer | None) -> str | None:
|
||||||
|
"""Render the one line that says what this run's delivered input can ground, or ``None`` when
|
||||||
|
there is nothing to warn about (P8).
|
||||||
|
|
||||||
|
ONE renderer with N callsites, never N copies of the wording (kø-(p)), taking the
|
||||||
|
ALREADY-MEASURED value rather than a text: a renderer that re-composed the grounding would be
|
||||||
|
a second resolution of the same rule, free to drift from the run it describes
|
||||||
|
(``cost_baseline_notice``'s rule).
|
||||||
|
|
||||||
|
``None`` when the run CAN anchor a cost line — omission, never an empty row
|
||||||
|
(``mandate.announce``'s rule) — and ``None`` on ``None``, which is the honest reading of "no
|
||||||
|
measurement was made". This is deliberately NOT ``proposal_review_notice``'s deviation: there,
|
||||||
|
silence on zero was ambiguous; here, a run that can anchor its lines has nothing to report that
|
||||||
|
the outcome does not already say.
|
||||||
|
|
||||||
|
BOTH numbers, because the pair is the diagnosis. "0 cost lines" alone reads as a restatement of
|
||||||
|
``cost_baseline_notice``; "50 identifiers" alone reads as good news. Together they say the
|
||||||
|
thing P8 measured: the input offers plenty to cite and nothing to cost, while the proposer
|
||||||
|
prompt asks for a cost line.
|
||||||
|
|
||||||
|
English, like every other line this CLI prints."""
|
||||||
|
if offer is None or offer.cost_lines > 0:
|
||||||
|
return None
|
||||||
|
return (
|
||||||
|
f" Grounding offer: {offer.identifiers} distinct identifier(s) and "
|
||||||
|
f"{offer.cost_lines} cost line(s) in the {offer.chars} characters this run was given — "
|
||||||
|
"the proposer is asked to restate a cost line this input does not carry, so every "
|
||||||
|
"candidate it invents will be refused as ungrounded"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def bundle_id_notice(resolved: okf.ResolvedBundleId | None) -> str | None:
|
def bundle_id_notice(resolved: okf.ResolvedBundleId | None) -> str | None:
|
||||||
"""Render the one line that says a base was mounted under a name it does not answer to, or
|
"""Render the one line that says a base was mounted under a name it does not answer to, or
|
||||||
``None`` when there is nothing to say.
|
``None`` when there is nothing to say.
|
||||||
|
|
@ -1166,6 +1217,17 @@ async def run_project(
|
||||||
"--derive-cost-baseline when the base carries a priced schedule"
|
"--derive-cost-baseline when the base carries a priced schedule"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# P8: what this run was GIVEN, composed ONCE. Bound HERE and not inside ``_evaluate`` below,
|
||||||
|
# and that placement is the measurement this seam rests on: this is the first point at which
|
||||||
|
# both halves exist AND it is above the ``--live-dry-run`` cut, so the offer can be reported
|
||||||
|
# on the FREE trip — before the first model call at ``debate.run``, let alone the three
|
||||||
|
# generation attempts. ``generate.py``'s own composition happens per attempt, AFTER
|
||||||
|
# ``_fetch_parsed`` has returned, so a report from there could only ever speak once an attempt
|
||||||
|
# had been paid for. ONE binding feeding both the report and the gate: two compositions of one
|
||||||
|
# text are free to disagree, which is exactly what a report must not be able to do (kø-(p)).
|
||||||
|
delivered = "\n".join([context, bundle_grounding])
|
||||||
|
offer = grounding_offer(project, baseline, delivered)
|
||||||
|
|
||||||
# Trekk B2 (krav 3): configured MCP servers become tools the AGENTS can call during the debate.
|
# Trekk B2 (krav 3): configured MCP servers become tools the AGENTS can call during the debate.
|
||||||
# Appended to BOTH paths — on the bundle path they are the first tools that path has ever had.
|
# Appended to BOTH paths — on the bundle path they are the first tools that path has ever had.
|
||||||
# Constructed here but NOT connected: an ``MCPTool`` is an async context manager, so the run
|
# Constructed here but NOT connected: an ``MCPTool`` is an async context manager, so the run
|
||||||
|
|
@ -1239,6 +1301,9 @@ async def run_project(
|
||||||
bundle_id_source=resolved_bundle_id,
|
bundle_id_source=resolved_bundle_id,
|
||||||
skipped_links=skipped_links,
|
skipped_links=skipped_links,
|
||||||
prepass=prepass_declaration,
|
prepass=prepass_declaration,
|
||||||
|
# P8, and this surface is the point: the offer is measured ABOVE this cut, so a dry
|
||||||
|
# run reports it having made no model call at all.
|
||||||
|
grounding_offer=offer,
|
||||||
)
|
)
|
||||||
# The MCP lifecycle (Trekk B2): entered HERE, after the dry-run cut above, so a dry run never
|
# The MCP lifecycle (Trekk B2): entered HERE, after the dry-run cut above, so a dry run never
|
||||||
# opens a connection — its promise to stop before the first call covers egress too. Constructed
|
# opens a connection — its promise to stop before the first call covers egress too. Constructed
|
||||||
|
|
@ -1378,7 +1443,7 @@ async def run_project(
|
||||||
# ``context`` is the DELIVERED rendering (pre-pass cut / bundle pointer / retrieved
|
# ``context`` is the DELIVERED rendering (pre-pass cut / bundle pointer / retrieved
|
||||||
# chunks) — never ``gen_context``, which on the debate path is the model's own
|
# chunks) — never ``gen_context``, which on the debate path is the model's own
|
||||||
# summary and would let a code the debate invented ground the proposal repeating it.
|
# summary and would let a code the debate invented ground the proposal repeating it.
|
||||||
grounding="\n".join([context, bundle_grounding]),
|
grounding=delivered,
|
||||||
)
|
)
|
||||||
refinements.extend(generated.refinements)
|
refinements.extend(generated.refinements)
|
||||||
return generated.outcome
|
return generated.outcome
|
||||||
|
|
@ -1587,6 +1652,9 @@ async def run_project(
|
||||||
debate_tool_calls=tuple(debate_tool_calls),
|
debate_tool_calls=tuple(debate_tool_calls),
|
||||||
expert_revisions=tuple(expert_reviews),
|
expert_revisions=tuple(expert_reviews),
|
||||||
prepass=prepass_declaration,
|
prepass=prepass_declaration,
|
||||||
|
# P8: read off the SAME single measurement the gate's own grounding descends from, so the
|
||||||
|
# record and the refusals cannot describe different inputs.
|
||||||
|
grounding_offer=offer,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -3836,6 +3904,12 @@ def main(argv: list[str] | None = None) -> int:
|
||||||
notice = cost_baseline_notice(report.cost_baseline_anchored)
|
notice = cost_baseline_notice(report.cost_baseline_anchored)
|
||||||
if notice is not None:
|
if notice is not None:
|
||||||
print(notice)
|
print(notice)
|
||||||
|
# P8, printed next to the line it qualifies: "stage 0 is skipped" says the gate lost a
|
||||||
|
# falsifier; this says what the input could have offered it instead. On the FREE trip, so
|
||||||
|
# an operator learns a run cannot be grounded without paying three attempts to find out.
|
||||||
|
offer_notice = grounding_offer_notice(report.grounding_offer)
|
||||||
|
if offer_notice is not None:
|
||||||
|
print(offer_notice)
|
||||||
# The second measured silence on this surface: a bundle with an unfollowable cross-link
|
# The second measured silence on this surface: a bundle with an unfollowable cross-link
|
||||||
# dry-ran to rc 0 with nothing said, so a half-read base looked exactly like a small one.
|
# dry-ran to rc 0 with nothing said, so a half-read base looked exactly like a small one.
|
||||||
nav_notice = skipped_links_notice(report.skipped_links)
|
nav_notice = skipped_links_notice(report.skipped_links)
|
||||||
|
|
@ -3925,6 +3999,11 @@ def main(argv: list[str] | None = None) -> int:
|
||||||
notice = cost_baseline_notice(result.provenance.cost_baseline_anchored)
|
notice = cost_baseline_notice(result.provenance.cost_baseline_anchored)
|
||||||
if notice is not None:
|
if notice is not None:
|
||||||
print(notice)
|
print(notice)
|
||||||
|
# Same renderer on the full run, read off the run's OWN measurement: a run that spent every
|
||||||
|
# attempt being refused as ungrounded is exactly where the input-side fact costs the most.
|
||||||
|
offer_notice = grounding_offer_notice(result.grounding_offer)
|
||||||
|
if offer_notice is not None:
|
||||||
|
print(offer_notice)
|
||||||
# Same renderer on the full run, and deliberately so: a run that PRODUCED a proposal from a
|
# Same renderer on the full run, and deliberately so: a run that PRODUCED a proposal from a
|
||||||
# half-read base is where the silence cost the most — the dry run at least produced nothing.
|
# half-read base is where the silence cost the most — the dry run at least produced nothing.
|
||||||
nav_notice = skipped_links_notice(result.skipped_links)
|
nav_notice = skipped_links_notice(result.skipped_links)
|
||||||
|
|
|
||||||
342
tests/test_grounding_offer_loadbearing.py
Normal file
342
tests/test_grounding_offer_loadbearing.py
Normal file
|
|
@ -0,0 +1,342 @@
|
||||||
|
"""P8 — the run SAYS what its delivered input can ground, before it spends an attempt on a proposal
|
||||||
|
that cannot be grounded.
|
||||||
|
|
||||||
|
P7 (økt 109) is right and landed: an identifier a proposal builds on must appear VERBATIM in the
|
||||||
|
input, or the verdict falls. Re-measuring it exposed a CONSEQUENCE no row stated. Over the three
|
||||||
|
free recordings, with the gate live, **29 of 29 cost codes in 13 of 13 delivered proposals are
|
||||||
|
ungrounded** — every one of them invented. That is not a fault in the gate. It is that the PROMPT
|
||||||
|
asks for something the delivered input cannot supply: ``_build_messages`` says each entry "must
|
||||||
|
restate a cost line as the project's price schedule already carries it", and MEASURED, K2's
|
||||||
|
delivered input carries **two** code-shaped tokens in the prompt (``SHA-01``/``SHA-10``), both of
|
||||||
|
them document numbers off a page footer, and ``derive_cost_baseline`` refuses the base outright —
|
||||||
|
there is no cost line in it to restate. A gate that always refuses is as useless as one that never
|
||||||
|
does, so the run must be able to say which of the two situations it is in.
|
||||||
|
|
||||||
|
**This is a REPORT, never a gate.** It does not block: a run with a null offer still runs, because
|
||||||
|
a blocking requirement is exactly ``--require-cost-baseline``, which F4 settled as opt-in and this
|
||||||
|
order freed. It also does not touch ``_ground_against_input``, which ten mutations hold.
|
||||||
|
|
||||||
|
Arms, each with a named detach point:
|
||||||
|
|
||||||
|
* **(a) the null offer is stated.** A delivered text with no identifier of any measured form and no
|
||||||
|
baseline reports ``identifiers=0, cost_lines=0``. This is the K2 CONTROL — without it a green
|
||||||
|
known positive proves nothing.
|
||||||
|
* **(b) the positive offer is stated.** N100's delivered input carries the BINDING known positive
|
||||||
|
``Krav 3.3.1—13`` (P7 premiss (vii): it stands in 6 of 6 prompts, EM-DASH U+2014), and the offer
|
||||||
|
is positive there. Paired with a control that the search CAN find the string at all.
|
||||||
|
* **(c) the offer is measured on the SAME text the gate will see.** ``grounding_offer`` composes
|
||||||
|
through ``generate._grounding_text`` — the one composer P7 uses — so a report built from a second
|
||||||
|
rendering, free to disagree with the one that was sent, is impossible by construction.
|
||||||
|
* **(d) the offer reaches the outcome the operator reads**, on BOTH surfaces: ``RunResult`` after a
|
||||||
|
full run and ``DryRunReport`` before the first model call. The dry-run arm is what proves the
|
||||||
|
measurement happens BEFORE the three attempts are spent — its client factory RAISES.
|
||||||
|
* **(e) ONE renderer**, silent when the run CAN anchor a cost line (omission, never an empty row —
|
||||||
|
``cost_baseline_notice``'s rule), speaking when it cannot.
|
||||||
|
* **(f) a run that exhausts its attempts on ungrounded identifiers leaves the input-side diagnosis
|
||||||
|
in the outcome**, not only "the model missed".
|
||||||
|
|
||||||
|
A NEW file rather than an extension of ``test_identifier_grounding_loadbearing.py``, stated as the
|
||||||
|
order asks: that file's subject is the GATE and it is held by ten mutations. A report mutated here
|
||||||
|
would go red in a file whose docstring promises a falsifier, and which seam a red test names is the
|
||||||
|
only thing a mutation table is for.
|
||||||
|
|
||||||
|
The fixtures are P7's own tracked generation prompts. Tracked test data on purpose (``scratchpad/``
|
||||||
|
is absent from ``git archive HEAD``) — and REUSED rather than extended with the delivered cuts,
|
||||||
|
because those cuts are 10 kB and 96 kB of a real Norwegian tender and a public road standard, and
|
||||||
|
committing corpus text into a repo published on ``open/`` is a publication decision that belongs to
|
||||||
|
the operator, not to this order. The 8-distinct / 435-distinct figures for the delivered cut and the
|
||||||
|
full grounding are MEASURED and reported in ``docs/2026-09-09-p8-forankringstilbudet.md``; what a
|
||||||
|
test asserts here is the property, on text this repo already ships.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from portfolio_optimiser.generate import GroundingOffer, _grounding_text, grounding_offer
|
||||||
|
from portfolio_optimiser.ir import CostBaseline, CostBaselineLine
|
||||||
|
from portfolio_optimiser.reference_domain import CostItem, Project
|
||||||
|
from portfolio_optimiser import run as run_mod
|
||||||
|
from portfolio_optimiser.run import grounding_offer_notice, run_project
|
||||||
|
from portfolio_optimiser.simulation import scripted_factory
|
||||||
|
from portfolio_optimiser.validator import Rejection
|
||||||
|
|
||||||
|
FIXTURES = Path(__file__).parent / "fixtures" / "p7-grounding"
|
||||||
|
SHARED = Path(__file__).resolve().parents[1] / "shared" / "examples"
|
||||||
|
|
||||||
|
#: P7 premiss (vii), the BINDING known positive: the model quoted this requirement number verbatim
|
||||||
|
#: in 6 of 6 replies and it stands in 6 of 6 prompts. EM-DASH (U+2014); the hyphen variant is 0/6.
|
||||||
|
KNOWN_POSITIVE = "Krav 3.3.1—13"
|
||||||
|
|
||||||
|
#: The base with a ``cost-baseline.json`` (one line) and the one without — the pair that makes the
|
||||||
|
#: renderer's omission arm reachable rather than asserted.
|
||||||
|
ANCHORED = SHARED / "veglys-fv-soer"
|
||||||
|
UNANCHORED = SHARED / "bygg-energi-mikro"
|
||||||
|
_UNANCHORED_PID = "BYGG-KONTOR-NORD"
|
||||||
|
_VERDICT_INPUT = {"decision": "approved", "rationale": "expert reviewed (sim)"}
|
||||||
|
_VALID_REPLY = (
|
||||||
|
'{"measure":"LED-retrofit","affected_items":'
|
||||||
|
'[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],'
|
||||||
|
'"claimed_saving_nok":30000}'
|
||||||
|
)
|
||||||
|
#: A code that appears NOWHERE in the base — the ungrounded case, which P7's stage 0b refuses on
|
||||||
|
#: every attempt, so the run exhausts its attempts and returns the last ``Rejection``.
|
||||||
|
_UNGROUNDED_REPLY = (
|
||||||
|
'{"measure":"invented","affected_items":'
|
||||||
|
'[{"code":"M-04-01","quantity":300000,"unit_cost":1.0}],'
|
||||||
|
'"claimed_saving_nok":30000}'
|
||||||
|
)
|
||||||
|
_CHECKER_REPLY = "Reasoning holds.\nVERDICT: APPROVE"
|
||||||
|
|
||||||
|
|
||||||
|
def _prompt(name: str) -> str:
|
||||||
|
return (FIXTURES / name).read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def _project(*codes: str) -> Project:
|
||||||
|
"""A project shaped like the bundle path's: ``_project_from_bundle`` builds ``cost_items=()``
|
||||||
|
(MEASURED, P7's own row), so the default carries no codes of its own and the offer measures the
|
||||||
|
delivered text alone."""
|
||||||
|
return Project(
|
||||||
|
id="X",
|
||||||
|
name="X",
|
||||||
|
description="",
|
||||||
|
currency="NOK",
|
||||||
|
cost_items=tuple(
|
||||||
|
CostItem(code=c, description=c, quantity=1.0, unit="stk", unit_cost=1.0)
|
||||||
|
for c in codes
|
||||||
|
),
|
||||||
|
docs_dir="",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _baseline(*codes: str) -> CostBaseline:
|
||||||
|
return CostBaseline(
|
||||||
|
project_id="X",
|
||||||
|
items={c: CostBaselineLine(quantity=1.0, unit_cost=1.0) for c in codes},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------ (a) the null offer is stated
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_delivered_input_with_no_cost_line_reports_zero() -> None:
|
||||||
|
"""LOAD-BEARING (a) — the K2 CONTROL. Both K2 generation prompts carry no identifier of any
|
||||||
|
measured form and the run is un-anchored, so the offer is null in BOTH numbers. Without this
|
||||||
|
control a green (b) would be satisfied by a report that always counts positive."""
|
||||||
|
for name in ("p6-k2-generation-prompt.txt", "s7c-k2-generation-prompt.txt"):
|
||||||
|
offer = grounding_offer(_project(), None, _prompt(name))
|
||||||
|
assert offer.identifiers == 0, (name, offer)
|
||||||
|
assert offer.cost_lines == 0, (name, offer)
|
||||||
|
assert offer.chars > 0, "the measurement must have had text to measure"
|
||||||
|
|
||||||
|
|
||||||
|
# -------------------------------------------------------------- (b) the positive offer is stated
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_known_positive_input_reports_a_positive_offer() -> None:
|
||||||
|
"""LOAD-BEARING (b) — the BINDING known positive. Paired with the control that the string is
|
||||||
|
actually there: a report that finds nothing because it searched for nothing would otherwise
|
||||||
|
pass (a) and look measured."""
|
||||||
|
text = _prompt("p4-n100-generation-prompt.txt")
|
||||||
|
assert KNOWN_POSITIVE in text, "the fixture no longer carries the known positive"
|
||||||
|
offer = grounding_offer(_project(), None, text)
|
||||||
|
assert offer.identifiers > 0, offer
|
||||||
|
assert offer.cost_lines == 0, "a road standard carries no cost lines (F4's own finding)"
|
||||||
|
|
||||||
|
|
||||||
|
def test_a_bare_number_is_not_counted_as_an_offer() -> None:
|
||||||
|
"""LOAD-BEARING (b), the other half. K2 carries 46 394 bare-number occurrences over 2 117
|
||||||
|
distinct values (P7 § 2), so counting them would make every report positive and the whole
|
||||||
|
measurement inert — the repo's cardinal class, a gate that can only come out green."""
|
||||||
|
offer = grounding_offer(_project(), None, "1234 5678 90 42.5 1000000")
|
||||||
|
assert offer.identifiers == 0, offer
|
||||||
|
|
||||||
|
|
||||||
|
# ----------------------------------------------- (c) measured on the SAME text the gate will see
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_offer_is_measured_on_the_text_the_gate_will_see() -> None:
|
||||||
|
"""LOAD-BEARING (c) — requirement (2). The offer composes through ``_grounding_text``, the one
|
||||||
|
composer P7's gate uses, so the report and the gate cannot describe different texts. Both of
|
||||||
|
the composer's OTHER two sources are exercised: a project cost line and a baseline code each
|
||||||
|
raise the count, which a report built from ``delivered`` alone cannot do."""
|
||||||
|
delivered = "nothing citable here"
|
||||||
|
project, baseline = _project("PRJ-77"), _baseline("BAS-88")
|
||||||
|
|
||||||
|
offer = grounding_offer(project, baseline, delivered)
|
||||||
|
|
||||||
|
assert offer.chars == len(_grounding_text(project, baseline, delivered))
|
||||||
|
assert offer.identifiers == 2, offer
|
||||||
|
assert offer.cost_lines == 1, offer
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------- (e) ONE renderer, omission when able
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_notice_speaks_on_a_null_offer_and_is_silent_when_anchored() -> None:
|
||||||
|
"""LOAD-BEARING (e). Silent when the run CAN anchor a cost line (omission, never an empty row);
|
||||||
|
speaking when it cannot, and carrying BOTH numbers — "50 identifiers, 0 cost lines" is the
|
||||||
|
diagnosis, and neither number alone says it."""
|
||||||
|
assert grounding_offer_notice(GroundingOffer(chars=9, identifiers=3, cost_lines=1)) is None
|
||||||
|
assert grounding_offer_notice(None) is None
|
||||||
|
|
||||||
|
line = grounding_offer_notice(GroundingOffer(chars=9, identifiers=3, cost_lines=0))
|
||||||
|
assert line is not None
|
||||||
|
assert "3" in line and "0" in line
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------- (d) the offer reaches BOTH surfaces the operator reads
|
||||||
|
|
||||||
|
|
||||||
|
async def test_the_dry_run_carries_the_offer_before_any_model_call() -> None:
|
||||||
|
"""LOAD-BEARING (d), and the proof of "BEFORE it spends its three attempts".
|
||||||
|
|
||||||
|
Asserted on CALLS, never on client CONSTRUCTION: ``fresh_workflow`` builds the proposer and
|
||||||
|
checker clients EAGERLY, above the dry-run cut (``workflow.py:64``), so a factory that raised
|
||||||
|
would be red against a working implementation. An empty sink is the measurement — session 57's
|
||||||
|
rule that a refusal after the spend is indistinguishable from one before it at the exit code.
|
||||||
|
"""
|
||||||
|
sink: list[str] = []
|
||||||
|
|
||||||
|
report = await run_project(
|
||||||
|
_UNANCHORED_PID,
|
||||||
|
"local",
|
||||||
|
docs_dir=str(UNANCHORED),
|
||||||
|
bundle_dir=str(UNANCHORED),
|
||||||
|
client_factory=scripted_factory({"proposer": "x", "checker": "x"}, sink),
|
||||||
|
live_dry_run=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert sink == [], "the offer was measured only after a model call was made"
|
||||||
|
|
||||||
|
offer = getattr(report, "grounding_offer", None)
|
||||||
|
assert offer is not None, "the dry run says nothing about what its input can ground"
|
||||||
|
assert offer.cost_lines == 0, offer
|
||||||
|
assert offer.chars > 0, offer
|
||||||
|
|
||||||
|
|
||||||
|
async def test_the_full_run_carries_the_offer() -> None:
|
||||||
|
"""LOAD-BEARING (d), the other surface — requirement (3). A measurement that never leaves
|
||||||
|
``run_project`` is a log line, not a report."""
|
||||||
|
result = await run_project(
|
||||||
|
_UNANCHORED_PID,
|
||||||
|
"local",
|
||||||
|
docs_dir=str(UNANCHORED),
|
||||||
|
bundle_dir=str(UNANCHORED),
|
||||||
|
verdict_input=_VERDICT_INPUT,
|
||||||
|
client_factory=scripted_factory(
|
||||||
|
{"proposer": _VALID_REPLY, "checker": _CHECKER_REPLY}, []
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
offer = getattr(result, "grounding_offer", None)
|
||||||
|
assert offer is not None, "the run says nothing about what its input can ground"
|
||||||
|
assert offer.cost_lines == 0, offer
|
||||||
|
|
||||||
|
|
||||||
|
async def test_an_anchored_run_reports_its_cost_lines() -> None:
|
||||||
|
"""LOAD-BEARING (d), the CONTROL that ``cost_lines`` is not a constant zero: the anchored base
|
||||||
|
ships exactly one line, and the run reports it."""
|
||||||
|
report = await run_project(
|
||||||
|
"VEGLYS-FV-SOER",
|
||||||
|
"local",
|
||||||
|
docs_dir=str(ANCHORED),
|
||||||
|
bundle_dir=str(ANCHORED),
|
||||||
|
client_factory=scripted_factory({"proposer": "x", "checker": "x"}, []),
|
||||||
|
live_dry_run=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
offer = getattr(report, "grounding_offer", None)
|
||||||
|
assert offer is not None and offer.cost_lines == 1, offer
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------- (f) an exhausted run leaves the INPUT-side diagnosis behind
|
||||||
|
|
||||||
|
|
||||||
|
async def test_a_run_exhausted_on_ungrounded_identifiers_says_the_input_was_the_problem() -> None:
|
||||||
|
"""LOAD-BEARING (f). P7's gate refuses every attempt, the loop returns its last ``Rejection``
|
||||||
|
(``generate.py``: ``last_ruling`` after the attempt loop — it does not crash), and the outcome
|
||||||
|
the operator reads carries the input-side fact as well: this input could anchor NOTHING, so no
|
||||||
|
attempt could ever have succeeded. Without it the record says only that the model missed."""
|
||||||
|
result = await run_project(
|
||||||
|
_UNANCHORED_PID,
|
||||||
|
"local",
|
||||||
|
docs_dir=str(UNANCHORED),
|
||||||
|
bundle_dir=str(UNANCHORED),
|
||||||
|
verdict_input=_VERDICT_INPUT,
|
||||||
|
client_factory=scripted_factory(
|
||||||
|
{"proposer": _UNGROUNDED_REPLY, "checker": _CHECKER_REPLY}, []
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert isinstance(result.outcome, Rejection)
|
||||||
|
assert "M-04-01" in result.outcome.reason
|
||||||
|
offer = getattr(result, "grounding_offer", None)
|
||||||
|
assert offer is not None and offer.cost_lines == 0, offer
|
||||||
|
assert grounding_offer_notice(offer) is not None
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------ (d) the CLI prints it, on BOTH free and paid
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_cli_dry_run_prints_the_offer(capsys) -> None:
|
||||||
|
"""LOAD-BEARING (d), stdout. A renderer that returns the line while no caller prints it is a
|
||||||
|
measurement the operator never sees — its own seam, so its own arm (the ``cost_baseline_notice``
|
||||||
|
precedent, whose CLI print carries a mutation of its own)."""
|
||||||
|
rc = run_mod.main(
|
||||||
|
[
|
||||||
|
_UNANCHORED_PID,
|
||||||
|
"--docs-dir",
|
||||||
|
str(UNANCHORED),
|
||||||
|
"--bundle-dir",
|
||||||
|
str(UNANCHORED),
|
||||||
|
"--live-dry-run",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert rc == 0
|
||||||
|
assert "Grounding offer" in capsys.readouterr().out
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_cli_says_nothing_when_the_run_can_anchor(capsys) -> None:
|
||||||
|
"""CONTROL for the omission arm on the SAME surface: the anchored base prints no offer line at
|
||||||
|
all. Without it, a notice that always fired would pass the arm above."""
|
||||||
|
rc = run_mod.main(
|
||||||
|
[
|
||||||
|
"VEGLYS-FV-SOER",
|
||||||
|
"--docs-dir",
|
||||||
|
str(ANCHORED),
|
||||||
|
"--bundle-dir",
|
||||||
|
str(ANCHORED),
|
||||||
|
"--live-dry-run",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert rc == 0
|
||||||
|
assert "Grounding offer" not in capsys.readouterr().out
|
||||||
|
|
||||||
|
|
||||||
|
def test_the_cli_full_run_prints_the_offer(tmp_path: Path, capsys) -> None:
|
||||||
|
"""LOAD-BEARING (d), the paid surface's stdout — so the line is a property of a RUN and not of
|
||||||
|
the dry-run branch alone."""
|
||||||
|
replies = tmp_path / "replies.json"
|
||||||
|
replies.write_text(
|
||||||
|
json.dumps({"proposer": _VALID_REPLY, "checker": _CHECKER_REPLY}), encoding="utf-8"
|
||||||
|
)
|
||||||
|
rc = run_mod.main(
|
||||||
|
[
|
||||||
|
_UNANCHORED_PID,
|
||||||
|
"--docs-dir",
|
||||||
|
str(UNANCHORED),
|
||||||
|
"--bundle-dir",
|
||||||
|
str(UNANCHORED),
|
||||||
|
"--scripted-replies",
|
||||||
|
str(replies),
|
||||||
|
"--decision",
|
||||||
|
"approved",
|
||||||
|
"--rationale",
|
||||||
|
"expert reviewed (test)",
|
||||||
|
]
|
||||||
|
)
|
||||||
|
assert rc == 0
|
||||||
|
assert "Grounding offer" in capsys.readouterr().out
|
||||||
Loading…
Add table
Add a link
Reference in a new issue