feat(run,generate): a run says what its delivered input can ground, before it spends an attempt [skip-docs]
P7 is right and landed, but re-measuring it exposed a consequence no row stated: with the gate live, 29 of 29 cost codes in 13 of 13 delivered proposals fall across the three free recordings (PM's denominator; 14 of 14 in 8 proposals on the PARSEABLE one -- the five blobs that separate the numbers are refused by pydantic's `claimed <= total` and never reach stage 0b). All 29 were invented, so the gate is right; but a gate that always refuses is as useless as one that never does. The cause is that the PROMPT asks for something the input cannot supply. `_build_messages` requires each affected_item to "restate a cost line as the project's price schedule already carries it", while K2's delivered input carries 9 occurrences / 2 distinct code-shaped tokens -- `SHA-01`/`SHA-10`, both document numbers off a page footer -- and `derive_cost_baseline` refuses the base outright. There is no cost line in it to restate. `GroundingOffer(chars, identifiers, cost_lines)` reports it. The PAIR is the diagnosis: "50 identifiers, 0 cost lines" says what neither number says alone. A REPORT, never a gate -- it blocks nothing, because a blocking requirement IS `--require-cost-baseline` (F4/D-3, opt-in, untouched), and `_ground_against_input` is untouched. The callsite is MEASURED, not chosen: `generate.py` composes the grounding per attempt, after `await _fetch_parsed`, so a report there could only speak once an attempt had been paid for; `run.py` binds both halves above the `--live-dry-run` cut and before the first `debate.run`, so the FREE trip says it. `delivered` is bound ONCE and the same variable feeds the report and `_evaluate`; the report composes THROUGH `_grounding_text`, the gate's own composer. A pattern is admissible here and not in the gate, and that is the difference between a report and a falsifier: an unknown form is a token left uncounted -- an under-count, never a false rejection. The forms are transcribed from the measurement; bare numbers are excluded with the number (46 394 / 2 117 in K2). `grounding_offer_notice` is the ONE renderer and is silent when the run CAN anchor -- omission, never an empty row. Load-bearing MEASURED (tests/test_grounding_offer_loadbearing.py, 12 arms), nine mutations all red against the WHOLE suite + green control 1570/5 (from 1558/5, strict superset, 0 removed) and the golden byte-unchanged (shasum -a 1 of the CONTENT = ea8c534773acdbe41ae68f2c55724d69aaf8be4f). Measurement: docs/2026-09-09-p8-forankringstilbudet.md Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
277bb95777
commit
455d611660
5 changed files with 767 additions and 2 deletions
43
CLAUDE.md
43
CLAUDE.md
|
|
@ -2316,6 +2316,49 @@ Python ≥3.10. MAF (`agent-framework-core` 1.16.0, `-orchestrations` 1.1.1 —
|
|||
den hostede flaten er urørt (feltet er i ingen av hostings tre sett); og
|
||||
`--require-cost-baseline` er IKKE gjort til default (F4/D-3). Måling:
|
||||
`docs/2026-09-09-p7-forankrede-identifikatorer.md`.
|
||||
- **En kjøring SIER hva den leverte inputen kan forankre, før den bruker et forsøk på et forslag
|
||||
som ikke kan bli forankret (P8, 09.09):** P7 er riktig og landet — men re-målingen avdekket en
|
||||
konsekvens ingen rad uttalte: med gaten live faller **29 av 29 koder i 13 av 13 leverte forslag**
|
||||
over de tre gratis-opptakene (PMs nevner; på den PARSEBARE nevneren 14 av 14 i 8 forslag — de
|
||||
fem blobene som skiller tallene avvises av pydantics `claimed <= total` og når aldri stadium 0b).
|
||||
Alle 29 VAR oppdiktet, så gaten er riktig; men en gate som alltid nekter er like ubrukelig som en
|
||||
som aldri gjør det. **Årsaken er at PROMPTEN ber om noe inputen ikke kan levere:**
|
||||
`_build_messages` krever at hver `affected_item` «restate a cost line as the project's price
|
||||
schedule already carries it», mens K2s LEVERTE input bærer **9 forekomster / 2 distinkte**
|
||||
kodeformede tokens — `SHA-01`/`SHA-10`, begge dokumentnumre fra en sidefot — og
|
||||
`derive_cost_baseline` nekter basen: det finnes ingen kostlinje å gjengi. `GroundingOffer(chars,
|
||||
identifiers, cost_lines)` er svaret, og **PARET er diagnosen**: «50 identifikatorer, 0
|
||||
kostlinjer» sier det ingen av tallene sier alene. **En RAPPORT, aldri en gate** — den blokkerer
|
||||
ikke, fordi et blokkerende krav ER `--require-cost-baseline` (F4/D-3, opt-in, urørt), og
|
||||
`_ground_against_input` er URØRT. **Kallstedet er MÅLT, ikke valgt:** `generate.py` komponerer
|
||||
grunnlaget PER FORSØK, etter `await _fetch_parsed`, så en rapport derfra kan først tale når ett
|
||||
forsøk er betalt; `run.py` binder begge halvdeler over `--live-dry-run`-kuttet og før første
|
||||
`debate.run`, så den FRIE turen sier det. `delivered` bindes ÉN gang og gir samme variabel til
|
||||
rapporten og til `_evaluate` (to komposisjoner av én tekst er kø-(p)), og rapporten komponerer
|
||||
GJENNOM `_grounding_text` — samme funksjon gaten bruker. **Et mønster er tillatt HER og ikke i
|
||||
gaten**, og det er skillet mellom rapport og gate: en ukjent form er et token som ikke telles,
|
||||
altså under-telling, aldri falsk avvisning. Formene er TRANSKRIBERT fra målingen (K2: 50
|
||||
distinkte kodeformede, 0 kravnumre; N-korpusene: 269–981 distinkte kravnumre, ≤3 kodeformede);
|
||||
**rene tall er BEVISST utelatt med tallet** (46 394 / 2 117 i K2 — å telle dem gjør hver rapport
|
||||
positiv og målingen inert). `grounding_offer_notice` er ENESTE renderer og tier når kjøringen KAN
|
||||
forankre — omisjon, aldri tom rad. Load-bearing MÅLT
|
||||
(`tests/test_grounding_offer_loadbearing.py`, 12 armer), **ni mutasjoner alle røde mot HELE
|
||||
suiten** + grønn kontroll **1570/5** (fra 1558/5, supersett, 0 fjernet) og golden BYTE-UENDRET
|
||||
(`shasum -a 1` av INNHOLDET = `ea8c534773acdbe41ae68f2c55724d69aaf8be4f`): M1 teller alltid null
|
||||
(2, kjent-positiven blant dem) · M2 teller alltid positivt (7, K2-kontrollen) · M3 bygg fra
|
||||
`delivered` alene (1) · M4 når aldri `RunResult` (3) · M5 når aldri `DryRunReport` (3) ·
|
||||
M6 rendereren skriver alltid linja (2) · M7 detach dry-run-utskriften (1) · M8 detach
|
||||
fullkjørings-utskriften (1) · M9 rene tall telles (2). **Ærlighets-grenser, uttalt:** ingen betalt
|
||||
kjøring bekrefter at rapporten endrer noe levende (tre opptak, ÉN modell, ETT deployment); den
|
||||
BLOKKERER ikke, så en kjøring med null tilbud kan fortsatt brenne tre forsøk — VALGT, ikke
|
||||
oversett; `cost_lines` er `len(baseline.items)`, altså forankringen som ANTALL, og står her fordi
|
||||
paret er diagnosen; testfixturene er P7s SPORede prompter, ikke de leverte kuttene (10 kB / 96 kB
|
||||
verbatim anbuds- og standardtekst inn i et repo publisert på `open/` er operatørens
|
||||
publiseringsbeslutning, ikke denne ordrens) — 8-/435-distinkt-tallene er MÅLT og står i
|
||||
dokumentet; portefølje-armen og den hostede flaten er BEVISST urørt; og tilbudsmålingen er gjort
|
||||
på okf sin GAMLE default-bundle (`K2-bundle-20260903`), så en re-måling på den nye (436 konsepter
|
||||
/ 832 filer) er en senere, separat ordre. Måling:
|
||||
`docs/2026-09-09-p8-forankringstilbudet.md`.
|
||||
- **STATE.md er local-only** (gitignored). Voyage session-state er efemert; STATE.md er kanonisk kontinuitet.
|
||||
- Prosess: Voyage-plugin (`/trekbrief → /trekplan → /trekexecute → /trekreview`) per større fase.
|
||||
|
||||
|
|
|
|||
227
docs/2026-09-09-p8-forankringstilbudet.md
Normal file
227
docs/2026-09-09-p8-forankringstilbudet.md
Normal file
|
|
@ -0,0 +1,227 @@
|
|||
# P8 — den leverte inputen bærer ingen kostlinjer, så ingen forankret proposal kan oppstå
|
||||
|
||||
Ordre `20260909T142951Z-365478643-from-.claude`, økt 110. HEAD ved start: `277bb95`.
|
||||
Alt i dette dokumentet er målt **gratis** — null modellkall, **NOK 0,00**.
|
||||
|
||||
## § 0 Hva som ER målt og hva som IKKE er det
|
||||
|
||||
**Målt:** hva de tre gratis opptakene faktisk leverte og hvor mange av kodene P7-gaten feller;
|
||||
hva den LEVERTE inputen i hvert korpus po har liggende kan tilby som lovlig `affected_item`-kode;
|
||||
og at rapporten som nå bygges sier null der tilbudet er null og positivt der det er positivt.
|
||||
|
||||
**Ikke målt:** at rapporten endrer noe LEVENDE. Ingen betalt kjøring er gjort, og ingen er
|
||||
bestilt. At en modell som får se linja oppfører seg annerledes er ikke vist — samme klasse som
|
||||
structured-output-grensen. Rapporten er dessuten skrevet på grunnlag av tre opptak fra **ÉN**
|
||||
modell og **ETT** deployment.
|
||||
|
||||
**Ikke bygget, med vilje:** prosa-skanningen (operatørbeslutning A) og
|
||||
`--require-cost-baseline` som default (operatørbeslutning, F4/D-3). Begge står uendret.
|
||||
|
||||
## § 1 Premissene (i)–(viii) — hver verifisert selv
|
||||
|
||||
| # | Premiss (PM) | Mitt utfall |
|
||||
|---|---|---|
|
||||
| (i) | HEAD `277bb95`, `git ls-remote origin main` = `eb41374`, upushet 2 | **BEKREFTET** ordrett |
|
||||
| (ii) | To utrackede: presentasjons-HTML (parallell økt) + `scratchpad/` | **BEKREFTET**, intet tredje |
|
||||
| (iii) | 1558 passed / 5 skipped, ruff rent, mypy rent (37 filer), golden `ea8c534…` | **BEKREFTET** alle fire |
|
||||
| (iv) | 13 forslag / 29 koder / 29 ugrunnet | **BEKREFTET på PMs nevner, med et avvik — se § 2** |
|
||||
| (v) | K2 leverer 9/2 distinkte kodeformede (`SHA-01`/`SHA-10`, dokumentnumre); N100 0 kodeformede, 17/8 kravnumre | **BEKREFTET ordrett** |
|
||||
| (vi) | Ved uttømte forsøk returneres siste `Rejection`, ingen krasj | **BEKREFTET**: `generate.py` returnerer `GenerationResult(outcome=last_ruling, …)` etter forsøksløkka |
|
||||
| (vii) | `--require-cost-baseline` er opt-in og skal ikke bli default | **RESPEKTERT**, urørt |
|
||||
| (viii) | (A) og (B) er operatørens | **RESPEKTERT**, ikke avgjort her |
|
||||
|
||||
## § 2 Konsekvensen — og AVVIKET mot PMs tall
|
||||
|
||||
PMs tall er reprodusert nøyaktig **på PMs nevner**, men den nevneren teller blobs som aldri
|
||||
NÅR P7-gaten. Begge tall er sanne om hver sin ting, og begge gir 100 %:
|
||||
|
||||
| Opptak | Rå JSON-blobs | Koder | Ugrunnet | Parsebare til IR | Koder | Ugrunnet |
|
||||
|---|---|---|---|---|---|---|
|
||||
| `p6-Aopen-records.json` | 2 | 4 | **4** | 2 | 4 | **4** |
|
||||
| `Aopen-records.json` | 8 | 22 | **22** | 3 | 7 | **7** |
|
||||
| `p4-n100-free-records.json` | 3 | 3 | **3** | 3 | 3 | **3** |
|
||||
| **Totalt** | **13** | **29** | **29 (100 %)** | **8** | **14** | **14 (100 %)** |
|
||||
|
||||
De fem blobene som skiller tallene avvises av en **annen og TIDLIGERE falsifiserer** —
|
||||
pydantics `claimed_saving_nok <= affected items' total` — og når derfor aldri stadium 0b.
|
||||
Det er ikke en svakhet i PMs måling; det er to nevnere om to ulike hendelser. Grunnlaget er
|
||||
det **mest sjenerøse** som finnes: unionen av ALLE prompter i opptaket. Selv der er ingen kode
|
||||
grunnet.
|
||||
|
||||
Et instrument-forbehold, skrevet ned fordi det først ga feil svar: en rå
|
||||
`SavingsProposal.model_validate_json` avviser **alle 13**, fordi svarene bærer WIRE-formen av
|
||||
`assumptions` (et array) som `_normalise_assumptions` folder tilbake. En måling som stoppet der
|
||||
ville rapportert 0 forslag — null fordi instrumentet ikke kunne lese formen, ikke fordi formen
|
||||
manglet.
|
||||
|
||||
## § 3 Årsaken — prompten ber om noe inputen ikke kan levere
|
||||
|
||||
`generate._build_messages` sier ordrett: «Each entry in `affected_items` must restate a cost line
|
||||
as the project's price schedule already carries it». Målt på record 0 (turen som bar bundelen):
|
||||
|
||||
* **K2** (P6: 96 567 tegn; S7c: 145 281 tegn) → **9 forekomster / 2 distinkte** kodeformede, og
|
||||
begge er `SHA-01` / `SHA-10`. Konteksten er entydig: `Oppdragsnr.: 52308329 Dokumentnr.: SHA-01
|
||||
Versjon: 01` — en sidefot i en SHA-plan — og `Dokumentnavn: SHA-10 - … Restrisikorapport`.
|
||||
**Dokumentnumre, ikke kostlinjer.**
|
||||
* **N100** (10 569 tegn) → **0** kodeformede, men **17 forekomster / 8 distinkte** kravnumre på
|
||||
`Krav X.Y.Z—N`-form (em-dash U+2014).
|
||||
|
||||
Modellen ble bedt om å gjengi en kostlinje som ikke finnes, og fabrikkerte den. Dette rimer med
|
||||
funn 4 (økt 107): K2s prisskjema bærer ingen mengdefortegnelse, 0 av 92 rader navngir
|
||||
kode/mengde/enhetspris.
|
||||
|
||||
## § 4 Tilbudet per korpus — målt med den SHIPPEDE funksjonen
|
||||
|
||||
Teksten er komponert nøyaktig som `run_project` komponerer P7s grunnlag
|
||||
(`"\n".join([context, bundle_grounding])`, deretter gjennom `generate._grounding_text`).
|
||||
«Kostlinje» er po sin EGEN definisjon (`okf.derive_cost_baseline`, MAJOR-4) — ikke en ny
|
||||
heuristikk oppfunnet her.
|
||||
|
||||
| Korpus | Konsepter | Grunnlag (tegn) | Distinkte identifikatorer | Kostlinjer | `derive_cost_baseline` |
|
||||
|---|---|---|---|---|---|
|
||||
| K2 s7c + `payload-open` | 630 | 1 991 597 | 50 | **0** | REFUSED |
|
||||
| K2 s7c + `payload-default` | 630 | 1 970 415 | 50 | **0** | REFUSED |
|
||||
| K2 s7a2 (peker-armen, P6) | 629 | 1 894 500 | 50 | **0** | REFUSED |
|
||||
| N100 `n100-2023` | 446 | 462 041 | 435 | **0** | REFUSED |
|
||||
| N200 `n200-2024` | 1 133 | 1 500 962 | 982 | **0** | REFUSED |
|
||||
| N500 `n500-2024` | 270 | 408 220 | 272 | **0** | REFUSED |
|
||||
| `shared/veglys-fv-soer` | 5 | 30 038 | 3 | **1** | REFUSED (men fila finnes) |
|
||||
| `shared/tunnel-hauglia` | 5 | 36 546 | 2 | **1** | REFUSED (men fila finnes) |
|
||||
| `shared/bygg-energi-mikro` | 4 | 11 019 | 2 | **0** | REFUSED |
|
||||
|
||||
**Klassene, målt.** K2s 50 distinkte kodeformede er tegningsnumre (`B-20-00-00`, `F-20-00-01`,
|
||||
`V-73-20-01-01`), dokumentnumre (`SHA-01`, `RIM-02`, `RIA-01`, `NOT-01`), stoff- og
|
||||
standardreferanser (`PCB-7`, `PAH-16`, `DALI-2`) og forskriftsnumre
|
||||
(`FOR-2011-12-06-1357`). **Ingen av dem er en kostkode.** N-korpusene bærer kravnumre og UUID-er
|
||||
(N200: 2 290 distinkte UUID-er), ingen kostkoder.
|
||||
|
||||
**Er et forankret forslag i det hele tatt MULIG?** For **K2: NEI**, og tallet er 0 —
|
||||
`derive_cost_baseline` nekter basen, og ingen av de 50 identifikatorene er en kostlinje. Bare
|
||||
**2 av 50** når i det hele tatt prompten. For **N-korpusene: NEI** på kostlinje — en vegnormal
|
||||
bærer ingen — men **JA** på identifikator: 435 / 982 / 272 distinkte kravnumre er sitérbare, og
|
||||
8 av dem sto i N100-prompten.
|
||||
|
||||
**Rene tall er farligst og bæres videre uendret:** 46 394 forekomster / 2 117 distinkte i K2
|
||||
(P7 § 2). Regelen er inert mot dem og feiler ÅPENT. Ikke bygget om på.
|
||||
|
||||
**Gammel default.** Alt over er målt på `K2-bundle-20260903`. okf melder (innboks
|
||||
`20260909T134141Z`, `reply-expected: no`, lukket med `coord-done`) at deres default-bygg sluttet å
|
||||
emittere den 2026-09-08, at gjeldende default er 436 konsepter / 832 filer, og at de leser
|
||||
formfunnet som fortsatt stående fordi endringen treffer FORMEN, ikke innholdet. Ingen ny bundle er
|
||||
bygget eller konsumert her — deres endring er ikke pushet, og en re-måling på den nye defaulten er
|
||||
en senere, separat ordre.
|
||||
|
||||
## § 5 Hva som er bygget — en RAPPORT, ikke en gate
|
||||
|
||||
`generate.GroundingOffer(chars, identifiers, cost_lines)` + `generate.grounding_offer(...)`,
|
||||
kalt fra `run.py`, rendret av `run.grounding_offer_notice`.
|
||||
|
||||
**Valget av kallsted, med grunnen (ordrens eget krav).** Begge kandidater ble lest først.
|
||||
`generate.py` komponerer grunnlaget **per forsøk, ETTER `await _fetch_parsed(messages)`** — en
|
||||
rapport derfra kan først tale når ett forsøk allerede er betalt, altså nøyaktig det ordren ber
|
||||
den om å komme foran. `run.py` binder begge halvdeler ved `run.py:1228`, **over
|
||||
`--live-dry-run`-kuttet og før første `debate.run`**. Valgt: **`run.py`**. Tellefunksjonen bor
|
||||
likevel i `generate.py`, ved siden av `_grounding_text` den måler — å skille dem ville gitt to
|
||||
steder å bli uenige på.
|
||||
|
||||
**De fire kravene:**
|
||||
|
||||
1. **BLOKKERER ikke.** En kjøring med null tilbud kjører som før. Et blokkerende krav ER
|
||||
`--require-cost-baseline`, som premiss (vii) fredet.
|
||||
2. **Måler den EKSAKTE teksten.** `grounding_offer` komponerer GJENNOM `_grounding_text` —
|
||||
samme funksjon P7s gate bruker — og `run.py` binder `delivered` **én gang** og gir samme
|
||||
variabel til både rapporten og `_evaluate`. To komposisjoner av én tekst er fri til å være
|
||||
uenige (kø-(p)); her er de identiske ved konstruksjon.
|
||||
3. **Når utfallet operatøren leser.** `RunResult.grounding_offer`, `DryRunReport.grounding_offer`
|
||||
og én linje på stdout i begge CLI-armene.
|
||||
4. **Gjenbruker P7s sømmer.** Ingen ny domstype ved siden av `Rejection`/`ValidatedProposal`.
|
||||
`_ground_against_input` er **URØRT**.
|
||||
|
||||
**Identifikator-formene er TRANSKRIBERT fra målingen i § 4**, ikke valgt: `[A-ZÆØÅ]{1,8}[-_]\d…`
|
||||
(K2s 50) og `Krav X.Y.Z—N` (N-korpusenes dominerende form). At et mønster er tillatt HER og ikke i
|
||||
`_ground_against_input` er selve skillet mellom en rapport og en gate: en form rapporten ikke
|
||||
kjenner er et token den unnlater å telle, altså en **under-telling** — aldri en falsk avvisning.
|
||||
**Rene tall er BEVISST utelatt, med tallet** (46 394 / 2 117): å telle dem ville gjort hver rapport
|
||||
positiv og målingen inert — repoets kardinalklasse, en gate som bare kan bli grønn.
|
||||
|
||||
Rendereren er **ÉN**, og den tier når kjøringen KAN forankre en kostlinje — omisjon, aldri en tom
|
||||
rad (`cost_baseline_notice`s regel). Den bærer **begge tall**, fordi paret er diagnosen:
|
||||
«0 kostlinjer» alene leses som en gjentakelse av `cost_baseline_notice`, «50 identifikatorer» alene
|
||||
leses som gode nyheter.
|
||||
|
||||
## § 6 Kjent-positiv og kontroll
|
||||
|
||||
**Kjent-positiven (bindende):** N100s input bærer `Krav 3.3.1—13` (em-dash U+2014; P7 premiss
|
||||
(vii): 6 av 6 prompter). Rapporten sier **positivt tilbud** der, og armen er paret med en kontroll
|
||||
på at strengen faktisk STÅR i fixturen — en rapport som fant null fordi den lette etter ingenting
|
||||
ville ellers bestått.
|
||||
|
||||
**Kontrollen på samme materiale:** K2s to genererings-prompter, der tilbudet er **null i begge
|
||||
tall**. Uten den beviser en grønn kjent-positiv ingenting.
|
||||
|
||||
**Fixturvalget, uttalt.** Testene leser P7s egne SPORede fixturer under
|
||||
`tests/fixtures/p7-grounding/`. De leverte KUTTENE (10 kB N100, 96 kB K2) er ikke sporet: å
|
||||
committe verbatim anbudstekst og standardtekst inn i et repo som publiseres på `open/` er en
|
||||
publiseringsbeslutning som tilhører operatøren, ikke denne ordren. 8-/435-distinkt-tallene står
|
||||
derfor i § 4 som MÅLINGER; det testene asserterer er EGENSKAPEN, på tekst repoet allerede
|
||||
shipper.
|
||||
|
||||
## § 7 Mutasjonstabellen
|
||||
|
||||
Grønn kontroll **1570 passed / 5 skipped** (fra 1558/5; **+12 node-ider, 0 fjernet**, målt med
|
||||
`comm` mot en liste bygget fra HEAD). Golden `demo-transcript.stdout` BYTE-UENDRET,
|
||||
`shasum -a 1` av INNHOLDET = `ea8c534773acdbe41ae68f2c55724d69aaf8be4f` (ALDRI git-blob-id-en).
|
||||
Hver mutasjon kjørt mot HELE suiten, én per kall, restaurert fra `scratchpad/p8/mut-backup/`
|
||||
med `shasum -c`, aldri `git checkout`.
|
||||
|
||||
| # | Mutasjon | Røde |
|
||||
|---|---|---|
|
||||
| M1 | rapporten teller alltid null | **2** (kjent-positiven blant dem) |
|
||||
| M2 | rapporten teller alltid positivt | **7** (K2-kontrollen først) |
|
||||
| M3 | rapporten bygges fra `delivered` alene, ikke gjennom `_grounding_text` | **1** (krav (2)-armen alene) |
|
||||
| M4 | rapporten når aldri `RunResult` | **3** |
|
||||
| M5 | rapporten når aldri `DryRunReport` | **3** |
|
||||
| M6 | rendereren skriver alltid linja | **2** (omisjonen er selv gatet, på begge flater) |
|
||||
| M7 | detach dry-run-utskriften | **1** |
|
||||
| M8 | detach fullkjørings-utskriften | **1** |
|
||||
| M9 | rene tall telles som tilbud | **2** |
|
||||
|
||||
Ni mutasjoner, **alle røde**. Ingen grønn mutasjon, altså ingen uvitnet søm i denne leveransen.
|
||||
|
||||
## § 8 Honesty limits
|
||||
|
||||
* **Alt er målt OFFLINE** på tre opptak fra **ÉN modell** og **ETT deployment**. Ingen betalt
|
||||
kjøring bekrefter at rapporten endrer noe levende, og at en modell som ser linja velger
|
||||
annerledes er **ikke vist** (structured-output-grensens klasse).
|
||||
* **Rapporten BLOKKERER ikke.** En kjøring med null tilbud kan fortsatt brenne tre forsøk på et
|
||||
forslag som ikke kan bli forankret. Det er **VALGT**, ikke oversett: et blokkerende krav er
|
||||
`--require-cost-baseline`, og F4/D-3 la den beslutningen hos operatøren.
|
||||
* **Identifikator-formene er et mønster**, og et mønster kan mangle en form. Retningen er
|
||||
under-telling, aldri falsk avvisning — men et korpus med en tredje form vil rapportere lavere
|
||||
enn det tilbyr, til noen måler den formen.
|
||||
* **`cost_lines` er `len(baseline.items)`**, lest av SAMME `baseline`-binding `_grounding_text`
|
||||
tar som sin tredje kilde. Den gjentar altså forankringen som et ANTALL. Den står her fordi
|
||||
paret er diagnosen, ikke fordi antallet er en ny kjensgjerning.
|
||||
* **Portefølje-armen er BEVISST ikke wiret.** `run_portfolio` skriver ingen slik linje;
|
||||
`bundle_id_notice`s avgjørelse, ikke `cost_baseline_notice`s. Asymmetrien står her fordi
|
||||
stillhet om den er det eneste gale svaret.
|
||||
* **Den hostede flaten er urørt.** Feltet er i ingen av hostings tre sett, så Fase 4es to
|
||||
halvdeler står uendret.
|
||||
* **Tilbudsmålingen er gjort på okf sin GAMLE default-bundle** (`K2-bundle-20260903`). En
|
||||
re-måling på den nye (436 konsepter / 832 filer) er en senere, separat ordre.
|
||||
* **(A) prosa-skanningen og (B) blindsone-valget forblir OPERATØRENS.** Målingen her informerer
|
||||
(B) — den handler om **TILBUDET** i inputen, ikke om hvilket utdrag modellen **VELGER** — men
|
||||
den avgjør den ikke.
|
||||
|
||||
## § 9 Reproduksjon
|
||||
|
||||
uv run pytest -q tests/test_grounding_offer_loadbearing.py
|
||||
uv run python -m portfolio_optimiser.run BYGG-KONTOR-NORD \
|
||||
--docs-dir shared/examples/bygg-energi-mikro \
|
||||
--bundle-dir shared/examples/bygg-energi-mikro --live-dry-run
|
||||
uv run python -m portfolio_optimiser.run VEGLYS-FV-SOER \
|
||||
--docs-dir shared/examples/veglys-fv-soer \
|
||||
--bundle-dir shared/examples/veglys-fv-soer --live-dry-run
|
||||
|
||||
Den første basen har ingen kostbaseline og skriver tilbudslinja; den andre har én og tier.
|
||||
Måleskriptene ligger i `scratchpad/p8/` (utracket).
|
||||
|
|
@ -22,6 +22,7 @@ Two entry points, because the LLM call is async while ``validator.self_repair``
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Callable, Mapping
|
||||
from dataclasses import dataclass, field, replace
|
||||
from typing import Any
|
||||
|
|
@ -456,6 +457,79 @@ def _grounding_text(project: Project, baseline: CostBaseline | None, delivered:
|
|||
)
|
||||
|
||||
|
||||
#: The identifier forms the DELIVERED corpora actually carry, TRANSCRIBED from the measurement
|
||||
#: (``docs/2026-09-09-p8-forankringstilbudet.md`` § 2) rather than chosen: over K2's two delivered
|
||||
#: bundles the first form finds 50 distinct tokens and the second none, while over the three
|
||||
#: N payloads the second finds 269-981 distinct and the first at most 3.
|
||||
#:
|
||||
#: **Why a pattern is admissible HERE and not in ``_ground_against_input``.** That is a GATE, and a
|
||||
#: pattern there would be a rule about shapes that can wrongly REFUSE a real code. This is a
|
||||
#: REPORT: a form it does not know is a token it fails to count, so it errs toward saying the input
|
||||
#: offers LESS than it does — an under-count is a quiet report, never a false rejection.
|
||||
#:
|
||||
#: Bare numbers are deliberately EXCLUDED, with the number: K2 carries 46 394 occurrences over
|
||||
#: 2 117 distinct values (P7 § 2), so counting them would make every report positive and the
|
||||
#: measurement inert — the repo's cardinal class, a gate that can only come out green.
|
||||
_IDENTIFIER_FORMS = (
|
||||
# ``SHA-01``, ``RIM-02``, ``B-20-00-00``, ``FOR-2011-12-06-1357`` (K2's 50).
|
||||
re.compile(r"\b[A-ZÆØÅ]{1,8}[-_]\d{1,4}(?:[-_]\d{1,4})*\b"),
|
||||
# ``Krav 3.3.1—13`` (the N corpora's dominant form). EM-DASH U+2014 AND the hyphen, because the
|
||||
# binding known positive is the em-dash spelling and only the em-dash spelling scores 6 of 6.
|
||||
re.compile(r"Krav\s+\d+(?:\.\d+)*\s*[\u2014-]\s*\d+"),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GroundingOffer:
|
||||
"""P8: what the DELIVERED input of one run can lawfully ground an ``affected_item`` code in.
|
||||
|
||||
A REPORT, never a gate. It blocks nothing — a run whose offer is null still runs — because a
|
||||
blocking requirement is exactly ``--require-cost-baseline``, which F4 settled as opt-in.
|
||||
|
||||
The PAIR is the whole diagnosis, and neither number says it alone. MEASURED on K2: the
|
||||
delivered text offers 50 citable identifiers and ZERO cost lines, while the proposer prompt
|
||||
asks each ``affected_item`` to "restate a cost line as the project's price schedule already
|
||||
carries it". An operator reading "identifiers: 50, cost lines: 0" learns that no attempt could
|
||||
have succeeded; reading either number by itself, they learn nothing of the sort.
|
||||
|
||||
``chars`` is the size of the exact text P7's gate will measure against — carried so the report
|
||||
and the gate can be seen to be talking about the same input, which is the only defence against
|
||||
a second rendering free to disagree with the one that was sent.
|
||||
"""
|
||||
|
||||
#: Length of ``_grounding_text``'s output — the text the gate itself will search.
|
||||
chars: int
|
||||
#: Distinct tokens of any ``_IDENTIFIER_FORMS`` shape the text carries.
|
||||
identifiers: int
|
||||
#: Cost lines this run can anchor one of them AS. 0 when the run is un-anchored, which is the
|
||||
#: state every free K2 arm measured. Read off the SAME ``baseline`` binding ``_grounding_text``
|
||||
#: takes as its third source, so the count and the anchoring can never disagree.
|
||||
cost_lines: int
|
||||
|
||||
|
||||
def grounding_offer(
|
||||
project: Project, baseline: CostBaseline | None, delivered: str
|
||||
) -> GroundingOffer:
|
||||
"""Measure what ``delivered`` can ground, on the EXACT text the gate will see.
|
||||
|
||||
Composed THROUGH ``_grounding_text`` — the one composer ``generate_via_llm`` passes to
|
||||
``validate_proposal`` — never re-assembled here. A second composition would be free to drift
|
||||
from the one that was actually sent, and a report about a text nobody was given is worse than
|
||||
no report: it reads as evidence.
|
||||
|
||||
Deterministic and free: no model call, no network, and no second walk of the bundle.
|
||||
"""
|
||||
text = _grounding_text(project, baseline, delivered)
|
||||
found: set[str] = set()
|
||||
for form in _IDENTIFIER_FORMS:
|
||||
found |= set(form.findall(text))
|
||||
return GroundingOffer(
|
||||
chars=len(text),
|
||||
identifiers=len(found),
|
||||
cost_lines=0 if baseline is None else len(baseline.items),
|
||||
)
|
||||
|
||||
|
||||
async def generate_via_llm(
|
||||
chat_client: BaseChatClient,
|
||||
project: Project,
|
||||
|
|
|
|||
|
|
@ -80,7 +80,12 @@ from portfolio_optimiser.explore import (
|
|||
tool_call_payload,
|
||||
trace_payload,
|
||||
)
|
||||
from portfolio_optimiser.generate import ParseFailure, generate_via_llm
|
||||
from portfolio_optimiser.generate import (
|
||||
GroundingOffer,
|
||||
ParseFailure,
|
||||
generate_via_llm,
|
||||
grounding_offer,
|
||||
)
|
||||
from portfolio_optimiser.ir import SavingsProposal
|
||||
from portfolio_optimiser.mandate import (
|
||||
OWN_PROPOSAL_ID,
|
||||
|
|
@ -248,6 +253,15 @@ class RunResult:
|
|||
#: Carried HERE and not on ``ProvenanceStamp``: the stamp describes the gate that judged ONE
|
||||
#: candidate, while this is a RUN-level fact about what the run was allowed to read at all.
|
||||
prepass: prepass.PrepassDeclaration | None = None
|
||||
#: What this run's DELIVERED input could ground an ``affected_item`` code in (P8): the size of
|
||||
#: the text P7's gate searched, how many distinct identifiers of a measured form it carries,
|
||||
#: and how many cost lines the run can anchor one of them AS.
|
||||
#:
|
||||
#: A RUN-level fact settled ONCE, before any candidate exists, which is why it is here and not
|
||||
#: on ``ProvenanceStamp`` (that describes the gate that judged ONE candidate) — ``skipped_links``'
|
||||
#: own placement rule. DEFAULTED, ``prepass``' half of the rule: ``None`` is the true statement
|
||||
#: "no measurement was made", and there is exactly one place that makes it.
|
||||
grounding_offer: GroundingOffer | None = None
|
||||
|
||||
@property
|
||||
def verdict_key(self) -> str:
|
||||
|
|
@ -313,6 +327,12 @@ class DryRunReport:
|
|||
#: resolves a payload, so a dry run can honestly report what it would have read — whereas a
|
||||
#: field resolved above that cut could only ever report zero.
|
||||
prepass: prepass.PrepassDeclaration | None = None
|
||||
#: What the delivered input of a REAL run of this configuration could ground an
|
||||
#: ``affected_item`` code in (P8). Carried here for ``cost_baseline_anchored``'s reason and,
|
||||
#: more sharply, because this is the surface on which "before it spends its three attempts"
|
||||
#: is provable at all: the dry-run cut returns before the first model call, so a dry run that
|
||||
#: reports a null offer has said the run cannot succeed WITHOUT paying to find out.
|
||||
grounding_offer: GroundingOffer | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
|
|
@ -781,6 +801,37 @@ def cost_baseline_notice(anchored: bool) -> str | None:
|
|||
return None if anchored else _UNANCHORED_NOTICE
|
||||
|
||||
|
||||
def grounding_offer_notice(offer: GroundingOffer | None) -> str | None:
|
||||
"""Render the one line that says what this run's delivered input can ground, or ``None`` when
|
||||
there is nothing to warn about (P8).
|
||||
|
||||
ONE renderer with N callsites, never N copies of the wording (kø-(p)), taking the
|
||||
ALREADY-MEASURED value rather than a text: a renderer that re-composed the grounding would be
|
||||
a second resolution of the same rule, free to drift from the run it describes
|
||||
(``cost_baseline_notice``'s rule).
|
||||
|
||||
``None`` when the run CAN anchor a cost line — omission, never an empty row
|
||||
(``mandate.announce``'s rule) — and ``None`` on ``None``, which is the honest reading of "no
|
||||
measurement was made". This is deliberately NOT ``proposal_review_notice``'s deviation: there,
|
||||
silence on zero was ambiguous; here, a run that can anchor its lines has nothing to report that
|
||||
the outcome does not already say.
|
||||
|
||||
BOTH numbers, because the pair is the diagnosis. "0 cost lines" alone reads as a restatement of
|
||||
``cost_baseline_notice``; "50 identifiers" alone reads as good news. Together they say the
|
||||
thing P8 measured: the input offers plenty to cite and nothing to cost, while the proposer
|
||||
prompt asks for a cost line.
|
||||
|
||||
English, like every other line this CLI prints."""
|
||||
if offer is None or offer.cost_lines > 0:
|
||||
return None
|
||||
return (
|
||||
f" Grounding offer: {offer.identifiers} distinct identifier(s) and "
|
||||
f"{offer.cost_lines} cost line(s) in the {offer.chars} characters this run was given — "
|
||||
"the proposer is asked to restate a cost line this input does not carry, so every "
|
||||
"candidate it invents will be refused as ungrounded"
|
||||
)
|
||||
|
||||
|
||||
def bundle_id_notice(resolved: okf.ResolvedBundleId | None) -> str | None:
|
||||
"""Render the one line that says a base was mounted under a name it does not answer to, or
|
||||
``None`` when there is nothing to say.
|
||||
|
|
@ -1166,6 +1217,17 @@ async def run_project(
|
|||
"--derive-cost-baseline when the base carries a priced schedule"
|
||||
)
|
||||
|
||||
# P8: what this run was GIVEN, composed ONCE. Bound HERE and not inside ``_evaluate`` below,
|
||||
# and that placement is the measurement this seam rests on: this is the first point at which
|
||||
# both halves exist AND it is above the ``--live-dry-run`` cut, so the offer can be reported
|
||||
# on the FREE trip — before the first model call at ``debate.run``, let alone the three
|
||||
# generation attempts. ``generate.py``'s own composition happens per attempt, AFTER
|
||||
# ``_fetch_parsed`` has returned, so a report from there could only ever speak once an attempt
|
||||
# had been paid for. ONE binding feeding both the report and the gate: two compositions of one
|
||||
# text are free to disagree, which is exactly what a report must not be able to do (kø-(p)).
|
||||
delivered = "\n".join([context, bundle_grounding])
|
||||
offer = grounding_offer(project, baseline, delivered)
|
||||
|
||||
# Trekk B2 (krav 3): configured MCP servers become tools the AGENTS can call during the debate.
|
||||
# Appended to BOTH paths — on the bundle path they are the first tools that path has ever had.
|
||||
# Constructed here but NOT connected: an ``MCPTool`` is an async context manager, so the run
|
||||
|
|
@ -1239,6 +1301,9 @@ async def run_project(
|
|||
bundle_id_source=resolved_bundle_id,
|
||||
skipped_links=skipped_links,
|
||||
prepass=prepass_declaration,
|
||||
# P8, and this surface is the point: the offer is measured ABOVE this cut, so a dry
|
||||
# run reports it having made no model call at all.
|
||||
grounding_offer=offer,
|
||||
)
|
||||
# The MCP lifecycle (Trekk B2): entered HERE, after the dry-run cut above, so a dry run never
|
||||
# opens a connection — its promise to stop before the first call covers egress too. Constructed
|
||||
|
|
@ -1378,7 +1443,7 @@ async def run_project(
|
|||
# ``context`` is the DELIVERED rendering (pre-pass cut / bundle pointer / retrieved
|
||||
# chunks) — never ``gen_context``, which on the debate path is the model's own
|
||||
# summary and would let a code the debate invented ground the proposal repeating it.
|
||||
grounding="\n".join([context, bundle_grounding]),
|
||||
grounding=delivered,
|
||||
)
|
||||
refinements.extend(generated.refinements)
|
||||
return generated.outcome
|
||||
|
|
@ -1587,6 +1652,9 @@ async def run_project(
|
|||
debate_tool_calls=tuple(debate_tool_calls),
|
||||
expert_revisions=tuple(expert_reviews),
|
||||
prepass=prepass_declaration,
|
||||
# P8: read off the SAME single measurement the gate's own grounding descends from, so the
|
||||
# record and the refusals cannot describe different inputs.
|
||||
grounding_offer=offer,
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -3836,6 +3904,12 @@ def main(argv: list[str] | None = None) -> int:
|
|||
notice = cost_baseline_notice(report.cost_baseline_anchored)
|
||||
if notice is not None:
|
||||
print(notice)
|
||||
# P8, printed next to the line it qualifies: "stage 0 is skipped" says the gate lost a
|
||||
# falsifier; this says what the input could have offered it instead. On the FREE trip, so
|
||||
# an operator learns a run cannot be grounded without paying three attempts to find out.
|
||||
offer_notice = grounding_offer_notice(report.grounding_offer)
|
||||
if offer_notice is not None:
|
||||
print(offer_notice)
|
||||
# The second measured silence on this surface: a bundle with an unfollowable cross-link
|
||||
# dry-ran to rc 0 with nothing said, so a half-read base looked exactly like a small one.
|
||||
nav_notice = skipped_links_notice(report.skipped_links)
|
||||
|
|
@ -3925,6 +3999,11 @@ def main(argv: list[str] | None = None) -> int:
|
|||
notice = cost_baseline_notice(result.provenance.cost_baseline_anchored)
|
||||
if notice is not None:
|
||||
print(notice)
|
||||
# Same renderer on the full run, read off the run's OWN measurement: a run that spent every
|
||||
# attempt being refused as ungrounded is exactly where the input-side fact costs the most.
|
||||
offer_notice = grounding_offer_notice(result.grounding_offer)
|
||||
if offer_notice is not None:
|
||||
print(offer_notice)
|
||||
# Same renderer on the full run, and deliberately so: a run that PRODUCED a proposal from a
|
||||
# half-read base is where the silence cost the most — the dry run at least produced nothing.
|
||||
nav_notice = skipped_links_notice(result.skipped_links)
|
||||
|
|
|
|||
342
tests/test_grounding_offer_loadbearing.py
Normal file
342
tests/test_grounding_offer_loadbearing.py
Normal file
|
|
@ -0,0 +1,342 @@
|
|||
"""P8 — the run SAYS what its delivered input can ground, before it spends an attempt on a proposal
|
||||
that cannot be grounded.
|
||||
|
||||
P7 (økt 109) is right and landed: an identifier a proposal builds on must appear VERBATIM in the
|
||||
input, or the verdict falls. Re-measuring it exposed a CONSEQUENCE no row stated. Over the three
|
||||
free recordings, with the gate live, **29 of 29 cost codes in 13 of 13 delivered proposals are
|
||||
ungrounded** — every one of them invented. That is not a fault in the gate. It is that the PROMPT
|
||||
asks for something the delivered input cannot supply: ``_build_messages`` says each entry "must
|
||||
restate a cost line as the project's price schedule already carries it", and MEASURED, K2's
|
||||
delivered input carries **two** code-shaped tokens in the prompt (``SHA-01``/``SHA-10``), both of
|
||||
them document numbers off a page footer, and ``derive_cost_baseline`` refuses the base outright —
|
||||
there is no cost line in it to restate. A gate that always refuses is as useless as one that never
|
||||
does, so the run must be able to say which of the two situations it is in.
|
||||
|
||||
**This is a REPORT, never a gate.** It does not block: a run with a null offer still runs, because
|
||||
a blocking requirement is exactly ``--require-cost-baseline``, which F4 settled as opt-in and this
|
||||
order freed. It also does not touch ``_ground_against_input``, which ten mutations hold.
|
||||
|
||||
Arms, each with a named detach point:
|
||||
|
||||
* **(a) the null offer is stated.** A delivered text with no identifier of any measured form and no
|
||||
baseline reports ``identifiers=0, cost_lines=0``. This is the K2 CONTROL — without it a green
|
||||
known positive proves nothing.
|
||||
* **(b) the positive offer is stated.** N100's delivered input carries the BINDING known positive
|
||||
``Krav 3.3.1—13`` (P7 premiss (vii): it stands in 6 of 6 prompts, EM-DASH U+2014), and the offer
|
||||
is positive there. Paired with a control that the search CAN find the string at all.
|
||||
* **(c) the offer is measured on the SAME text the gate will see.** ``grounding_offer`` composes
|
||||
through ``generate._grounding_text`` — the one composer P7 uses — so a report built from a second
|
||||
rendering, free to disagree with the one that was sent, is impossible by construction.
|
||||
* **(d) the offer reaches the outcome the operator reads**, on BOTH surfaces: ``RunResult`` after a
|
||||
full run and ``DryRunReport`` before the first model call. The dry-run arm is what proves the
|
||||
measurement happens BEFORE the three attempts are spent — its client factory RAISES.
|
||||
* **(e) ONE renderer**, silent when the run CAN anchor a cost line (omission, never an empty row —
|
||||
``cost_baseline_notice``'s rule), speaking when it cannot.
|
||||
* **(f) a run that exhausts its attempts on ungrounded identifiers leaves the input-side diagnosis
|
||||
in the outcome**, not only "the model missed".
|
||||
|
||||
A NEW file rather than an extension of ``test_identifier_grounding_loadbearing.py``, stated as the
|
||||
order asks: that file's subject is the GATE and it is held by ten mutations. A report mutated here
|
||||
would go red in a file whose docstring promises a falsifier, and which seam a red test names is the
|
||||
only thing a mutation table is for.
|
||||
|
||||
The fixtures are P7's own tracked generation prompts. Tracked test data on purpose (``scratchpad/``
|
||||
is absent from ``git archive HEAD``) — and REUSED rather than extended with the delivered cuts,
|
||||
because those cuts are 10 kB and 96 kB of a real Norwegian tender and a public road standard, and
|
||||
committing corpus text into a repo published on ``open/`` is a publication decision that belongs to
|
||||
the operator, not to this order. The 8-distinct / 435-distinct figures for the delivered cut and the
|
||||
full grounding are MEASURED and reported in ``docs/2026-09-09-p8-forankringstilbudet.md``; what a
|
||||
test asserts here is the property, on text this repo already ships.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from portfolio_optimiser.generate import GroundingOffer, _grounding_text, grounding_offer
|
||||
from portfolio_optimiser.ir import CostBaseline, CostBaselineLine
|
||||
from portfolio_optimiser.reference_domain import CostItem, Project
|
||||
from portfolio_optimiser import run as run_mod
|
||||
from portfolio_optimiser.run import grounding_offer_notice, run_project
|
||||
from portfolio_optimiser.simulation import scripted_factory
|
||||
from portfolio_optimiser.validator import Rejection
|
||||
|
||||
FIXTURES = Path(__file__).parent / "fixtures" / "p7-grounding"
|
||||
SHARED = Path(__file__).resolve().parents[1] / "shared" / "examples"
|
||||
|
||||
#: P7 premiss (vii), the BINDING known positive: the model quoted this requirement number verbatim
|
||||
#: in 6 of 6 replies and it stands in 6 of 6 prompts. EM-DASH (U+2014); the hyphen variant is 0/6.
|
||||
KNOWN_POSITIVE = "Krav 3.3.1—13"
|
||||
|
||||
#: The base with a ``cost-baseline.json`` (one line) and the one without — the pair that makes the
|
||||
#: renderer's omission arm reachable rather than asserted.
|
||||
ANCHORED = SHARED / "veglys-fv-soer"
|
||||
UNANCHORED = SHARED / "bygg-energi-mikro"
|
||||
_UNANCHORED_PID = "BYGG-KONTOR-NORD"
|
||||
_VERDICT_INPUT = {"decision": "approved", "rationale": "expert reviewed (sim)"}
|
||||
_VALID_REPLY = (
|
||||
'{"measure":"LED-retrofit","affected_items":'
|
||||
'[{"code":"ENERGI-TOTAL-EL","quantity":300000,"unit_cost":1.0}],'
|
||||
'"claimed_saving_nok":30000}'
|
||||
)
|
||||
#: A code that appears NOWHERE in the base — the ungrounded case, which P7's stage 0b refuses on
|
||||
#: every attempt, so the run exhausts its attempts and returns the last ``Rejection``.
|
||||
_UNGROUNDED_REPLY = (
|
||||
'{"measure":"invented","affected_items":'
|
||||
'[{"code":"M-04-01","quantity":300000,"unit_cost":1.0}],'
|
||||
'"claimed_saving_nok":30000}'
|
||||
)
|
||||
_CHECKER_REPLY = "Reasoning holds.\nVERDICT: APPROVE"
|
||||
|
||||
|
||||
def _prompt(name: str) -> str:
|
||||
return (FIXTURES / name).read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def _project(*codes: str) -> Project:
|
||||
"""A project shaped like the bundle path's: ``_project_from_bundle`` builds ``cost_items=()``
|
||||
(MEASURED, P7's own row), so the default carries no codes of its own and the offer measures the
|
||||
delivered text alone."""
|
||||
return Project(
|
||||
id="X",
|
||||
name="X",
|
||||
description="",
|
||||
currency="NOK",
|
||||
cost_items=tuple(
|
||||
CostItem(code=c, description=c, quantity=1.0, unit="stk", unit_cost=1.0)
|
||||
for c in codes
|
||||
),
|
||||
docs_dir="",
|
||||
)
|
||||
|
||||
|
||||
def _baseline(*codes: str) -> CostBaseline:
|
||||
return CostBaseline(
|
||||
project_id="X",
|
||||
items={c: CostBaselineLine(quantity=1.0, unit_cost=1.0) for c in codes},
|
||||
)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ (a) the null offer is stated
|
||||
|
||||
|
||||
def test_a_delivered_input_with_no_cost_line_reports_zero() -> None:
|
||||
"""LOAD-BEARING (a) — the K2 CONTROL. Both K2 generation prompts carry no identifier of any
|
||||
measured form and the run is un-anchored, so the offer is null in BOTH numbers. Without this
|
||||
control a green (b) would be satisfied by a report that always counts positive."""
|
||||
for name in ("p6-k2-generation-prompt.txt", "s7c-k2-generation-prompt.txt"):
|
||||
offer = grounding_offer(_project(), None, _prompt(name))
|
||||
assert offer.identifiers == 0, (name, offer)
|
||||
assert offer.cost_lines == 0, (name, offer)
|
||||
assert offer.chars > 0, "the measurement must have had text to measure"
|
||||
|
||||
|
||||
# -------------------------------------------------------------- (b) the positive offer is stated
|
||||
|
||||
|
||||
def test_the_known_positive_input_reports_a_positive_offer() -> None:
|
||||
"""LOAD-BEARING (b) — the BINDING known positive. Paired with the control that the string is
|
||||
actually there: a report that finds nothing because it searched for nothing would otherwise
|
||||
pass (a) and look measured."""
|
||||
text = _prompt("p4-n100-generation-prompt.txt")
|
||||
assert KNOWN_POSITIVE in text, "the fixture no longer carries the known positive"
|
||||
offer = grounding_offer(_project(), None, text)
|
||||
assert offer.identifiers > 0, offer
|
||||
assert offer.cost_lines == 0, "a road standard carries no cost lines (F4's own finding)"
|
||||
|
||||
|
||||
def test_a_bare_number_is_not_counted_as_an_offer() -> None:
|
||||
"""LOAD-BEARING (b), the other half. K2 carries 46 394 bare-number occurrences over 2 117
|
||||
distinct values (P7 § 2), so counting them would make every report positive and the whole
|
||||
measurement inert — the repo's cardinal class, a gate that can only come out green."""
|
||||
offer = grounding_offer(_project(), None, "1234 5678 90 42.5 1000000")
|
||||
assert offer.identifiers == 0, offer
|
||||
|
||||
|
||||
# ----------------------------------------------- (c) measured on the SAME text the gate will see
|
||||
|
||||
|
||||
def test_the_offer_is_measured_on_the_text_the_gate_will_see() -> None:
|
||||
"""LOAD-BEARING (c) — requirement (2). The offer composes through ``_grounding_text``, the one
|
||||
composer P7's gate uses, so the report and the gate cannot describe different texts. Both of
|
||||
the composer's OTHER two sources are exercised: a project cost line and a baseline code each
|
||||
raise the count, which a report built from ``delivered`` alone cannot do."""
|
||||
delivered = "nothing citable here"
|
||||
project, baseline = _project("PRJ-77"), _baseline("BAS-88")
|
||||
|
||||
offer = grounding_offer(project, baseline, delivered)
|
||||
|
||||
assert offer.chars == len(_grounding_text(project, baseline, delivered))
|
||||
assert offer.identifiers == 2, offer
|
||||
assert offer.cost_lines == 1, offer
|
||||
|
||||
|
||||
# ---------------------------------------------------------- (e) ONE renderer, omission when able
|
||||
|
||||
|
||||
def test_the_notice_speaks_on_a_null_offer_and_is_silent_when_anchored() -> None:
|
||||
"""LOAD-BEARING (e). Silent when the run CAN anchor a cost line (omission, never an empty row);
|
||||
speaking when it cannot, and carrying BOTH numbers — "50 identifiers, 0 cost lines" is the
|
||||
diagnosis, and neither number alone says it."""
|
||||
assert grounding_offer_notice(GroundingOffer(chars=9, identifiers=3, cost_lines=1)) is None
|
||||
assert grounding_offer_notice(None) is None
|
||||
|
||||
line = grounding_offer_notice(GroundingOffer(chars=9, identifiers=3, cost_lines=0))
|
||||
assert line is not None
|
||||
assert "3" in line and "0" in line
|
||||
|
||||
|
||||
# --------------------------------------- (d) the offer reaches BOTH surfaces the operator reads
|
||||
|
||||
|
||||
async def test_the_dry_run_carries_the_offer_before_any_model_call() -> None:
|
||||
"""LOAD-BEARING (d), and the proof of "BEFORE it spends its three attempts".
|
||||
|
||||
Asserted on CALLS, never on client CONSTRUCTION: ``fresh_workflow`` builds the proposer and
|
||||
checker clients EAGERLY, above the dry-run cut (``workflow.py:64``), so a factory that raised
|
||||
would be red against a working implementation. An empty sink is the measurement — session 57's
|
||||
rule that a refusal after the spend is indistinguishable from one before it at the exit code.
|
||||
"""
|
||||
sink: list[str] = []
|
||||
|
||||
report = await run_project(
|
||||
_UNANCHORED_PID,
|
||||
"local",
|
||||
docs_dir=str(UNANCHORED),
|
||||
bundle_dir=str(UNANCHORED),
|
||||
client_factory=scripted_factory({"proposer": "x", "checker": "x"}, sink),
|
||||
live_dry_run=True,
|
||||
)
|
||||
|
||||
assert sink == [], "the offer was measured only after a model call was made"
|
||||
|
||||
offer = getattr(report, "grounding_offer", None)
|
||||
assert offer is not None, "the dry run says nothing about what its input can ground"
|
||||
assert offer.cost_lines == 0, offer
|
||||
assert offer.chars > 0, offer
|
||||
|
||||
|
||||
async def test_the_full_run_carries_the_offer() -> None:
|
||||
"""LOAD-BEARING (d), the other surface — requirement (3). A measurement that never leaves
|
||||
``run_project`` is a log line, not a report."""
|
||||
result = await run_project(
|
||||
_UNANCHORED_PID,
|
||||
"local",
|
||||
docs_dir=str(UNANCHORED),
|
||||
bundle_dir=str(UNANCHORED),
|
||||
verdict_input=_VERDICT_INPUT,
|
||||
client_factory=scripted_factory(
|
||||
{"proposer": _VALID_REPLY, "checker": _CHECKER_REPLY}, []
|
||||
),
|
||||
)
|
||||
|
||||
offer = getattr(result, "grounding_offer", None)
|
||||
assert offer is not None, "the run says nothing about what its input can ground"
|
||||
assert offer.cost_lines == 0, offer
|
||||
|
||||
|
||||
async def test_an_anchored_run_reports_its_cost_lines() -> None:
|
||||
"""LOAD-BEARING (d), the CONTROL that ``cost_lines`` is not a constant zero: the anchored base
|
||||
ships exactly one line, and the run reports it."""
|
||||
report = await run_project(
|
||||
"VEGLYS-FV-SOER",
|
||||
"local",
|
||||
docs_dir=str(ANCHORED),
|
||||
bundle_dir=str(ANCHORED),
|
||||
client_factory=scripted_factory({"proposer": "x", "checker": "x"}, []),
|
||||
live_dry_run=True,
|
||||
)
|
||||
|
||||
offer = getattr(report, "grounding_offer", None)
|
||||
assert offer is not None and offer.cost_lines == 1, offer
|
||||
|
||||
|
||||
# ------------------------------------- (f) an exhausted run leaves the INPUT-side diagnosis behind
|
||||
|
||||
|
||||
async def test_a_run_exhausted_on_ungrounded_identifiers_says_the_input_was_the_problem() -> None:
|
||||
"""LOAD-BEARING (f). P7's gate refuses every attempt, the loop returns its last ``Rejection``
|
||||
(``generate.py``: ``last_ruling`` after the attempt loop — it does not crash), and the outcome
|
||||
the operator reads carries the input-side fact as well: this input could anchor NOTHING, so no
|
||||
attempt could ever have succeeded. Without it the record says only that the model missed."""
|
||||
result = await run_project(
|
||||
_UNANCHORED_PID,
|
||||
"local",
|
||||
docs_dir=str(UNANCHORED),
|
||||
bundle_dir=str(UNANCHORED),
|
||||
verdict_input=_VERDICT_INPUT,
|
||||
client_factory=scripted_factory(
|
||||
{"proposer": _UNGROUNDED_REPLY, "checker": _CHECKER_REPLY}, []
|
||||
),
|
||||
)
|
||||
|
||||
assert isinstance(result.outcome, Rejection)
|
||||
assert "M-04-01" in result.outcome.reason
|
||||
offer = getattr(result, "grounding_offer", None)
|
||||
assert offer is not None and offer.cost_lines == 0, offer
|
||||
assert grounding_offer_notice(offer) is not None
|
||||
|
||||
|
||||
# ------------------------------------------------ (d) the CLI prints it, on BOTH free and paid
|
||||
|
||||
|
||||
def test_the_cli_dry_run_prints_the_offer(capsys) -> None:
|
||||
"""LOAD-BEARING (d), stdout. A renderer that returns the line while no caller prints it is a
|
||||
measurement the operator never sees — its own seam, so its own arm (the ``cost_baseline_notice``
|
||||
precedent, whose CLI print carries a mutation of its own)."""
|
||||
rc = run_mod.main(
|
||||
[
|
||||
_UNANCHORED_PID,
|
||||
"--docs-dir",
|
||||
str(UNANCHORED),
|
||||
"--bundle-dir",
|
||||
str(UNANCHORED),
|
||||
"--live-dry-run",
|
||||
]
|
||||
)
|
||||
assert rc == 0
|
||||
assert "Grounding offer" in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_the_cli_says_nothing_when_the_run_can_anchor(capsys) -> None:
|
||||
"""CONTROL for the omission arm on the SAME surface: the anchored base prints no offer line at
|
||||
all. Without it, a notice that always fired would pass the arm above."""
|
||||
rc = run_mod.main(
|
||||
[
|
||||
"VEGLYS-FV-SOER",
|
||||
"--docs-dir",
|
||||
str(ANCHORED),
|
||||
"--bundle-dir",
|
||||
str(ANCHORED),
|
||||
"--live-dry-run",
|
||||
]
|
||||
)
|
||||
assert rc == 0
|
||||
assert "Grounding offer" not in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_the_cli_full_run_prints_the_offer(tmp_path: Path, capsys) -> None:
|
||||
"""LOAD-BEARING (d), the paid surface's stdout — so the line is a property of a RUN and not of
|
||||
the dry-run branch alone."""
|
||||
replies = tmp_path / "replies.json"
|
||||
replies.write_text(
|
||||
json.dumps({"proposer": _VALID_REPLY, "checker": _CHECKER_REPLY}), encoding="utf-8"
|
||||
)
|
||||
rc = run_mod.main(
|
||||
[
|
||||
_UNANCHORED_PID,
|
||||
"--docs-dir",
|
||||
str(UNANCHORED),
|
||||
"--bundle-dir",
|
||||
str(UNANCHORED),
|
||||
"--scripted-replies",
|
||||
str(replies),
|
||||
"--decision",
|
||||
"approved",
|
||||
"--rationale",
|
||||
"expert reviewed (test)",
|
||||
]
|
||||
)
|
||||
assert rc == 0
|
||||
assert "Grounding offer" in capsys.readouterr().out
|
||||
Loading…
Add table
Add a link
Reference in a new issue