feat(consume): the payload says what of the question it reached, and row 4 reads it
`coverage` carries three lists: the terms the pre-pass read the question as, the terms no concept in the bundle answers, and the terms no delivered excerpt answers. Without it a reader holding eight excerpts cannot tell a bundle that ANSWERED its question from one that merely ranked something -- the two payloads have the same shape. FACTS, AND NO VERDICT, which is a measurement and not caution. Two readings were built and both falsified over 81 questions (16 synthetic, 65 across the three real sets, 2026-09-20): the share of question terms a delivered excerpt answers separates the synthetic controls at 0.33 against 0.50 and REVERSES on real data (covered questions down to 0.27, one genuinely uncovered question at 0.71); the share of a bundle tying the best lexical match is ~0.00 for every real question either way. Question style dominates the first, corpus size the second. The one bar this repository declares is the gate's: `UNANSWERED_BAR = 2/3` over `unanswered_in_bundle`, swept and collapsing at both ends -- at 0.50 eleven real covered questions are marked, at 0.70 the row falls to 5 of 6, at 2/3 the row is 6 of 6 and 0 of 65 real questions are marked. The margin is thin (0.6087 against 0.6667) and is published that way, together with what it does not catch: r761-sk2's own known-negative sits at 0.2857. Row 4: 3 of 6 RED -> 6 of 6 GREEN, with the 10 answered synthetic questions held unmarked as the known-negative. The contract's SS 8 gains point 7, the consumption skill is told to read the block, and the SS 7.4 known-positive moves with the document (14 721/375 -> 16 389/417). Suite 2292 passed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
90394c383d
commit
05cb19087a
8 changed files with 216 additions and 42 deletions
|
|
@ -199,6 +199,16 @@ at which this strategy stops fitting the budget.>`
|
|||
The payload reports three counts — `considered`, `withheld`, `delivered` — and
|
||||
`considered == withheld + delivered`. Carry them into your output.
|
||||
|
||||
It also reports **what of the question it reached**. `coverage` carries the
|
||||
terms the pre-pass read the question as, the terms no concept in the bundle
|
||||
answers, and the terms no delivered excerpt answers. Read it before you answer.
|
||||
It carries no score and no verdict — deliberately: two were built and both
|
||||
reversed on real corpora, so the judgement is yours. Where the bundle answers
|
||||
none of the terms that make the question specific, say so and stop; do not
|
||||
compose an answer out of excerpts that were ranked anyway. A cut always returns
|
||||
its best candidates, so an ungrounded answer looks exactly like a grounded one
|
||||
until somebody checks which of the asked-about words actually arrived.
|
||||
|
||||
Any claim of the form "there is no X", "nothing further was found" or "all N are
|
||||
Y" reports the denominator it was measured over and the command that produced
|
||||
it. A negative result whose scope is unstated is **unmeasured**, and is reported
|
||||
|
|
|
|||
|
|
@ -210,7 +210,7 @@ carries its denominator.
|
|||
| Limit | `120000` |
|
||||
| Unit | `utf-8 bytes of emitted JSON` |
|
||||
| Instrument | `okf_consume.measure (len of the ensure_ascii=False JSON encoding, utf-8)` |
|
||||
| Known-positive | `docs/consumption-contract.md, encoded as a JSON string` at `14721` |
|
||||
| Known-positive | `docs/consumption-contract.md, encoded as a JSON string` at `16389` |
|
||||
|
||||
The instrument reproduces the known-positive figure before any of its own
|
||||
numbers are believed. Report what the run actually spent.
|
||||
|
|
@ -263,6 +263,16 @@ full one is not evidence of presence.** When the delivered excerpts do not
|
|||
actually answer the question, say `[sourced-not-sufficient]` and report that
|
||||
the cut found nothing responsive.
|
||||
|
||||
It also reports **what of the question it reached**. `coverage` carries the
|
||||
terms the pre-pass read the question as, the terms no concept in the bundle
|
||||
answers, and the terms no delivered excerpt answers. Read it before you answer.
|
||||
It carries no score and no verdict — deliberately: two were built and both
|
||||
reversed on real corpora, so the judgement is yours. Where the bundle answers
|
||||
none of the terms that make the question specific, say so and stop; do not
|
||||
compose an answer out of excerpts that were ranked anyway. A cut always returns
|
||||
its best candidates, so an ungrounded answer looks exactly like a grounded one
|
||||
until somebody checks which of the asked-about words actually arrived.
|
||||
|
||||
Any claim of the form "there is no X", "nothing further was found" or "all N are
|
||||
Y" reports the denominator it was measured over and the command that produced
|
||||
it. A negative result whose scope is unstated is **unmeasured**, and is reported
|
||||
|
|
|
|||
|
|
@ -11,10 +11,10 @@
|
|||
"spent": 2289,
|
||||
"known_positive": {
|
||||
"case": "docs/consumption-contract.md, encoded as a JSON string",
|
||||
"expected": 14721,
|
||||
"measured": 14721,
|
||||
"raw_bytes": 14346,
|
||||
"encoding_delta": 375
|
||||
"expected": 16389,
|
||||
"measured": 16389,
|
||||
"raw_bytes": 15972,
|
||||
"encoding_delta": 417
|
||||
}
|
||||
},
|
||||
"denominators": {
|
||||
|
|
@ -23,6 +23,22 @@
|
|||
"delivered": 3
|
||||
},
|
||||
"question": "Hva sier veiledningen om krav?",
|
||||
"coverage": {
|
||||
"question_terms": [
|
||||
"hva",
|
||||
"sier",
|
||||
"veiledningen",
|
||||
"krav"
|
||||
],
|
||||
"unanswered_in_bundle": [
|
||||
"hva",
|
||||
"sier"
|
||||
],
|
||||
"unanswered_in_payload": [
|
||||
"hva",
|
||||
"sier"
|
||||
]
|
||||
},
|
||||
"excerpts": [
|
||||
{
|
||||
"bundle_id": "b-golden-segmented-okf-v0-2",
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue