Kjell Tore Guttormsen
17c49fc04b
The pre-pass delivered the right concept and the answer could not name it.
Measured by portfolio-optimiser 2026-09-08 over three paid arms: the gold
concept came back at rank 1 of 8 on 3 of 3 bundles, and the model answered
correctly on 1 of 3, because a delivered excerpt carried `concept_id`, body
text and nothing the document is known by. The previous session measured the
same gap from the other side: the provenance it had just written into every
concept did not reach the payload at all.
`excerpt_for` now carries `title` unconditionally, and `req_number`, the SPEC
5.1 address `sources` and each locator key (`source_pages`, `source_sheet`,
`source_rows`, `source_lines`, `source_offset`) when the concept has them. A key
the producer did not write stays absent: an empty value would assert that they
wrote an empty one, which is the contract's 6.4 failure.
`sources` is read in BOTH YAML forms, on a measurement rather than a taste. K2
writes the flow form on 629 of 629 concepts; the largest N-bundle writes the
block form on 270 of 270 and carries no locator key at all, so a flow-only
reader delivers that bundle with no address whatsoever. Reading the block form
is not a licence to write it - the emission rule is untouched, because the
line-oriented parser still cannot round-trip a block list. A `sources` value
this reader cannot decode is named (`sources_unreadable`), never dropped into
the same silence as an absent one.
Contract 8 gains the requirement and the checker gains its code
(`excerpt_unnamed`, 15 rules now, was 14): an excerpt a reader cannot name is
one an answer cannot cite, whatever its rank. `req_number`, `sources` and the
locators are SHOULD, not MUST - they are conditional on the producer, and a
bundle whose concepts carry no identifier cannot deliver one.
K2 controls, same question and same k, before against a frozen copy of the tool
at b6a8c8b: the RANKING does not move - the same 8 concept ids in the same
order, identical `text_sha256`, identical `withheld`, identical denominators
(629 = 621 + 8). The FIELD is what moved: payload 108 877 -> 111 744 B
(+2.63 %), budget spent 18 606 -> 20 907 (+287.6 B per excerpt), excerpt
members 9 -> 15, 83 changed lines. The contract document's own bytes moved with
8, so the budget instrument's known-positive moves with it: 10 349 -> 12 049
measured, 10 060 -> 11 719 raw, delta 289 -> 330.
New fixture `tests/fixtures/consume-provenance`: the two address forms and a
concept carrying neither address nor identifier. Purpose-built, because the two
real bundles are complementary and neither exercises both forms.
Suite 1347 (1339 before), ruff clean, mypy src clean.
Co-Authored-By: Claude <claude-opus-5>
70 lines
1.8 KiB
JSON
70 lines
1.8 KiB
JSON
{
|
|
"contract": "okf-consumption/1",
|
|
"bundle": {
|
|
"bundle_id": "example-corpus",
|
|
"ref": "9dd86b1f0c2e4a7b8d3f5061a2c4e6809bdf1357"
|
|
},
|
|
"budget": {
|
|
"unit": "o200k_base tokens",
|
|
"instrument": "<BUDGET_INSTRUMENT>",
|
|
"limit": 30000,
|
|
"spent": 18412,
|
|
"known_positive": {
|
|
"case": "commons reference document",
|
|
"expected": 10406,
|
|
"measured": 10406
|
|
}
|
|
},
|
|
"denominators": {
|
|
"considered": 5,
|
|
"withheld": 2,
|
|
"delivered": 3
|
|
},
|
|
"excerpts": [
|
|
{
|
|
"bundle_id": "example-corpus",
|
|
"concept_id": "fire-compartmentation-3-1",
|
|
"path": "concepts/fire-compartmentation-3-1.md",
|
|
"sha256": "39ccd5fdef696327c9b8bed873cb5d085fc2460e9a01cd12f872488cd8c74580",
|
|
"adjudication": "adjudicated",
|
|
"trust_tier": "human-reviewed",
|
|
"title": "Fire compartmentation",
|
|
"req_number": "Req 3.1-1",
|
|
"sources": [
|
|
{
|
|
"resource": "fire-code.pdf",
|
|
"title": "Fire code 2024"
|
|
}
|
|
],
|
|
"source_pages": "[11, 14]"
|
|
},
|
|
{
|
|
"bundle_id": "example-corpus",
|
|
"concept_id": "escape-routes-3-2",
|
|
"path": "concepts/escape-routes-3-2.md",
|
|
"sha256": "4352bb77959800ef35bb2b7afd82fbe764a93adc4252d635263388c8ef8811a6",
|
|
"adjudication": "proposed",
|
|
"trust_tier": "machine-confirmed",
|
|
"title": "Escape routes"
|
|
},
|
|
{
|
|
"bundle_id": "example-corpus",
|
|
"concept_id": "scope-1-0",
|
|
"path": "concepts/scope-1-0.md",
|
|
"sha256": "735ee94323f323b17bb567b9f02fa0926ade222f520c66641581e9be3c007088",
|
|
"adjudication": "unknown",
|
|
"trust_tier": "unverified",
|
|
"title": "Scope"
|
|
}
|
|
],
|
|
"withheld": [
|
|
{
|
|
"concept_id": "appendix-a-tables",
|
|
"rule": "over budget after knapsack"
|
|
},
|
|
{
|
|
"concept_id": "revision-log",
|
|
"rule": "type not in question scope"
|
|
}
|
|
]
|
|
}
|