open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(consume): give every excerpt the name and the address an answer must cite

Browse source 
The pre-pass delivered the right concept and the answer could not name it.
Measured by portfolio-optimiser 2026-09-08 over three paid arms: the gold
concept came back at rank 1 of 8 on 3 of 3 bundles, and the model answered
correctly on 1 of 3, because a delivered excerpt carried `concept_id`, body
text and nothing the document is known by. The previous session measured the
same gap from the other side: the provenance it had just written into every
concept did not reach the payload at all.

`excerpt_for` now carries `title` unconditionally, and `req_number`, the SPEC
5.1 address `sources` and each locator key (`source_pages`, `source_sheet`,
`source_rows`, `source_lines`, `source_offset`) when the concept has them. A key
the producer did not write stays absent: an empty value would assert that they
wrote an empty one, which is the contract's 6.4 failure.

`sources` is read in BOTH YAML forms, on a measurement rather than a taste. K2
writes the flow form on 629 of 629 concepts; the largest N-bundle writes the
block form on 270 of 270 and carries no locator key at all, so a flow-only
reader delivers that bundle with no address whatsoever. Reading the block form
is not a licence to write it - the emission rule is untouched, because the
line-oriented parser still cannot round-trip a block list. A `sources` value
this reader cannot decode is named (`sources_unreadable`), never dropped into
the same silence as an absent one.

Contract 8 gains the requirement and the checker gains its code
(`excerpt_unnamed`, 15 rules now, was 14): an excerpt a reader cannot name is
one an answer cannot cite, whatever its rank. `req_number`, `sources` and the
locators are SHOULD, not MUST - they are conditional on the producer, and a
bundle whose concepts carry no identifier cannot deliver one.

K2 controls, same question and same k, before against a frozen copy of the tool
at b6a8c8b: the RANKING does not move - the same 8 concept ids in the same
order, identical `text_sha256`, identical `withheld`, identical denominators
(629 = 621 + 8). The FIELD is what moved: payload 108 877 -> 111 744 B
(+2.63 %), budget spent 18 606 -> 20 907 (+287.6 B per excerpt), excerpt
members 9 -> 15, 83 changed lines. The contract document's own bytes moved with
8, so the budget instrument's known-positive moves with it: 10 349 -> 12 049
measured, 10 060 -> 11 719 raw, delta 289 -> 330.

New fixture `tests/fixtures/consume-provenance`: the two address forms and a
concept carrying neither address nor identifier. Purpose-built, because the two
real bundles are complementary and neither exercises both forms.

Suite 1347 (1339 before), ruff clean, mypy src clean.

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 15:01:41 +02:00
commit 17c49fc04b
14 changed files with 454 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

10
CLAUDE.md
View file

@ -277,6 +277,16 @@ and fixtures, never code.
**assertion**, never an override: the emitted identity is always the computed
one, because § 3.3 exists to stop a payload being labelled with an identity
its bytes do not have. Three exit codes: 0 written, 1 refused, 2 did not run.
**Every excerpt carries the concept's `title`**, plus `req_number`, the § 5.1
address `sources` and one locator (`source_pages` / `source_sheet` +
`source_rows` / `source_lines`, and `source_offset`) when the producer wrote
them; an absent key stays absent and an undecodable address is named
(`sources_unreadable`). `sources` is READ in both YAML forms because the two
real bundles disagree (flow 629/629 on one, block 270/270 on the other) —
reading block is not a licence to write it, the emission rule is unchanged.
Contract § 8 makes `title` a MUST (checker code `excerpt_unnamed`) and the
rest SHOULD, because they are conditional on the producer. The measurement
behind it: rank 1 of 8 on 3 of 3 bundles, correct answer on 1 of 3.
The first instantiated consumption skill is `skills/okf-consume/`; the
measurement behind it, including the control that FAILED, is
`docs/2026-09-07-okf-konsumskill-maaling.md`. **The ranking is this

9
README.md
View file

@ -156,6 +156,15 @@ It emits the § 8 shape — `contract`, `bundle` (`bundle_id` plus a
`sha256-tree:` content identity), `budget` (unit, instrument, limit, spent and a
validated known-positive), `denominators`, `excerpts` and `withheld` — and every
withheld concept names the rule that dropped it, from a closed set of six.
Every excerpt carries the concept's `title`, and — when the producer wrote them
— `req_number`, the SPEC § 5.1 address `sources`, and one locator into that
address (`source_pages`, `source_sheet` with `source_rows`, or `source_lines`,
plus `source_offset`). A key the producer did not write stays absent rather than
arriving empty, and an address this reader cannot decode is named
(`sources_unreadable`) rather than dropped into the same silence. The reason is
a measurement: with `concept_id` and body text alone, a delivered gold concept
at rank 1 still left the answer unable to name the document it was quoting.
`considered == withheld + delivered` closes by construction, and the payload is
refused rather than reported when it does not.

25
docs/consumption-contract.md
View file

@ -157,7 +157,11 @@ are permitted and are not read by the checker.
"concept_id": "<bundle-local id>",
"sha256": "<hex digest of the concept file>",
"adjudication": "proposed",
"trust_tier": "machine-confirmed"
"trust_tier": "machine-confirmed",
"title": "<the concept's own title>",
"req_number": "<the identifier the producer wrote, when there is one>",
"sources": [ { "resource": "<what the concept came from>" } ],
"source_pages": "<a locator into that resource, when the producer wrote one>"
}
],
"withheld": [ { "concept_id": "<bundle-local id>", "rule": "<why it was cut>" } ]
@ -168,6 +172,25 @@ are permitted and are not read by the checker.
equal `denominators.withheld`. The counts and the lists are two statements of
the same fact, and a payload where they disagree is refused.
2. `contract` MUST be present so a reader can tell which revision it is holding.
3. Every excerpt MUST carry a non-empty `title`. An excerpt a reader cannot name
is one an answer cannot cite, whatever its rank. This is a measurement, not a
preference: in three paid arms run 2026-09-08 the pre-pass delivered the gold
concept at rank 1 of 8 on 3 of 3 bundles and the model answered correctly on
1 of 3, because the delivered unit carried `concept_id` and body text and
nothing the document is known by. § 3.1's tuple is an identity for a
*machine*; a title is what a citation is made of.
4. An excerpt SHOULD carry the identifier the producer wrote (`req_number`), the
§ 5.1 address of what it came from (`sources`), and the locator into that
address the producer wrote. These are SHOULD and not MUST because they are
conditional on the producer: a bundle whose concepts carry no identifier
cannot deliver one, and § 6.4 forbids reading their absence as a claim that
the source has none. A payload MUST NOT write an empty value in place of an
absent one, and MUST distinguish an address it could not read from one that
is not there.
5. The locator keys are the *profile's*, not this contract's: SPEC § 5.1 has no
field for a place within a resource, so where a page number, a sheet row or a
line range is named — and under which key — is stated by the profile and read
from the concept, never invented here.
## 9. Prohibitions

10
skills/okf-consume-template/SKILL.md
View file

@ -53,6 +53,16 @@ and the cut; it decides nothing about the question.
Every claim carries exactly one of these five literals, plus a pointer to the
excerpt it rests on — `(bundle_id, concept_id)` and the excerpt's `sha256`.
**Name the document, do not merely point at it.** Each excerpt also carries
`title`, and — when the producer wrote them — `req_number`, the § 5.1 address
`sources`, and a locator into that address (`source_pages`, `source_sheet` with
`source_rows`, or `source_lines`). Quote those values as they stand; they are the
difference between "the bundle says X" and "X, from `<title>` `<req_number>`,
`<resource>` at `<locator>`". Absent keys are absent because the producer wrote
none — never because the source has none, and never something to fill in. An
excerpt carrying `sources_unreadable` has an address this reader could not
decode: say so rather than reporting no address.
| Marking | Use when |
|---|---|
| `extracted` | the bundle states it directly |

17
skills/okf-consume-template/references/example-payload.json
View file

@ -27,7 +27,16 @@
"path": "concepts/fire-compartmentation-3-1.md",
"sha256": "39ccd5fdef696327c9b8bed873cb5d085fc2460e9a01cd12f872488cd8c74580",
"adjudication": "adjudicated",
"trust_tier": "human-reviewed"
"trust_tier": "human-reviewed",
"title": "Fire compartmentation",
"req_number": "Req 3.1-1",
"sources": [
{
"resource": "fire-code.pdf",
"title": "Fire code 2024"
}
],
"source_pages": "[11, 14]"
},
{
"bundle_id": "example-corpus",
@ -35,7 +44,8 @@
"path": "concepts/escape-routes-3-2.md",
"sha256": "4352bb77959800ef35bb2b7afd82fbe764a93adc4252d635263388c8ef8811a6",
"adjudication": "proposed",
"trust_tier": "machine-confirmed"
"trust_tier": "machine-confirmed",
"title": "Escape routes"
},
{
"bundle_id": "example-corpus",
@ -43,7 +53,8 @@
"path": "concepts/scope-1-0.md",
"sha256": "735ee94323f323b17bb567b9f02fa0926ade222f520c66641581e9be3c007088",
"adjudication": "unknown",
"trust_tier": "unverified"
"trust_tier": "unverified",
"title": "Scope"
}
],
"withheld": [

16
skills/okf-consume/SKILL.md
View file

@ -81,6 +81,16 @@ alongside the set that was chosen).
Every claim carries exactly one of these five literals, plus a pointer to the
excerpt it rests on — `(bundle_id, concept_id)` and the excerpt's `sha256`.
**Name the document, do not merely point at it.** Each excerpt also carries
`title`, and — when the producer wrote them — `req_number`, the § 5.1 address
`sources`, and a locator into that address (`source_pages`, `source_sheet` with
`source_rows`, or `source_lines`). Quote those values as they stand; they are the
difference between "the bundle says X" and "X, from `<title>` `<req_number>`,
`<resource>` at `<locator>`". Absent keys are absent because the producer wrote
none — never because the source has none, and never something to fill in. An
excerpt carrying `sources_unreadable` has an address this reader could not
decode: say so rather than reporting no address.
| Marking | Use when |
|---|---|
| `extracted` | the bundle states it directly |
@ -145,7 +155,7 @@ other states is a claim about the fixture rather than about this corpus.
| Limit | `120000` |
| Unit | `utf-8 bytes of emitted JSON` |
| Instrument | `okf_consume.measure` — `len(json.dumps(value, ensure_ascii=False).encode("utf-8"))` |
| Known-positive | `docs/consumption-contract.md, encoded as a JSON string` at `10349` |
| Known-positive | `docs/consumption-contract.md, encoded as a JSON string` at `12049` |
The instrument reproduces the known-positive figure before any of its own
numbers are believed. Report what the run actually spent.
@ -154,8 +164,8 @@ The known-positive is a **shipped artefact rather than this bundle**, and the
reason is that a per-bundle one cannot work: it would be either a constant wrong
for every bundle but one, or the instrument's own output, which makes
`expected == measured` true by construction and § 7.4 decorative. It is checked
by a **second, independent route**: `wc -c` reports 10 060 raw bytes for the same
file, and the 289-byte difference is that file's JSON quoting and escaping
by a **second, independent route**: `wc -c` reports 11 719 raw bytes for the same
file, and the 330-byte difference is that file's JSON quoting and escaping
overhead. The delta moves the moment the instrument changes what it counts.
`spent` is the cost of the **delivered set**, per § 7.2 — not of the whole

37
skills/okf-consume/references/example-payload.json
View file

@ -8,13 +8,13 @@
"unit": "utf-8 bytes of emitted JSON",
"instrument": "okf_consume.measure (len of the ensure_ascii=False JSON encoding, utf-8)",
"limit": 120000,
"spent": 1306,
"spent": 1774,
"known_positive": {
"case": "docs/consumption-contract.md, encoded as a JSON string",
"expected": 10349,
"measured": 10349,
"raw_bytes": 10060,
"encoding_delta": 289
"expected": 12049,
"measured": 12049,
"raw_bytes": 11719,
"encoding_delta": 330
}
},
"denominators": {
@ -31,6 +31,15 @@
"adjudication": "proposed",
"trust_tier": "unverified",
"bundle_id_inherited": false,
"title": "Foerste krav",
"sources": [
{
"resource": "veiledning.md",
"title": "veiledning.md"
}
],
"source_lines": "[5, 8]",
"source_offset": "[94, 176]",
"text_sha256": "08447c690087834883a78fb9f40d474c68b2526ed57d0dafe01c4868db3c04f0",
"text": "\n## 1.1 Foerste krav\n\nEt krav som beskriver den foerste betingelsen i eksempelet.",
"rank": 1
@ -42,6 +51,15 @@
"adjudication": "proposed",
"trust_tier": "unverified",
"bundle_id_inherited": false,
"title": "Veiledning for eksempelbundel",
"sources": [
{
"resource": "veiledning.md",
"title": "veiledning.md"
}
],
"source_lines": "[1, 4]",
"source_offset": "[0, 94]",
"text_sha256": "82d67ff01367a1a56532130e077956048004ac047e6062eae2143d96704c4a92",
"text": "\n# Veiledning for eksempelbundel\n\nDenne teksten er syntetisk og finnes kun for a pinne bytes.",
"rank": 2
@ -53,6 +71,15 @@
"adjudication": "proposed",
"trust_tier": "unverified",
"bundle_id_inherited": false,
"title": "Andre krav",
"sources": [
{
"resource": "veiledning.md",
"title": "veiledning.md"
}
],
"source_lines": "[9, 11]",
"source_offset": "[176, 253]",
"text_sha256": "4c181e1debb141d3cb1a7bd47249fdac1c0b2c5db1b47cf255ef8f04f83ec840",
"text": "\n## 1.2 Andre krav\n\nEt krav som beskriver den andre betingelsen i eksempelet.",
"rank": 3

15
tests/fixtures/consume-provenance/blokkform-krav.md vendored Normal file
View file

@ -0,0 +1,15 @@
---
type: Krav
title: Krav 10.2-2 Beredskap
req_number: Krav 10.2-2
ingested_at: 2026-09-08T00:00:00Z
adjudication: proposed
bundle_id: provenance-fixture
sources:
- resource: https://example.invalid/api/nisosts/1?languageCode=nb
title: Fiktiv normal 2024
---
## Krav
Beredskapsplanen skal oppdateres hvert aar.

14
tests/fixtures/consume-provenance/flytform-side.md vendored Normal file
View file

@ -0,0 +1,14 @@
---
type: reference
title: Sideoppslag i regelverket
ingested_at: 2026-09-08T00:00:00Z
adjudication: proposed
bundle_id: provenance-fixture
sources: [{ resource: Fiktivt grunnlag.pdf, title: Fiktivt grunnlag.pdf }]
source_pages: [4, 6]
source_offset: [120, 480]
---
## Sideoppslag
Teksten staar paa sidene fire til seks i det fiktive grunnlaget.

8
tests/fixtures/consume-provenance/index.md vendored Normal file
View file

@ -0,0 +1,8 @@
---
okf_version: 0.2
bundle_id: provenance-fixture
---
- [Krav 10.2-2 Beredskap](blokkform-krav.md)
- [Sideoppslag i regelverket](flytform-side.md)
- [Notat uten adresse](uten-adresse.md)

11
tests/fixtures/consume-provenance/uten-adresse.md vendored Normal file
View file

@ -0,0 +1,11 @@
---
type: reference
title: Notat uten adresse
ingested_at: 2026-09-08T00:00:00Z
adjudication: proposed
bundle_id: provenance-fixture
---
## Notat
Dette konseptet har verken sources, kravnummer eller lokator.

138
tests/test_okf_consume.py
View file

@ -29,6 +29,7 @@ import socket
import subprocess
import sys
import unicodedata
from collections.abc import Mapping
from pathlib import Path
import pytest
@ -1615,6 +1616,10 @@ def _synthetic(concept_id: str, title: str, body: str) -> okf_consume.Concept:
source_file="synthetic.md",
adjudication="unknown",
adjudication_present=False,
req_number="",
sources=(),
sources_present=False,
locators={},
frontmatter={},
body=body,
)
@ -2090,3 +2095,136 @@ def test_the_lookup_keeps_the_fused_order_among_the_concepts_it_lifts() -> None:
assert [concept.concept_id for concept, _, _ in ranked[len(hits) :]] == sorted(
concept.concept_id for concept in concepts if concept.concept_id not in hits
)
# --- Step 13: the excerpt carries the key the question is asked ON (C1 step 0) -
#: A bundle purpose-built for one measurement: the two SS 5.1 `sources` forms
#: and a concept carrying neither address nor locator. It exists because the
#: two real bundles are complementary and neither exercises both forms --
#: measured 2026-09-08, K2 writes `sources` in FLOW form on 629 of 629 concepts
#: and carries `req_number` on 0, while the N500 bundle writes BLOCK form on 270
#: of 270, carries `req_number` on 269 and carries no locator key at all.
PROVENANCE = PROJECT_ROOT / "tests" / "fixtures" / "consume-provenance"
PROVENANCE_QUESTION = "Hva krever Krav 10.2-2?"
def _provenance_excerpts() -> dict[str, Mapping[str, object]]:
# Every concept, not only the delivered ones: the cut is a different
# question and would make this measurement depend on a ranking.
excerpts: dict[str, Mapping[str, object]] = {}
for concept_id in okf_consume.enumerate_concepts(PROVENANCE):
excerpt = okf_consume.excerpt_for(
okf_consume.read_concept(
PROVENANCE / f"{concept_id}.md",
bundle_root=PROVENANCE,
root_bundle_id="provenance-fixture",
)
)
assert excerpt is not None
excerpts[concept_id] = excerpt
return excerpts
def test_every_delivered_excerpt_names_the_concept_the_answer_would_cite() -> None:
# po measured 2026-09-08 (`e7ffe9e`) that the pre-pass delivers the gold
# concept at rank 1 and the model still cannot name what it is citing: the
# excerpt carried `concept_id` and `text` and no `title`. The ranking found
# the document; the delivery dropped the key.
payload = okf_consume.build_payload(PROVENANCE, question=PROVENANCE_QUESTION)
delivered = payload["excerpts"]
assert isinstance(delivered, list)
assert delivered, "the cut delivered nothing, so the assertion below is vacuous"
for excerpt in delivered:
concept = okf_consume.read_concept(
PROVENANCE / f"{excerpt['concept_id']}.md", # type: ignore[index]
bundle_root=PROVENANCE,
root_bundle_id="provenance-fixture",
)
assert excerpt["title"] == concept.title # type: ignore[index]
assert excerpt["title"] # type: ignore[index]
def test_an_excerpt_carries_the_identifier_the_question_spells_and_never_invents_one() -> None:
excerpts = _provenance_excerpts()
assert excerpts["blokkform-krav"]["req_number"] == "Krav 10.2-2"
# A concept without the key gets no key. Emitting `""` would assert that the
# producer wrote an empty identifier, which is SS 6.4's failure.
assert "req_number" not in excerpts["flytform-side"]
assert "req_number" not in excerpts["uten-adresse"]
def test_a_flow_form_address_and_its_locator_reach_the_excerpt() -> None:
excerpt = _provenance_excerpts()["flytform-side"]
assert excerpt["sources"] == [
{"resource": "Fiktivt grunnlag.pdf", "title": "Fiktivt grunnlag.pdf"}
]
assert excerpt["source_pages"] == "[4, 6]"
assert excerpt["source_offset"] == "[120, 480]"
assert "source_lines" not in excerpt
def test_a_block_form_address_reaches_the_excerpt_rather_than_becoming_an_empty_string() -> None:
# `parse_frontmatter` SKIPS indented lines, so the block form arrives as
# `''` -- present and unreadable. Measured 2026-09-08: 270 of 270 concepts
# in the largest N-bundle write it that way, so a reader that only handles
# the flow form delivers that bundle with no address at all.
excerpt = _provenance_excerpts()["blokkform-krav"]
assert excerpt["sources"] == [
{
"resource": "https://example.invalid/api/nisosts/1?languageCode=nb",
"title": "Fiktiv normal 2024",
}
]
def test_a_concept_with_no_address_and_no_locator_carries_neither_key() -> None:
excerpt = _provenance_excerpts()["uten-adresse"]
for key in ("sources", "sources_unreadable", *okf_consume.LOCATOR_KEYS):
assert key not in excerpt, key
def test_an_unreadable_address_is_named_rather_than_dropped(tmp_path: Path) -> None:
# The third state. A `sources` value this reader cannot decode is neither an
# address nor an absence, and collapsing it into either reports a fact
# nobody measured.
root = tmp_path / "bundle"
_copy_bundle(PROVENANCE, root)
target = root / "uten-adresse.md"
target.write_text(
target.read_text(encoding="utf-8").replace(
"bundle_id: provenance-fixture",
"bundle_id: provenance-fixture\nsources: not-a-list",
),
encoding="utf-8",
)
concept = okf_consume.read_concept(
target, bundle_root=root, root_bundle_id="provenance-fixture"
)
excerpt = okf_consume.excerpt_for(concept)
assert excerpt is not None
assert excerpt["sources_unreadable"] is True
assert "sources" not in excerpt
def test_the_golden_bundle_excerpt_carries_the_address_and_locator_o3_wrote() -> None:
payload = okf_consume.build_payload(GOLDEN, question="Hva sier veiledningen om krav?")
excerpts = payload["excerpts"]
assert isinstance(excerpts, list)
by_id = {entry["concept_id"]: entry for entry in excerpts} # type: ignore[index,union-attr]
first = by_id["krav/1-1/foerste-krav"]
assert first["title"] == "Foerste krav"
assert first["sources"] == [{"resource": "veiledning.md", "title": "veiledning.md"}]
assert first["source_lines"] == "[5, 8]"
assert first["source_offset"] == "[94, 176]"
def test_the_checker_refuses_an_excerpt_that_cannot_be_named() -> None:
# The SS 8 update, with its own code: a payload whose excerpts carry no
# `title` is the defect po measured, and a checker that passes it certifies
# a payload a model cannot cite from.
payload = json.loads((SKILL.parent / "references" / "example-payload.json").read_text("utf-8"))
assert okf_contract_check.check(SKILL.read_text(encoding="utf-8"), payload).findings == ()
del payload["excerpts"][0]["title"]
codes = [f.code for f in okf_contract_check.check(SKILL.read_text("utf-8"), payload).findings]
assert codes == ["excerpt_unnamed"]

138
tools/okf_consume.py
View file

@ -262,6 +262,19 @@ class Concept:
#: `False` when the key was absent. `adjudication == "unknown"` already says
#: so, but a separate flag keeps the two facts from being one inference.
adjudication_present: bool
#: The identifier the producer wrote, or `""` when there is no key. The
#: number a lookup question is asked ON, and the one thing a reader needs to
#: name the concept the answer rests on.
req_number: str
#: The SS 5.1 address entries, in either YAML form.
sources: tuple[Mapping[str, str], ...]
#: `True` when a `sources` key was there, whatever this reader made of it.
#: With `sources == ()` that is the third state: present and unreadable.
sources_present: bool
#: The `LOCATOR_KEYS` this concept carries, values as written. Absent keys
#: are absent, never `""`: SS 6.4 forbids reading the absence of a
#: conditionally-written field as the negation of what it asserts.
locators: Mapping[str, str]
frontmatter: Mapping[str, str]
body: str
@ -301,6 +314,7 @@ def read_concept(path: Path, *, bundle_root: Path, root_bundle_id: str) -> Conce
code="adjudication_unknown_value",
)
declared = frontmatter.get("bundle_id")
entries, sources_present = read_sources(_frontmatter_lines(path))
return Concept(
path=path,
concept_id=concept_id,
@ -312,6 +326,12 @@ def read_concept(path: Path, *, bundle_root: Path, root_bundle_id: str) -> Conce
source_file=frontmatter.get("source_file", ""),
adjudication=adjudication,
adjudication_present=raw is not None,
req_number=frontmatter.get("req_number", ""),
sources=entries,
sources_present=sources_present,
locators={
key: frontmatter[key] for key in LOCATOR_KEYS if frontmatter.get(key, "").strip()
},
frontmatter=frontmatter,
body=_body(path),
)
@ -428,6 +448,93 @@ def _split_top_level(body: str, opener: str, closer: str) -> list[str]:
return [part for part in parts if part.strip()]
#: The locator keys O3 (`b6a8c8b`) writes on every SEGMENTED concept, in the
#: order they are emitted so a reader comparing an excerpt against the concept
#: file reads one sequence. The ADDRESS is SPEC SS 5.1's `sources`; these are
#: this library's OWN top-level keys, because SS 5.1 has no field for a place
#: within a resource and the guard rejects every route to putting one inside a
#: `sources` entry. Their VALUES pass through as the frontmatter's own strings:
#: `source_pages: [2, 27]` reaches the payload as `"[2, 27]"`, which is what
#: makes an excerpt greppable against the file it came from.
LOCATOR_KEYS = (
"source_pages",
"source_sheet",
"source_rows",
"source_lines",
"source_offset",
)
def _frontmatter_lines(path: Path) -> list[str]:
"""The raw lines between the two `---` fences, indentation intact.
`parse_frontmatter` SKIPS indented lines on purpose -- a nested `title:`
arriving later would SUBSTITUTE for the document's. That refusal is right
for a flat mapping and it is why the block form has to be read from the
raw lines instead.
"""
lines = path.read_text(encoding="utf-8").splitlines()
if not lines or lines[0].strip() != "---":
return []
block: list[str] = []
for line in lines[1:]:
if line.strip() == "---":
break
block.append(line)
return block
def read_sources(lines: Sequence[str]) -> tuple[tuple[Mapping[str, str], ...], bool]:
"""The SS 5.1 address entries, and whether the key was there at all.
Three states, kept apart because collapsing any two reports something
nobody measured: `((), False)` the concept has no `sources` key; `((), True)`
it has one this reader cannot decode; a non-empty tuple, the entries.
BOTH YAML forms are read, and that is a measurement rather than a
preference. Measured 2026-09-08: K2 writes the flow form on 629 of 629
concepts, the N500 bundle writes the block form on 270 of 270. A reader
handling one form delivers the other bundle with no address at all -- and
for N500 there is nothing else, because it carries zero locator keys.
Reading the block form is not a licence to WRITE it: this library's
line-oriented parser still cannot round-trip block lists, so the emission
rule (flow only) is untouched.
"""
for position, line in enumerate(lines):
if line[:1] in (" ", "\t") or not line.startswith("sources:"):
continue
value = line.partition(":")[2].strip()
if value:
flow = _parse_flow_mappings(value)
if flow is None:
return (), True
return tuple(flow), True
entries: list[dict[str, str]] = []
for nested in lines[position + 1 :]:
if not nested.strip():
continue
if nested[:1] not in (" ", "\t"):
break
item = nested.strip()
if item.startswith("- "):
entries.append({})
item = item[2:].strip()
elif not entries:
# An indented line before any `- ` opens no entry. Refused
# rather than folded into one, which would invent an entry the
# document does not have.
return (), True
key, separator, raw = item.partition(":")
if not separator:
return (), True
entries[-1][key.strip()] = raw.strip()
if not entries:
return (), True
return tuple(entries), True
return (), False
# --- The budget instrument (SS 7) --------------------------------------------
#: SS 7.5 fixes no unit deliberately -- "a token is one encoder family's unit
@ -464,14 +571,14 @@ KNOWN_POSITIVE_CASE = "docs/consumption-contract.md, encoded as a JSON string"
#: `measure()`'s own answer for that file. Vacuous ALONE -- which is why the
#: delta below exists.
KNOWN_POSITIVE_EXPECTED = 10_349
KNOWN_POSITIVE_EXPECTED = 12_049
#: The second, independent route. `wc -c` reports 10 060 raw bytes for the same
#: The second, independent route. `wc -c` reports 11 719 raw bytes for the same
#: file; the difference is this file's JSON quoting and escaping overhead. A
#: reader can derive it without running `measure()` at all, and it moves the
#: moment `measure()` changes what it counts -- which is what stops
#: `expected == measured` from proving nothing.
KNOWN_POSITIVE_ENCODING_DELTA = 289
KNOWN_POSITIVE_ENCODING_DELTA = 330
_KNOWN_POSITIVE_PATH = Path(__file__).resolve().parents[1] / "docs" / "consumption-contract.md"
@ -1027,6 +1134,14 @@ def excerpt_for(concept: Concept) -> dict[str, object] | None:
- **`bundle_id_inherited`** -- whether the first half of the SS 3.1 identity
tuple came from the concept or from the root index.
And, since C1 step 0, the keys that let a reader NAME what it is citing.
`title` is unconditional (SS 8, this revision); `req_number`, `sources` and
each locator are written only when the concept carries them, because a key
with an empty value asserts that the producer wrote one. po measured
2026-09-08 (`e7ffe9e`) that the pre-pass delivered the gold concept at rank
1 on three bundles while the model could not name it: the ranking found the
document and the delivery dropped the key.
Trailing whitespace is stripped per line: a spreadsheet render is padded to
hundreds of trailing spaces per line, and unstripped, most of a budget goes
on padding.
@ -1037,16 +1152,27 @@ def excerpt_for(concept: Concept) -> dict[str, object] | None:
text = "\n".join(
line.rstrip() for line in unicodedata.normalize("NFC", concept.body).split("\n")
)
return {
excerpt: dict[str, object] = {
"bundle_id": concept.bundle_id,
"concept_id": concept.concept_id,
"sha256": concept.sha256,
"adjudication": concept.adjudication,
"trust_tier": tier,
"bundle_id_inherited": concept.bundle_id_inherited,
"text_sha256": hashlib.sha256(text.encode("utf-8")).hexdigest(),
"text": text,
"title": concept.title,
}
if concept.req_number:
excerpt["req_number"] = concept.req_number
if concept.sources:
excerpt["sources"] = [dict(entry) for entry in concept.sources]
elif concept.sources_present:
# The third state, written rather than silently dropped: the concept
# HAS an address and this reader could not decode it.
excerpt["sources_unreadable"] = True
excerpt.update(concept.locators)
excerpt["text_sha256"] = hashlib.sha256(text.encode("utf-8")).hexdigest()
excerpt["text"] = text
return excerpt
def excerpt_weight(excerpt: Mapping[str, object]) -> int:

24
tools/okf_contract_check.py
View file

@ -187,6 +187,29 @@ def rule_excerpt_source_marking(ctx: Context) -> list[Finding]:
return findings
def rule_excerpt_named(ctx: Context) -> list[Finding]:
"""SS 8: every excerpt carries a `title`.
Added 2026-09-08 on a measurement, not a preference: `portfolio-optimiser`
ran three paid arms in which the pre-pass delivered the gold concept at rank
1 of 8 on 3 of 3 bundles and the model answered correctly on 1 of 3, because
the excerpt carried `concept_id` and `text` and nothing a reader could name
the document by. A payload no answer can cite from is not conformant; the
identity fields are what SS 3.1's tuple is FOR.
"""
if not ctx.payload_is_mapping:
return []
return [
Finding(
"excerpt_unnamed",
f"excerpt {position} carries no 'title'; an excerpt a reader cannot "
"name is one an answer cannot cite, whatever its rank (SS 8)",
)
for position, raw in enumerate(_sequence(ctx.payload.get("excerpts")))
if not _text(_mapping(raw).get("title"))
]
def rule_excerpt_states(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
@ -407,6 +430,7 @@ RULES: tuple[Callable[[Context], list[Finding]], ...] = (
rule_contract_version,
rule_bundle_ref,
rule_excerpt_source_marking,
rule_excerpt_named,
rule_excerpt_states,
rule_denominator_identity,
rule_denominator_lists,