llm-ingestion-okf/docs/plan/okf-bundle-consumption-contract.md

The generic OKF bundle consumption contract

Status: plan. Nothing here is implemented, and nothing here asks to be. No module, no profile field, no version bump follows from this document. It fixes what the contract is and who owns it, so that the first bundle-specific consumption skill — written later, in whichever repo needs it — has something to be conformant to.

Provenance. Operator decision 2026-08-25, relayed through .claude as PM (order 20260825T122454Z-4632317874-from-.claude). The reason ownership landed here is stated in the order and is the same reason Phase 3 exists: the production side of the bundle contract is already carried by a profile rather than by constants in this repo (README, "Upstream OKF versions"). A consumption contract that did not share profile identity with the production side would drift from it.

Second order, 2026-08-25 (20260825T133057Z-193098605-from-.claude), carrying an operator directive verbatim: how an AI agent finds relevant information in a bundle must be token-efficient, and this is a property the contract carries rather than an optimisation that can arrive later. That order prescribed the goal and left the form to us, including the invitation to say so if "token-efficiency" is the wrong abstraction for a generic layer. It is, for three reasons; C6 is the property that was built instead.

This repo does not own any bundle. It owns the contract both sides of a bundle are held to. That distinction is the whole of §2.

---

0. What was measured before this was written

The order supplied figures and line numbers and asked that they be re-measured rather than copied. They were. One of them does not survive.

Claim in the order	Measured	Verdict
This repo's README says the bundle contract is carried "by a profile rather than by constants"	README.md l. 12–13 (the order says 11–13; the range contains it)	holds
The wiki masterplan fixes three marking values (extracted / derived / [uverifiable-from-bundle]), decision 2026-07-24, attributed dead	docs/masterplan.md l. 535 and l. 1027	holds
The wiki bundle's consumption engine documents six equal markings	its README.md l. 24, CLAUDE.md l. 56–57 (the order says 57; the heading is on 56), skills/advise/SKILL.md l. 118–123	holds
The 2026-08-13 ratification claim "originates from the consumer's own log, not from the source it cites" — evidenced by grep -rln '2026-08-13' <wiki>/docs/*.md returning nothing	see below	does not hold

The failing measurement, and why. The grep is correct and its result is correct: zero hits, exit 1, against a denominator of 16 files in docs/. But the repository holds 639 .md files, and the query saw 16 of them. Re-run across the whole tree, 2026-08-13 appears in 9 files — among them the wiki's own STATE.md, which is tracked and committed (2026-08-14) in the source repository, not a log kept by the consumer. It records the operator decision of 2026-08-13 by item id: F.10 and F.11 as ratified, G.13 as approved — G.13 being, in its own words, a new vocabulary value for "checked in the target repo, source outside the bundle", explicitly distinct from derived. The ratification is real, it happened where the consumer says it happened, and the consumer cited it accurately.

So the correction runs the other way, and .claude invited exactly this: the absence was a property of the query's scope, not of the world. A control run (2026-07-24, same scope) returns three files, so the query could find — it was pointed at the wrong 2.5 % of the tree.

What is nonetheless true, and is the real defect. docs/masterplan.md was last modified 2026-07-25. The document that the consumption engine names as the requirements owner still states three values and has never been updated to carry the decision that superseded it. The decision lives in a session-state file; the normative document does not know about it. That is a documentation lag in another repo — reportable, not ours to fix (§5).

One further distinction the order's framing flattens, and which turns out to carry weight in §3: the 08-13 decision did not ratify six values in one motion. F.11 ratified a five-value set ([unread] and [sourced-not-sufficient] moving from provisional to settled). G.13 approved a sixth as a separate item, for a case defined by being outside the bundle. The upstream record already splits along the seam this document draws.

0.1 What was measured for the second revision

The second order supplied four claims and asked, as the first did, that they be re-measured. All four hold; a fifth thing was measured that the order did not know, and it changes §2.

Every measurement in this table was taken against this document at d205b70, before the revision it prompted. The ref is stated because C2 requires it and because the first row is a claim about a file the same commit then changed: re-running that grep at HEAD now returns dozens of hits, and without the ref the row would read as false rather than as superseded.

Claim in the order	Command	Verdict
This document says nothing about what a read costs	grep -in 'token|kostnad|cost|budget|effektiv|context window|kontekstvindu' over the file, exit 0	holds — one hit, l. 245, on an unrelated sense of "costs"
A consuming repo measured one imported base at 250 785 tokens, three bases at 430 144, and one listing call at 112 116	the measurement document at the named commit, read read-only	holds
That instrument was validated against three known-good figures before its own numbers were consumed	same document	holds — and it is C3's second corollary applied to a positive quantity
The reading model navigates rather than searches: start at the index, follow the links, read each file whole	the reading-model analysis §9.1, quoted	holds

The fifth measurement was prompted by a coordination message from the producing repo, which reported the flat-index consequence from its own side and asked a contract question about IndexPolicy.per_directory. Its three claims about this library's source were checked line by line and all three hold. What the message could not see is that per_directory is a judging field rather than an emitting one, which is recorded in §2 because it changes what a consumption profile may assume.

---

1. What the generic contract consists of

Bundle-independent means: true of any OKF bundle, whatever its types, depth, index shape or subject. Anything that needs to name a type, a frontmatter key or a directory is not in this layer — it is in a consumption profile (§2).

Six elements. Four are the first order's; the fifth was found in the sources it sent us to read first; the sixth arrived on a second order and is marked, along with the name it did not keep.

C1 — Source marking on every claim

Every claim an agent emits carries exactly one marking from the required set, plus a pointer to where it came from. The set, and the extension rule, are §3.

C2 — Pin anchoring: an advisory names the ref it read, not the version

A consuming agent states the bundle ref — commit or equivalent content identity — that it read, not the bundle's declared version. This repo has paid for the difference twice and recorded both: a tree can lose its identity without the version moving (upstream OKF, measured 2026-08-23), and a measured fact can go stale inside the same session. A version string is an assertion by the producer; a ref is a fact about bytes. An advisory anchored to a version cannot be re-measured later, which makes every claim in it unfalsifiable the moment the bundle moves.

C3 — Denominator reporting

Any claim of the form "there is no X", "nothing further was found", or "all N are Y" reports the denominator: how many units were actually examined, by which command. A negative result whose scope is unstated is not zero — it is unmeasured, and must be reported as such. §0 of this document is an instance of the failure this element exists to prevent, which is the strongest argument available for making it a required element rather than advice.

Two corollaries a conformant skill must satisfy:

• The query that produced a negative result is validated against a known-positive case, so it is shown capable of finding.
• Exit status is read from the command that matters. A pipeline reports the status of its last stage, so grep … | head; echo $? measures head.

C4 — Division of labour: the script cuts, the agent judges

A consumption skill is two parts and the boundary between them is normative. A deterministic part reads, ranks and cuts the bundle to a bounded context. The agent judges what it was given, and does not go looking for context that was deliberately held back. The form is taken from the wiki bundle's consumption engine, which states this division verbatim in its own skill.

The obligation this creates runs the other way and is the part that is easy to drop: the cut must be declared, not silent. An agent that was handed a bounded window and reports as if it had read the bundle is producing C3's failure with extra steps. This is precisely what [unread] is for, which is why C1 and C4 are not independent — the marking set is the receipt for the cut.

C5 — Conditional fields: absence is a measurement, not a fact (addition)

Beyond the order's four; grounded in a measurement in the second corpus's consumption profile, and generic on inspection.

Every bundle has fields that are written only when some build-time condition held. In the second corpus, a supersession field is written only when the build was given a predecessor document, and six of seven source documents are built without one. An agent reading that field's absence as "nothing supersedes this" would be confidently wrong; the truthful reading is "this build did not check".

Generically: a consuming agent must not read the absence of a frontmatter field as the negation of what the field asserts, unless the profile says the field is unconditional. The generic obligation is therefore on the profile: it enumerates its conditionally-written fields and states, for each, what absence does and does not mean. Without that enumeration, an agent has no way to tell a conditional field from an unconditional one, and the failure is silent.

C6 — Retrieval cost is declared: instrument, budget, denominator, breaking point (second order)

Beyond the first order's four and the addition above. The operator directive this element carries used the words "token-efficient"; the element does not, and the reasons are the first thing it has to say.

Why not "token efficiency". Three objections, each one this document already runs on elsewhere:

• A token is one encoder family's unit. One vendor's encoding is not what another vendor's reader counts, and a generic contract that fixed the unit would adopt one vendor's arithmetic as everyone's — the same move §3 refuses when it declines to adopt one consumer's marking set as the required set. What survives the vendor is a named instrument and a stated number, never the instrument's name.
• "Efficient" is a comparative with the denominator missing. Efficient against what? C3 exists because a claim whose scope is unstated is not a measurement; an unanchored efficiency claim is that same failure with the sign flipped, and it is harder to catch because it sounds like good news.
• Efficiency is a property of a run; a contract binds a document. No contract can oblige a skill to be cheap. It can oblige the skill to state what its read cost, so that expense is visible instead of silent — which is the precondition for anyone making it cheap, and the thing whose absence is the actual defect.

The property, stated. The cost of answering a question should track the question, not the corpus. A retrieval strategy whose cost is the same whatever is asked has not cut anything; it has deferred the whole corpus into every run.

This is not a prohibition on reading a whole bundle. For a small corpus, reading everything is the cheapest correct strategy, and a rule forbidding it would be wrong. What the contract forbids is doing so undeclared. The obligation is therefore the same shape as C4's: the cut must be declared, and C6 is the half of that declaration which carries a number.

What a conformant consumption skill declares. Three things, none of which requires a particular unit, engine or bundle shape:

1. A budget and the instrument that measures against it. The bounded context the cut targets, as a number, in a unit named together with the tool that produced it. "Bounded" without a bound is C3's failure in prose.
2. The cost of the cut against the whole-bundle denominator. What a typical answer costs to assemble, and what reading the entire bundle at that ref would cost, by the same instrument. The fraction held back is what makes [unread] (C1, C4) a quantity rather than a gesture.
3. Which way its cost scales, and the breaking point. Whether cost tracks the question or the corpus, and — stated as a number — the corpus size at which the declared strategy stops fitting the declared budget. A strategy with no stated breaking point cannot be observed to have passed it.

The instrument is validated before its numbers are believed. C3's second corollary — a query is shown capable of finding before its negative result is consumed — transfers unchanged to a positive quantity: a cost instrument reproduces known-good figures before its own are used. This is not a hypothetical discipline. The measurement that prompted this element did exactly that, reproducing three published reference figures exactly before any new number was consumed, and it is cited here as form, never as authority.

The failure this exists to prevent, measured. A corpus of three imported bases was measured at 430 144 tokens in one encoding, one base alone at 250 785 — which is more than a 128k-context reader can hold at all — and a single listing call at 112 116. None of those numbers was wrong, hidden or anyone's bug. They were simply never stated, because nothing in any contract required them to be, and the reading model in play reads every linked file whole, so a flat index of several hundred links is several hundred whole files in every run. The cost was a fact about the corpus long before it was a finding, and it became a finding only because somebody happened to measure it.

What this element does not reach. C6 binds the reading side: what an agent must be able to say about its own retrieval. Whether a bundle is shaped so that a cheap cut is available — nested indexes, branch granularity, section splitting — is the producer's question, is being decided under a separate order in the repo that owns that corpus, and is deliberately not decided here. The two halves meet at the ref (C2): the producer changes the shape, and the consumption profile's cost figures are re-measured against the new ref rather than carried forward.

---

2. How this relates to the profile mechanism we already own

Decision: a parallel document, bound to the production profile by shared identity and a pin. Not a field on BundleProfile.

The pull toward one object is real and is the order's stated reason for placing ownership here — two documents drift. It is nonetheless the wrong shape, for three measured reasons:

1. BundleProfile is a construction-time gate in one runtime. It refuses illegal profiles at construction (TypePolicy.__post_init__ refuses the reserved layer; the schema refuses the timestamp/generated pair). A consumption rule cannot be enforced that way — it is judged by an agent, at read time, possibly with no Python present at all. The wiki bundle's engine is Node; Phase 4 of this repo is Node. A contract that only exists where our Python package is installed is not the generic layer.
2. The profile deliberately excludes judgement vocabulary. profiles.py states that security is the guard's and that no disposition, origin or channel vocabulary belongs on a profile. Marking vocabulary is the same category of thing: it describes an agent's epistemic position, not a bundle's shape.
3. The production profile is about bytes; the consumption contract is about reading. Nothing in C1–C5 can be expressed as "which keys are emitted, in which order".

How drift is prevented instead — identity plus pin, not colocation. A consumption profile:

• names the production profile it reads (DEFAULT, STRICT_V1, or the named profile a corpus builds under), so the two halves share one identity;
• names the bundle ref it was measured against (C2), so every structural claim it makes has a stated as-of;
• re-derives its structural claims by command, never by restating the production profile's fields in prose. A consumption profile that says "concepts live at the root, one index" must have measured it against a bundle built by the named production profile. Prose copied from the other half is the drift; a command is not copyable and goes red when it stops being true.

One binding already exists and must be honoured. IndexPolicy carries entries_match_directory: whether the index is derived from the directory or authored. The two consumers hold opposite postures on this and neither is a library invariant. The consumption side inherits it unchanged: a consuming agent enumerates a directory only where the named profile says the index is derived. Where the index is authored, the directory is not the index's denominator and a listing is not a check.

That rule was written as a correctness rule — a listing is not a check where the index is authored. It is also a cost rule, because it bounds what the agent reads, and naming both readings is what keeps it from being discarded as pedantry by someone optimising (C6).

A second binding, measured for this revision, and it cuts against the optimistic reading. IndexPolicy also carries per_directory — "an index at every level, or only at the bundle root" — off by default. It is a judging field: it is read by required_indexes and violations, which tell a caller what to require and what to flag. No code path in this library emits a nested index. The materializer refuses STRICT_V1 for exactly this reason, in its own words: that profile "sets per_directory and entries_match_directory, neither of which this materializer honours". The external-bundle door writes one root index and flattens each merged concept into the root namespace.

Two consequences for a consumption profile:

• A permitted shape is not an emitted shape. That a named profile allows nesting says nothing about whether the bundle in front of the agent has any. "This bundle is nested" is a structural claim and must be re-derived by command against the ref, never read off the profile's field list — which is the third bullet above, in the one case where the temptation is strongest.
• A cost figure inherits the ref, not the profile. Because shape can change without the profile changing, a consumption profile's C6 numbers are anchored to the ref they were measured at, and go stale when it moves.

This is the same rule Phase 3 already runs on, one level up: a profile names the key, a caller owns the value. Here: the generic contract names the required marking set; a consumption profile declares its extensions.

---

3. The source-marking vocabulary, decided

Decision: five required values, plus a declared-extension mechanism. [verified-in-target] is an extension, not a required value.

Required in every conformant consumption skill, whatever the bundle:

Marking	Meaning
extracted	the bundle states it directly
derived	inferred from the bundle; the reasoning is shown
[unverifiable-from-bundle]	outside what the bundle covers
[unread]	the source exists in the bundle and was not read
[sourced-not-sufficient]	the quote is real but does not carry the conclusion

[unverifiable-from-bundle] is one literal string — no variants, no translations — and that English spelling is the ratified wire literal (item F.10). A repo may keep a translated concept name in its own prose; the wire form does not translate.

Which document is normative — and why the question dissolves

Neither. The order asks us to pick between the wiki masterplan (three values) and the wiki bundle's consumption engine (six). Both are artefacts of one consumer: the masterplan owns the vocabulary for that bundle, the engine implements it for that bundle. Deciding that either is normative for the generic layer would be adopting one consumer's contract as everyone's — the mistake this repo has a name for (O2: we do not change, or inherit, another repo's contract from here).

Taking ownership of the generic layer means the generic set is ours, and both of those documents become conformant profiles of it. Neither has to change. That is also why the masterplan's documentation lag (§0) does not block this decision: it is a defect in how one consumer records its own contract, and the generic layer does not depend on which of its two documents wins.

Why exactly these five, and why the sixth is not among them

The line is drawn by a property, not by counting: a required value must be meaningful for every bundle. [verified-in-target] means "read at its source in the target repo the advisory is about" — it presupposes a target outside the bundle that the agent can read. The second corpus measured a case where that presupposition fails: its consumers advise about a physical installation or an exemption application, not about a codebase, so there is no target to verify in. Its profile also rejects the nearest-looking substitute — fields verified by a human curator at build time are not verified by the agent at consume time; those are two different acts and collapsing them would launder one into the other.

A generic contract that required all six would force a marking onto bundles where it can only be misapplied. Requiring five and letting the sixth be declared costs nothing and is exactly how the upstream record already reads: F.11 settled five; G.13 approved the sixth separately, for a case defined by being outside the bundle.

The extension rule

A consumption profile may add markings. It must declare them: the literal, what it means for that bundle, and which required value it would otherwise collapse into. An undeclared extension is the defect, not the extension — [verified-in-target] itself exists because a fact read at its source was falling to derived, reporting a checked fact as an inferred one.

Under this rule, both existing sets are conformant as they stand: the wiki engine is five plus a declared [verified-in-target]; the second corpus is five with no extension and an explicit statement of why the sixth does not transfer.

---

4. What a bundle-specific consumption skill must deliver to be conformant

A checklist, not a design. Each item is checkable by reading the skill — and items 2, 7 and 8 are checkable by re-running the command the skill states, which is the point of requiring a command rather than a claim.

1. Names its production profile and bundle ref. Which profile the bundle is built under, and the ref the profile's structural claims were measured against (C2, §2).
2. States its structural claims as commands, with denominators. File counts, type breakdown, index shape and depth, each with the command that produced it and the total it is a fraction of (C3, §2).
3. States the retrieval unit and what would falsify it. Which unit is fetched (one concept, one section, one release), and the concrete observation that would show the choice wrong. A retrieval hypothesis with no falsifier is a preference.
4. Enumerates conditionally-written fields and, for each, what absence does and does not mean (C5).
5. Declares its marking set: the required five verbatim, plus any extension with its meaning and the required value it would otherwise collapse into (C1, §3).
6. Draws the cut boundary explicitly: what the deterministic part reads, ranks and cuts; what the agent is forbidden to go fetch; and how the cut is surfaced in output as [unread] rather than passing silently (C4).
7. Declares a retrieval budget and names the instrument. The bound the cut targets, as a number in a stated unit, with the tool that measures it and the known-good figures that tool was shown to reproduce first (C6).
8. States its cost against the whole-bundle denominator, and its breaking point. What a typical answer costs to assemble, what the whole bundle at that ref would cost by the same instrument, whether cost tracks the question or the corpus, and the corpus size at which the strategy stops fitting the budget (C6). A skill that cannot state these has not measured its own cut.
9. Enumerates a directory only where the named profile says the index is derived (§2) — a correctness rule and a cost rule at once.
10. Carries its own quality claim honestly. If no run has been scored against its checklist, it says so. The one implemented precedent we have is explicitly unmeasured by its own admission, and was used here as a model for form, never as evidence that the form works.

---

5. What this does not decide, and what is owed elsewhere

• No skill is designed or built here, generic or bundle-specific. This document has no implementation follow-on queued.
• Where the first generic implementation would live — this repo's Python half, the Node half, or a skill — is untouched. Ownership of the contract does not carry ownership of an engine, and no consumer has asked for one.
• Not ours to fix, to be reported by coord: the wiki masterplan states three markings and has not been updated with the 2026-08-13 decision that settled five and approved a sixth. The decision is recorded in that repo's tracked state file; the document its own consumption engine names as requirements owner does not carry it. Reported, not edited (O2).
• .claude's premise correction (§0) is owed back as a coord message: the ratification exists in the source repo; the query that found it absent was scoped to 16 of 639 files. (Sent; the correction is accepted and recorded upstream.)
• Bundle shape is not decided here. C6 binds what a reading agent must be able to say about its retrieval. Whether a corpus is nested, split or branched so that a cheap cut exists at all belongs to the repo that produces it, and is under a separate order there. Deciding it from here would be O2.
• No instrument is named, shipped or blessed. C6 requires that a skill name the instrument it used and show it reproducing known-good figures. Which tool that is stays the skill's choice, because the unit is vendor-specific and fixing one would be the mistake C6's first paragraph refuses.
• No cost engine, ranker or cutter is designed here, in either runtime. C6 is a disclosure obligation on a document, not a retrieval algorithm, and nothing about how to achieve a cheap cut follows from it.