open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

docs(plan): take ownership of the generic OKF bundle consumption contract

Browse source 
Plan work only, per operator decision 2026-08-25 relayed through .claude
(order 20260825T122454Z-4632317874). No code, no module, no version bump.

Fixes four things a bundle-specific consumption skill can be held to:

- The contract's five bundle-independent elements: source marking, pin
  anchoring by ref rather than version, denominator reporting, the
  script-cuts/agent-judges division of labour, and (added here, grounded
  in a measurement in a second corpus) conditionally-written fields,
  whose absence is a measurement rather than a fact.
- Architecture: a parallel document, not a field on BundleProfile. The
  profile is a construction-time gate in one runtime and deliberately
  excludes judgement vocabulary. Drift is prevented by shared profile
  identity plus a pin plus a requirement that structural claims be
  re-measured by command, never restated in prose. IndexPolicy's
  entries_match_directory binds the consumption side unchanged.
- The source-marking vocabulary: five required values plus a declared
  extension mechanism. [verified-in-target] is an extension, not a
  required value, because it presupposes a target outside the bundle
  and a measured corpus exists where that presupposition fails.
- A conformance checklist for a bundle-specific consumption skill.

The order's premise that the 2026-08-13 ratification was unfounded does
not survive re-measurement: the supporting grep was scoped to 16 of 639
files and missed the tracked state file that records the decision. The
real defect is narrower and belongs to another repo — the document its
own engine names as requirements owner has not been updated. Reported
by coord, not edited (O2).
This commit is contained in:
Kjell Tore Guttormsen 2026-08-25 15:06:51 +02:00
commit d205b707da
1 changed files with 308 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

308
docs/plan/okf-bundle-consumption-contract.md Normal file
View file

@ -0,0 +1,308 @@
# The generic OKF bundle consumption contract
**Status: plan. Nothing here is implemented, and nothing here asks to be.**
No module, no profile field, no version bump follows from this document. It
fixes what the contract *is* and who owns it, so that the first bundle-specific
consumption skill — written later, in whichever repo needs it — has something
to be conformant to.
**Provenance.** Operator decision 2026-08-25, relayed through `.claude` as PM
(order `20260825T122454Z-4632317874-from-.claude`). The reason ownership landed
here is stated in the order and is the same reason Phase 3 exists: the
production side of the bundle contract is already carried *by a profile rather
than by constants* in this repo (README, "Upstream OKF versions"). A consumption
contract that did not share profile identity with the production side would
drift from it.
This repo does not own any *bundle*. It owns the contract both sides of a bundle
are held to. That distinction is the whole of §2.
---
## 0. What was measured before this was written
The order supplied figures and line numbers and asked that they be re-measured
rather than copied. They were. One of them does not survive.
| Claim in the order | Measured | Verdict |
|---|---|---|
| This repo's README says the bundle contract is carried "by a profile rather than by constants" | `README.md` l. 12–13 (the order says 11–13; the range contains it) | **holds** |
| The wiki masterplan fixes three marking values (`extracted` / `derived` / `[uverifiable-from-bundle]`), decision 2026-07-24, `attributed` dead | `docs/masterplan.md` l. 535 and l. 1027 | **holds** |
| The wiki bundle's consumption engine documents six equal markings | its `README.md` l. 24, `CLAUDE.md` l. 56–57 (the order says 57; the heading is on 56), `skills/advise/SKILL.md` l. 118–123 | **holds** |
| The 2026-08-13 ratification claim "originates from the consumer's own log, not from the source it cites" — evidenced by `grep -rln '2026-08-13' <wiki>/docs/*.md` returning nothing | see below | **does not hold** |
**The failing measurement, and why.** The `grep` is correct and its result is
correct: zero hits, exit 1, against a denominator of 16 files in `docs/`. But
the repository holds 639 `.md` files, and the query saw 16 of them. Re-run
across the whole tree, `2026-08-13` appears in 9 files — among them the wiki's
own `STATE.md`, which is **tracked and committed** (2026-08-14) in the source
repository, not a log kept by the consumer. It records the operator decision of
2026-08-13 by item id: `F.10` and `F.11` as *ratified*, `G.13` as *approved* —
`G.13` being, in its own words, a new vocabulary value for "checked in the
target repo, source outside the bundle", explicitly distinct from `derived`.
The ratification is real, it happened where the consumer says it happened, and
the consumer cited it accurately.
So the correction runs the other way, and `.claude` invited exactly this: the
absence was a property of the query's scope, not of the world. A control run
(`2026-07-24`, same scope) returns three files, so the query could find — it was
pointed at the wrong 2.5 % of the tree.
**What is nonetheless true, and is the real defect.** `docs/masterplan.md` was
last modified 2026-07-25. The document that the consumption engine names as the
requirements owner still states three values and has never been updated to
carry the decision that superseded it. The decision lives in a session-state
file; the normative document does not know about it. That is a documentation
lag in another repo — reportable, not ours to fix (§5).
One further distinction the order's framing flattens, and which turns out to
carry weight in §3: the 08-13 decision did not ratify six values in one motion.
`F.11` ratified a **five**-value set (`[unread]` and `[sourced-not-sufficient]`
moving from provisional to settled). `G.13` *approved* a **sixth** as a separate
item, for a case defined by being outside the bundle. The upstream record
already splits along the seam this document draws.
---
## 1. What the generic contract consists of
Bundle-independent means: true of any OKF bundle, whatever its types, depth,
index shape or subject. Anything that needs to name a type, a frontmatter key or
a directory is not in this layer — it is in a consumption profile (§2).
Five elements. Four are the order's; the fifth was found in the sources it sent
us to read first, and is marked as an addition.
### C1 — Source marking on every claim
Every claim an agent emits carries exactly one marking from the required set,
plus a pointer to where it came from. The set, and the extension rule, are §3.
### C2 — Pin anchoring: an advisory names the ref it read, not the version
A consuming agent states the bundle **ref** — commit or equivalent content
identity — that it read, not the bundle's declared version. This repo has paid
for the difference twice and recorded both: *a tree can lose its identity
without the version moving* (upstream OKF, measured 2026-08-23), and *a measured
fact can go stale inside the same session*. A version string is an assertion by
the producer; a ref is a fact about bytes. An advisory anchored to a version
cannot be re-measured later, which makes every claim in it unfalsifiable the
moment the bundle moves.
### C3 — Denominator reporting
Any claim of the form "there is no X", "nothing further was found", or "all N
are Y" reports the denominator: how many units were actually examined, by which
command. A negative result whose scope is unstated is not zero — it is
**unmeasured**, and must be reported as such. §0 of this document is an instance
of the failure this element exists to prevent, which is the strongest argument
available for making it a required element rather than advice.
Two corollaries a conformant skill must satisfy:
- The query that produced a negative result is validated against a
known-positive case, so it is shown capable of finding.
- Exit status is read from the command that matters. A pipeline reports the
status of its **last** stage, so `grep … | head; echo $?` measures `head`.
### C4 — Division of labour: the script cuts, the agent judges
A consumption skill is two parts and the boundary between them is normative.
A deterministic part reads, ranks and **cuts** the bundle to a bounded context.
The agent judges what it was given, and does **not** go looking for context that
was deliberately held back. The form is taken from the wiki bundle's consumption
engine, which states this division verbatim in its own skill.
The obligation this creates runs the other way and is the part that is easy to
drop: **the cut must be declared, not silent.** An agent that was handed a
bounded window and reports as if it had read the bundle is producing C3's
failure with extra steps. This is precisely what `[unread]` is for, which is why
C1 and C4 are not independent — the marking set is the receipt for the cut.
### C5 — Conditional fields: absence is a measurement, not a fact (addition)
*Beyond the order's four; grounded in a measurement in the second corpus's
consumption profile, and generic on inspection.*
Every bundle has fields that are written only when some build-time condition
held. In the second corpus, a supersession field is written only when the build
was given a predecessor document, and six of seven source documents are built
without one. An agent reading that field's absence as "nothing supersedes this"
would be confidently wrong; the truthful reading is "this build did not check".
Generically: a consuming agent must not read the absence of a frontmatter field
as the negation of what the field asserts, unless the profile says the field is
unconditional. The generic obligation is therefore on the *profile*: it
enumerates its conditionally-written fields and states, for each, what absence
does and does not mean. Without that enumeration, an agent has no way to tell a
conditional field from an unconditional one, and the failure is silent.
---
## 2. How this relates to the profile mechanism we already own
**Decision: a parallel document, bound to the production profile by shared
identity and a pin. Not a field on `BundleProfile`.**
The pull toward one object is real and is the order's stated reason for placing
ownership here — two documents drift. It is nonetheless the wrong shape, for
three measured reasons:
1. **`BundleProfile` is a construction-time gate in one runtime.** It refuses
illegal profiles at construction (`TypePolicy.__post_init__` refuses the
reserved layer; the schema refuses the `timestamp`/`generated` pair). A
consumption rule cannot be enforced that way — it is judged by an agent, at
read time, possibly with no Python present at all. The wiki bundle's engine
is Node; Phase 4 of this repo is Node. A contract that only exists where our
Python package is installed is not the generic layer.
2. **The profile deliberately excludes judgement vocabulary.** `profiles.py`
states that security is the guard's and that no disposition, origin or
channel vocabulary belongs on a profile. Marking vocabulary is the same
category of thing: it describes an agent's epistemic position, not a bundle's
shape.
3. **The production profile is about bytes; the consumption contract is about
reading.** Nothing in C1–C5 can be expressed as "which keys are emitted, in
which order".
**How drift is prevented instead — identity plus pin, not colocation.** A
consumption profile:
- **names the production profile it reads** (`DEFAULT`, `STRICT_V1`, or the
named profile a corpus builds under), so the two halves share one identity;
- **names the bundle ref** it was measured against (C2), so every structural
claim it makes has a stated as-of;
- **re-derives its structural claims by command**, never by restating the
production profile's fields in prose. A consumption profile that says
"concepts live at the root, one index" must have measured it against a bundle
built by the named production profile. Prose copied from the other half is the
drift; a command is not copyable and goes red when it stops being true.
**One binding already exists and must be honoured.** `IndexPolicy` carries
`entries_match_directory`: whether the index is derived from the directory or
authored. The two consumers hold opposite postures on this and neither is a
library invariant. The consumption side inherits it unchanged: **a consuming
agent enumerates a directory only where the named profile says the index is
derived.** Where the index is authored, the directory is not the index's
denominator and a listing is not a check.
This is the same rule Phase 3 already runs on, one level up: **a profile names
the key, a caller owns the value.** Here: the generic contract names the
required marking set; a consumption profile declares its extensions.
---
## 3. The source-marking vocabulary, decided
**Decision: five required values, plus a declared-extension mechanism.
`[verified-in-target]` is an extension, not a required value.**
Required in every conformant consumption skill, whatever the bundle:
| Marking | Meaning |
|---|---|
| `extracted` | the bundle states it directly |
| `derived` | inferred from the bundle; the reasoning is shown |
| `[unverifiable-from-bundle]` | outside what the bundle covers |
| `[unread]` | the source exists in the bundle and was not read |
| `[sourced-not-sufficient]` | the quote is real but does not carry the conclusion |
`[unverifiable-from-bundle]` is one literal string — no variants, no
translations — and that English spelling is the ratified wire literal
(item `F.10`). A repo may keep a translated *concept name* in its own prose; the
wire form does not translate.
### Which document is normative — and why the question dissolves
Neither. The order asks us to pick between the wiki masterplan (three values)
and the wiki bundle's consumption engine (six). Both are artefacts of **one**
consumer: the masterplan owns the vocabulary *for that bundle*, the engine
implements it *for that bundle*. Deciding that either is normative for the
generic layer would be adopting one consumer's contract as everyone's — the
mistake this repo has a name for (O2: we do not change, or inherit, another
repo's contract from here).
Taking ownership of the generic layer means the generic set is ours, and both of
those documents become **conformant profiles** of it. Neither has to change.
That is also why the masterplan's documentation lag (§0) does not block this
decision: it is a defect in how one consumer records its own contract, and the
generic layer does not depend on which of its two documents wins.
### Why exactly these five, and why the sixth is not among them
The line is drawn by a property, not by counting: **a required value must be
meaningful for every bundle.** `[verified-in-target]` means "read at its source
in the target repo the advisory is about" — it presupposes a target outside the
bundle that the agent can read. The second corpus measured a case where that
presupposition fails: its consumers advise about a physical installation or an
exemption application, not about a codebase, so there is no target to verify in.
Its profile also rejects the nearest-looking substitute — fields verified by a
human curator at **build** time are not verified by the agent at **consume**
time; those are two different acts and collapsing them would launder one into
the other.
A generic contract that required all six would force a marking onto bundles
where it can only be misapplied. Requiring five and letting the sixth be
declared costs nothing and is exactly how the upstream record already reads:
`F.11` settled five; `G.13` approved the sixth separately, for a case defined by
being outside the bundle.
### The extension rule
A consumption profile may add markings. It must **declare** them: the literal,
what it means for that bundle, and which required value it would otherwise
collapse into. An undeclared extension is the defect, not the extension —
`[verified-in-target]` itself exists because a fact read at its source was
falling to `derived`, reporting a checked fact as an inferred one.
Under this rule, both existing sets are conformant as they stand: the wiki
engine is five plus a declared `[verified-in-target]`; the second corpus is five
with no extension and an explicit statement of why the sixth does not transfer.
---
## 4. What a bundle-specific consumption skill must deliver to be conformant
A checklist, not a design. Each item is checkable by reading the skill.
1. **Names its production profile and bundle ref.** Which profile the bundle is
built under, and the ref the profile's structural claims were measured
against (C2, §2).
2. **States its structural claims as commands, with denominators.** File counts,
type breakdown, index shape and depth, each with the command that produced it
and the total it is a fraction of (C3, §2).
3. **States the retrieval unit and what would falsify it.** Which unit is
fetched (one concept, one section, one release), and the concrete observation
that would show the choice wrong. A retrieval hypothesis with no falsifier is
a preference.
4. **Enumerates conditionally-written fields** and, for each, what absence does
and does not mean (C5).
5. **Declares its marking set**: the required five verbatim, plus any extension
with its meaning and the required value it would otherwise collapse into
(C1, §3).
6. **Draws the cut boundary explicitly**: what the deterministic part reads,
ranks and cuts; what the agent is forbidden to go fetch; and how the cut is
surfaced in output as `[unread]` rather than passing silently (C4).
7. **Enumerates a directory only where the named profile says the index is
derived** (§2).
8. **Carries its own quality claim honestly.** If no run has been scored against
its checklist, it says so. The one implemented precedent we have is
explicitly unmeasured by its own admission, and was used here as a model for
*form*, never as evidence that the form works.
---
## 5. What this does not decide, and what is owed elsewhere
- **No skill is designed or built here**, generic or bundle-specific. This
document has no implementation follow-on queued.
- **Where the first generic implementation would live** — this repo's Python
half, the Node half, or a skill — is untouched. Ownership of the *contract*
does not carry ownership of an *engine*, and no consumer has asked for one.
- **Not ours to fix, to be reported by coord:** the wiki masterplan states three
markings and has not been updated with the 2026-08-13 decision that settled
five and approved a sixth. The decision is recorded in that repo's tracked
state file; the document its own consumption engine names as requirements
owner does not carry it. Reported, not edited (O2).
- **`.claude`'s premise correction** (§0) is owed back as a coord message: the
ratification exists in the source repo; the query that found it absent was
scoped to 16 of 639 files.