open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(cli): okf project/consume/check/skill, and a generated skill with no path into a checkout

Browse source 
The reading direction existed only for someone standing in a clone. `consume`,
`contract_check` and `skill` moved from `tools/` into the package and are
reachable as `okf consume`, `okf check` and `okf skill`; `okf project` is new
and does the whole thing in one command.

The red measurement: a consumption skill generated from a checkout carried 4
lines naming that checkout by absolute path, 2 of them the commands the skill
tells a reader to run. It now names `okf consume` and `okf check`, and a test
asserts this repository appears in it nowhere, with a known-positive so the
zero is a measurement rather than a search that could not find.

The `tools/` files stay as ALIASES, not re-exports: a re-export binds copies of
the names into a second module object, so a caller patching one patches a
binding the implementation never reads. Two tests that monkeypatch okf_consume
went green again only under the alias. Every published reproduction block runs
unchanged.

The template and docs/consumption-contract.md (the section 7.4 known-positive)
are force-included into the wheel from the file they are authored in, so both
travel with the commands that cannot run without them and there is still one
authored copy of each.

Step 0, before any of it: okf build's default gained Arm E (--table-grid),
with --no-table-grid as its opt-out. The default moved to D plus F earlier the
same day on Arm F's published 5 of 12 -- a figure measured with Arm E ON.
Without it the fold has no joined table to fold, and the shipped default scored
2 of 12 with docx 0 of 3. Measured on the operator's folder: 30 md / 15
concepts on the new default against 43 / 28 without Arm E.

Install measurement from a fresh uv tool install, empty folder, this repository
nowhere on PYTHONPATH: 5 documents in, 15 concepts out, 0 references to tools/
in the generated skill, okf check conformant (15 rules, 0 findings).

Deviation stated rather than hidden: the order asked that
tests/test_okf_consume.py be left untouched. Two assertions in it read a PATH,
which is the one thing this work changes. Both were moved and the second made
stronger -- it now asserts every command the README recipe names is a
subcommand the CLI registers, which a file existing on disk never proved.

Suite 1414 -> 1427. ruff clean, mypy --strict clean over 21 files.
Record: docs/2026-09-08-o5-okf-project.md

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 21:47:38 +02:00
commit 1fb1bbd716
21 changed files with 4017 additions and 2833 deletions
Download patch file Download diff file Expand all files Collapse all files

58
CHANGELOG.md
View file

@ -7,8 +7,65 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
## [0.6.0] — 2026-09-08
The first tag since `v0.5.0a2`, so everything that had accumulated as
Unreleased is in it — those sections are kept below, under their own heading,
rather than folded together. What follows first is what O5 added, and why the
release needed a minor of its own: the installed command grew from one
subcommand to five, and one build default moved.
### Added
- **`okf consume`, `okf check`, `okf skill` and `okf project` are subcommands
of the installed `okf` command.** Until now the pre-pass, the contract
checker and the skill generator lived in `tools/` and were reachable only
from a clone; a consumer who installed this library could build a bundle and
had no way to read one back. Measured before the move: a consumption skill
generated from a checkout carried **four** lines naming that checkout by
absolute path, two of them the commands the skill tells a reader to run, so
the skill could not be moved, shared, or run by anyone else. The generated
skill now names `okf consume` and `okf check` — names on PATH — and a test
asserts the repository appears in it nowhere, with a known-positive so the
zero is a measurement rather than a search that could not find.
`tools/okf_consume.py`, `tools/okf_contract_check.py` and `tools/okf_skill.py`
remain as thin aliases, so every published reproduction block still runs.
- **`okf project <folder>`: a folder of documents to a bundle you can ask a
question of, in one command.** It runs `okf build` with this package's
default into `<out>/.okf/<id>/`, generates the skill into
`<out>/.claude/skills/<id>-consume/`, and prints what it read, what it wrote,
which documents are in the folder but not in the bundle, and which landed
whole as a single concept — the two cases a question can only be answered
`[sourced-not-sufficient]` in. `<out>` defaults to the current directory and
`<id>` to the folder's name reduced to `[a-z0-9-]`. It owns no flag that
changes a bundle's bytes, and a test holds the project bundle byte-equal to
the `okf build` bundle of the same folder.
- **`skills/okf-prosjekt/`**, a Claude Code skill (Norwegian) that wraps
`okf project` and reads its summary back.
- **The template and the contract document travel in the wheel.** `okf skill`
instantiates `skills/okf-consume-template/SKILL.md` and `okf consume`
measures `docs/consumption-contract.md` as its section 7.4 known-positive;
neither was installable before. Both are force-included from the file they
are authored in, so there is still exactly one copy of each.
### Changed
- **`okf build`'s default now includes Arm E (`--table-grid`), with
`--no-table-grid` as its opt-out.** The default moved to Arm D plus Arm F
earlier in the same day; measured afterwards, that combination is Arm F with
nothing to fold. The fold's table clause folds a table back into the heading
that introduces it, and with Arm E off a grid table is not one block but one
block per rule line. On the operator's twelve-document reference the shipped
default scored **2 of 12** and `docx` **0 of 3**, against the **5 of 12** the
fold was published with — which had been measured with Arm E on. Three arms
are now on by default, each with an explicit opt-out; `--outline-run 0
--no-table-grid --no-unit-fold` reproduces the pre-2026-09-08 bytes.
- **This tag does not make OKF v0.2 generally available.** `OKF_LATEST` is
unchanged.
### Also in this release: everything that had accumulated since v0.5.0a2
- **The corpus harness (`tools/okf_corpus_run.py`) can replay segmentation
plans, and it writes the bundle's `log.md`.** `--plans-dir` names the
proposals to replay and the profile follows from it; `--bundle-id` and
@ -600,6 +657,7 @@ Phase 1 (Door A) implemented against the normative `ingest-spec.md` owned by
- The Door A public surface: `materialize_bundle` plus the typed error hierarchy
rooted in `IngestError`.
[0.6.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.5.0a2...v0.6.0
[0.5.0a2]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.5.0a1...v0.5.0a2
[0.5.0a1]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.4.0...v0.5.0a1
[0.4.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.2...v0.4.0

46
CLAUDE.md
View file

@ -247,6 +247,13 @@ and fixtures, never code.
- Test: `pytest`
- Lint: `ruff check .` + `ruff format --check .`
- Type check: `mypy --strict src/`
- Folder to questionable bundle in ONE command: `okf project <folder>` —
`okf build` with the package default into `<out>/.okf/<id>/` plus `okf skill`
into `<out>/.claude/skills/<id>-consume/`, `<out>` defaulting to cwd and
`<id>` to the folder name reduced to `[a-z0-9-]`. It owns NO flag that moves
a bundle's bytes and a test holds it byte-equal to `okf build`; two build
paths would leave every measurement report pinned to a bundle nobody
produces. `skills/okf-prosjekt/` is the Claude Code skill over it.
- Build a bundle: `okf build <folder> --bundle <dir> --bundle-id <id>
--okf-version <v>` — the installed console script (`[project.scripts]`),
the packaged form of what used to be a shell loop over two `tools/`
@ -259,11 +266,14 @@ and fixtures, never code.
collide); `--ingested-at` and `--proposed-at` default to one shared epoch
constant rather than the clock, because a wall-clock default takes
rebuild-equals-incremental away from anyone who omits them.
**Six segmentation rules are REACHABLE here, and since 2026-09-08 TWO of
them are ON by default** -- `--outline-run 3` and `--unit-fold`, the
operator's D3 answer (alternative b), each with an explicit opt-out
(`--outline-run 0`, `--no-unit-fold`) that together reproduce the pre-move
bytes. **The proposer's own defaults did NOT move** (`propose.py`'s rules stay
**Six segmentation rules are REACHABLE here, and since 2026-09-08 THREE of
them are ON by default** -- `--outline-run 3`, `--table-grid` and
`--unit-fold`, each with an explicit opt-out (`--outline-run 0`,
`--no-table-grid`, `--no-unit-fold`) that together reproduce the pre-move
bytes. Arm E joined a session after the other two, on a number measured
AFTER the first move: without it Arm F's table clause has no joined table to
fold, and the shipped D+F default scored 2 of 12 with `docx` 0 of 3 against
the 5 of 12 the fold was published with. **The proposer's own defaults did NOT move** (`propose.py`'s rules stay
off): the goldens and every published reproduction block are pinned to them,
so the two layers disagree on purpose and `cli.DEFAULT_OUTLINE_RUN` /
`cli.DEFAULT_UNIT_FOLD` say where. The cost to a consumer is a re-run and it
@ -297,15 +307,18 @@ and fixtures, never code.
`docs/2026-09-08-k3-arm-f-mot-enhetsarket.md`,
`docs/2026-09-08-k3-runde2-per-filtype.md` and
`docs/2026-09-08-k3-runde3-per-filtype.md`.
- Consume a bundle: `python3 tools/okf_consume.py <bundle> --question "<q>"
- Consume a bundle: `okf consume <bundle> --question "<q>"
[--k N] [--limit N] [--out PATH] [--ref IDENTITY]` — the **pre-pass**
`docs/consumption-contract.md` § 1 defines, and the only reading direction
this library has. It lives in `tools/` for the reason
`okf_contract_check.py` states for itself: outside `src/`, so no consumer's
install surface changes because it exists. Its entry point is
`build_payload(...)` with the CLI a thin `main()`, so lifting it into `src/`
the day a consumer asks for a wheel-installed command is a move, not a
rewrite. Deterministic and offline by construction: no model call, no socket,
this library has. **It moved into the package 2026-09-08 (O5)** and the move
it was written for is the one that happened: `build_payload(...)` was always
the entry point with the CLI a thin `main()`, so it was a move and not a
rewrite. What forced it was the generated skill — from `tools/` it emitted
`python3 <absolute path>/tools/okf_consume.py`, so the skill could not be
moved, shared or run by anyone without that clone. `tools/okf_consume.py`
remains as an ALIAS (`sys.modules[__name__] = _impl`, never a re-export: a
re-export binds copies, and a caller patching one patches a binding the
implementation never reads). Deterministic and offline by construction: no model call, no socket,
no clock, stdlib plus this package only. It **walks the index tree, never a
directory** — § 9.2 forbids enumerating one unless the named profile says the
index is derived, and measured, `entries_match_directory` is `True` for
@ -326,12 +339,17 @@ and fixtures, never code.
`excerpt_unnamed`) and the rest SHOULD, because they are conditional on the
producer. The measurement behind it: rank 1 of 8 on 3 of 3 bundles, correct
answer on 1 of 3.
- Connect a bundle to Claude Code: `python3 tools/okf_skill.py <bundle> --out
- Connect a bundle to Claude Code: `okf skill <bundle> --out
<dir>` instantiates `skills/okf-consume-template/` for THAT bundle — its id,
ref, concept count, conditional-field denominators, whole-bundle cost and
breaking point, all measured, plus a reference payload the checker accepts.
In `tools/` for the reason the other two are, and because a wheel-installed
It was kept in `tools/` until 2026-09-08 because a wheel-installed
`okf skill` would emit a command pointing at a file the wheel does not carry.
That objection was about what the GENERATED skill NAMES, and O5 answered it
by changing that: the emitted commands are `okf consume` and `okf check`,
names on PATH. The template and `docs/consumption-contract.md` (the § 7.4
known-positive) are force-included into the wheel from the file they are
authored in — one authored copy, no committed duplicate.
The form was chosen on a measurement: the contract checker passes the
UNFILLED template and passes a skill built for another bundle, so it cannot
tell the two apart — the choice rests on § 5/§ 6.4/§ 7.6 being per-bundle

112
README.md
View file

@ -48,9 +48,15 @@ do not. `main` has since moved its own pin to `>=1.2,<2.0` (see
not in the commands above. Reading a pin off this branch and installing it
against `v0.4.0` is the one combination that fails.
`v0.4.0` is the current stable tag. `v0.5.0a2` is a pre-release for the named
OKF v0.2 pilot set only; pin it only if you are one of them (see
[Upstream OKF versions](#upstream-okf-versions)).
`v0.6.0` is the current tag and the one the three-line form under [Consume in
Claude Code](#consume-in-claude-code) installs: it is the first tag carrying the
`okf project`, `okf consume`, `okf check` and `okf skill` subcommands, without
which that form does not exist. `v0.4.0` is the last tag before the OKF v0.2
work. `v0.5.0a2` is a pre-release for the named OKF v0.2 pilot set only.
**`v0.6.0` does not make OKF v0.2 generally available.** `OKF_LATEST` is
unchanged and still points at `DEFAULT`; flipping that alias is the GA event and
this tag is not it (see [Upstream OKF versions](#upstream-okf-versions)).
## Build
@ -235,65 +241,71 @@ hit@8 was **5 of 6** questions at rank 1 against a chance baseline of **1.35 of
## Consume in Claude Code
A folder of documents to an answer a model can cite, in three commands. Every
command below was run end to end on 2026-09-08 against a nine-document folder
and a 270-concept third-party bundle; nothing here is untested.
A folder of documents to an answer a model can cite, in **three lines**. You do
not need this repository — the first line installs the command, the second
builds the bundle and writes a skill beside it, the third asks.
```sh
SRC=/tmp/c1-fresh-src # the folder of documents
BUNDLE=/tmp/c1-fresh-bundle # where the OKF bundle goes
PROJECT=/tmp/c1-scratch # the project you will ask the question from
uv tool install "llm-ingestion-okf[extract] @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.6.0"
okf project ~/my-documents
claude
```
**1. Build the bundle.**
`okf project` writes the bundle to `.okf/<id>/` and a skill to
`.claude/skills/<id>-consume/` in the **current directory**, then prints what it
read, what it wrote, and which documents a question cannot reach. Start `claude`
in that directory and ask in plain language; the generated skill runs the
pre-pass and the contract check itself and marks every claim with its source.
`<id>` is the folder's name reduced to `[a-z0-9-]`. Run it once per folder with
`--id <name>` to have several bundles reachable at once — each skill carries its
own `bundle_id`, which is what lets a model pick between them. `--out <dir>`
puts the project somewhere other than the current directory.
Measured 2026-09-08 from a fresh `uv tool install` with this repository nowhere
on the path: 5 documents in, 15 concepts out, a skill carrying **0** paths into
any checkout, and `okf check` conformant on its own payload (15 rules, 0
findings). Before that day the same result took a `PYTHONPATH`, a snapshot of a
clone, and a generated skill that named that clone by absolute path on four
lines — so it could not be moved, shared, or run by anyone else.
### The same thing in steps, if you want to see the payload
```sh
okf build "$SRC" --bundle "$BUNDLE" --bundle-id c1-fresh-20260908 --okf-version 0.2 --ingested-at 2026-09-08T00:00:00Z
okf build ./documents --bundle ./bundle --bundle-id my-bundle --okf-version 0.2
okf skill ./bundle --out ./project/.claude/skills/my-bundle-consume
okf consume ./bundle --question "your question" --out /tmp/payload.json
okf check --skill ./project/.claude/skills/my-bundle-consume/SKILL.md --payload /tmp/payload.json
```
**2. Generate a skill for that bundle**, straight into the project's skill
directory. The skill is instantiated for these bytes: its id, ref, concept
count, per-field denominators, whole-bundle cost and breaking point are all
measured from the bundle, and it ships a reference payload the checker accepts.
A bundle you only have read access to is fine — the generator only reads it.
```sh
python3 tools/okf_skill.py "$BUNDLE" --out "$PROJECT/.claude/skills/c1-fresh-20260908-consume"
```
### The honest limits
Repeat for every bundle you want reachable; each one gets its own skill named
after its `bundle_id`, which is what lets a model pick between them. A bundle
you only have read access to is fine — the generator only reads it.
**3. Ask.** From `$PROJECT`, in Claude Code:
```sh
claude -p "Hvordan skal prisene fylles ut?"
```
Measured with two bundles installed side by side: the model selected the right
skill from the question alone, ran the pre-pass and the contract check itself,
quoted the requirement verbatim, and named the document, the requirement number,
the source resource and the locator inside it. Across three questions, **0**
numbers or identifiers appeared in an answer that were not in the delivered set
or in the payload's own identities. On a question the bundle does not cover it
answered `[sourced-not-sufficient]` and reported the denominator rather than
inventing an answer.
The pre-pass and the checker are the same two commands the skill runs for you,
if you want to see the payload first:
```sh
python3 tools/okf_consume.py "$BUNDLE" --question "your question" --out /tmp/payload.json
python3 tools/okf_contract_check.py --skill "$PROJECT/.claude/skills/c1-fresh-20260908-consume/SKILL.md" --payload /tmp/payload.json
```
The honest limits: this was measured on **four questions** across two bundles,
which is a demonstration and not a hit rate. The ranking is lexical, and one of
the four found a topic the bundle **does** cover and did not rank it into the
cut — the skill then said so with its denominator instead of answering, which is
the behaviour the contract asks for, but a miss is still a miss.
Measured on **four questions** across two bundles, which is a demonstration and
not a hit rate. The ranking is lexical, and one of the four found a topic the
bundle **does** cover and did not rank it into the cut — the skill then said so
with its denominator instead of answering, which is the behaviour the contract
asks for, but a miss is still a miss.
`docs/2026-09-08-claude-code-skill-vilkaarlig-bundle.md` has the runs.
Two more things the summary tells you and this paragraph will not repeat: a
document that landed **whole** (no heading, table or numbered outline to cut it
on) comes back as one excerpt, which the budget often refuses and which often
does not carry the answer at the place you asked about; and a document that is
in the folder but **not** in the bundle cannot be quoted at all. Both cases are
answered `[sourced-not-sufficient]`, and `okf project` names the documents.
### The skill that runs this for you
`skills/okf-prosjekt/` in this repository is a Claude Code skill (Norwegian)
that wraps the command above: it takes a folder, runs `okf project`, and reads
the summary back. Install it for your user account after cloning:
```sh
mkdir -p ~/.claude/skills && cp -R skills/okf-prosjekt ~/.claude/skills/
```
## Implemented scope (v1)
The library provides three entry points for getting content into an OKF

178
docs/2026-09-08-o5-okf-project.md Normal file
View file

@ -0,0 +1,178 @@
# O5: one command from a folder to a bundle you can ask a question of
2026-09-08. Session 113, order `20260908T184152Z-1340419920`, on top of
`efa92ce` (K3 round 3).
This is the session that made the reading direction installable. Everything
here already worked from a clone; none of it worked for anyone else.
## 1. The red measurement
The consumption skill generator was run against a bundle and the result
grepped for paths:
```sh
python3 tools/okf_skill.py ~/okf-test/v1-bundle --out /tmp/o5-red/skill
grep -c "^/" /tmp/o5-red/skill/SKILL.md # absolute paths
```
**4 lines**, of which two are the commands the skill tells a reader to run:
```
17: The contract this skill is held to is `<HOME>/repos/llm-ingestion-okf/docs/consumption-contract.md`.
25: python3 <HOME>/repos/llm-ingestion-okf/tools/okf_consume.py \
26: <HOME>/okf-test/v1-bundle \
40: python3 <HOME>/repos/llm-ingestion-okf/tools/okf_contract_check.py \
```
Line 26 is the caller's own data and belongs there. The other three name a
checkout. A skill carrying them cannot be moved, shared, or run by anyone
without that clone at that exact path — which is every reader except the one
who generated it.
## 2. Step 0: the default moved again, because the number behind it moved
Round 3 set `okf build`'s flagless default to `--outline-run 3 --unit-fold`
(Arm D plus Arm F) on Arm F's published "5 of 12". That figure had been
measured with `--table-grid` ON. Without Arm E, Arm F's table clause — fold a
table back into the heading that introduces it — has no joined table to fold,
because a grid table is then not one block but one block per rule line.
Arm E therefore joined the default, with `--no-table-grid` as its opt-out.
Measured here on the operator's five-document folder, at the same stamp:
| build | `.md` files | of which concepts | `index.md` | `log.md` |
|---|---|---|---|---|
| new default (D+E+F) | 30 | **15** | 14 | 1 |
| `--no-table-grid` (D+F, round 3's) | 43 | **28** | 14 | 1 |
The `.md` totals reproduce the PM measurement exactly (43 and 30). The concept
counts differ from the 29 and 16 it reported by exactly one each, and the
decomposition says why: `md` files minus `index.md` counts `log.md` as a
concept. The index walk does not, and neither does the pre-pass.
`--keep-table-heading` did NOT join the default: measured on two K2 bundles in
round 3 it buys 35 bytes and zero rank positions.
The red test is `test_the_build_default_is_now_arm_d_plus_arm_e_plus_arm_f` on
a grid-table fixture, where the arm's effect is visible in the TITLES: with it
off the sheet lands as `7 Romskjema` plus `Tabell linje 6`, `Tabell linje 8`
and `Tabell linje 10` — concepts named after a line number. Its opt-out half
asserts those four titles come back under `--no-table-grid`.
## 3. The move
Three modules moved from `tools/` into `src/llm_ingestion_okf/`:
| was | is | reached as |
|---|---|---|
| `tools/okf_consume.py` | `consume.py` | `okf consume` |
| `tools/okf_contract_check.py` | `contract_check.py` | `okf check` |
| `tools/okf_skill.py` | `skill.py` | `okf skill` |
and one is new: `project.py`, reached as `okf project`.
The `tools/` files stay, because published reproduction blocks name them and a
measurement whose command no longer runs is a measurement nobody can repeat.
They are **aliases, not re-exports**: `sys.modules[__name__] = _impl`. A
re-export binds copies of the names into a second module object, so a caller
patching one patches a binding the implementation never reads — measured, two
tests that `monkeypatch` `okf_consume` went green again only under the alias.
**Two authored files had to learn to travel.** `okf skill` instantiates
`skills/okf-consume-template/SKILL.md`, and `okf consume` measures
`docs/consumption-contract.md` as its § 7.4 known-positive and refuses without
it. Neither is under `src/`. Both are force-included into the wheel from the
file they are authored in, and both resolvers prefer the packaged copy and fall
back to the source tree — one authored copy, no committed duplicate, because a
duplicate is the drift the generator exists to prevent.
**The generated skill names commands, not paths.** `okf consume` and
`okf check`, resolved by the shell. The two attribution lines that named
`tools/okf_skill.py` now name `okf skill`.
## 4. What `okf project` does, and what it refuses to decide
`okf project <folder>` runs `okf build` with **this package's default** into
`<out>/.okf/<id>/`, generates the skill into
`<out>/.claude/skills/<id>-consume/`, and prints a summary. `<out>` defaults to
the current directory, `<id>` to the folder's name NFC-normalised and reduced to
`[a-z0-9-]`.
It owns no flag that changes a bundle's bytes. A test holds the project bundle
byte-equal to the `okf build` bundle of the same folder at the same stamp: two
build paths would mean every measurement report is pinned to a bundle nobody
produces.
The summary reports two things a concept count cannot:
- documents in the folder that **no concept names** — nothing can quote them;
- documents that landed **whole**, as one flat concept at the bundle root
(measured on the artefact: a concept id with no `/`) — reachable, but a
question returns the entire document as one excerpt, which the budget often
refuses and which often does not carry the answer where it was asked.
Both are `[sourced-not-sufficient]` cases, and the summary names the documents.
## 5. The install measurement
From a fresh `uv tool install` into an isolated tool directory, run from an
empty folder with this repository nowhere on `PYTHONPATH` and `PATH` reduced to
the install plus `/usr/bin:/bin`:
```
Read 5 document(s) from <HOME>/okf-test/dokumenter.
Wrote 15 concept(s) to /private/tmp/o5-user/.okf/dokumenter.
Wrote the skill to /private/tmp/o5-user/.claude/skills/dokumenter-consume/SKILL.md.
0 of 5 document(s) were left out of the bundle. [...]
1 of 5 document(s) landed WHOLE [...] Expect `[sourced-not-sufficient]` there:
- Del II Bilag 6 - Teknisk oppsett.pdf
NEXT: start claude again in /private/tmp/o5-user and ask your question.
```
Exit 0. The generated skill carries **0** references to `tools/` or to any
checkout. `okf consume` on that bundle exits 0, and `okf check` on the pair is
**conformant: 15 rules over 8 excerpts and 7 withheld entries, 0 findings**.
**The first install measurement was wrong and is recorded here rather than
dropped.** `uv tool install` from the unchanged local path returned a cached
build of the previous version, and `okf project` failed with `invalid choice:
'project'`. The version bump was already required; without it the measurement
would have reported the old wheel as the new one.
## 6. What was NOT done
- **`docs/consumption-contract.md` was not edited.** Its § "What is
mechanically checked" still names `tools/okf_contract_check.py`, which is
true — the alias exists. Its bytes are the pre-pass's known-positive
(`KNOWN_POSITIVE_EXPECTED = 12563`), so editing it makes the pre-pass refuse
until the literal is updated. That coupling is deliberate and this was not
the session to spend it.
- **`skills/okf-consume/` was not regenerated**, only its four command and
attribution lines were updated in place. Regenerating needs the K2 bundle,
which is never copied into this repository.
- **No push, and the tag is local.** The order and the operator own that.
- **`tests/test_okf_consume.py` was not left untouched, and the order asked
that it be.** Two assertions in it read a PATH, which is the one thing this
work exists to change: one read the pre-pass's source text at
`tools/okf_consume.py`, the other required the README recipe to invoke
`python3 tools/<script>.py`. Left alone, the first would have measured a
wrapper with no vocabulary in it and passed, and the second would have
failed. Both were moved to the new location and the second was **made
stronger** — it now asserts every command the recipe names is a subcommand
the CLI registers, which a file existing on disk never proved. Nothing else
in the file changed; the pre-pass's behavioural contract is unaltered.
- **Round 4's per-file-type questions** (`xlsx` at 12 against a reference of
11, `pdf` at 3 of 8 and losing 940 characters of an opening) were not
touched. They stand as next.
## 7. Numbers
| | before | after |
|---|---|---|
| tests | 1414 | **1427** |
| `okf` subcommands | 1 | **5** |
| absolute paths in a generated skill | 4 | **0** |
| `mypy --strict` source files | 17 | 21 |

15
pyproject.toml
View file

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "llm-ingestion-okf"
version = "0.5.0a2"
version = "0.6.0"
description = "Shared OKF (Open Knowledge Format) ingestion library: spec-based connectors, bundle inbox, and external-bundle import, with security delegated to llm-ingestion-guard."
readme = "README.md"
license = "MIT"
@ -94,6 +94,19 @@ dev = ["pytest>=8", "mypy>=1.14", "ruff>=0.9"]
[tool.hatch.build.targets.wheel]
packages = ["src/llm_ingestion_okf"]
# Two AUTHORED files the packaged commands cannot run without, carried into the
# wheel from where they are edited rather than committed a second time under
# `src/`. A duplicate would drift, and both of these are checked against
# literals in the code: a template whose blocks are pinned by a test, and a
# known-positive artefact whose byte count is a constant in `consume.py`.
#
# `okf skill` instantiates the template; `okf consume` measures the contract
# document as its section 7.4 known-positive and refuses without it. Before
# 2026-09-08 neither command was installable, so neither file had to travel.
[tool.hatch.build.targets.wheel.force-include]
"skills/okf-consume-template/SKILL.md" = "llm_ingestion_okf/_data/okf-consume-template.md"
"docs/consumption-contract.md" = "llm_ingestion_okf/_data/consumption-contract.md"
[tool.ruff]
line-length = 100
target-version = "py310"

4
skills/okf-consume-template/SKILL.md
View file

@ -11,7 +11,7 @@ assembled, at one ref.
**This file is a template.** Every `<PLACEHOLDER>` is a hole a per-corpus copy
fills; none of them has a default, and a copy that leaves one unfilled is not
configured, it is unfinished. The section headings are fixed:
`tools/okf_contract_check.py` reads them, and a missing one makes the skill
`okf check` reads them, and a missing one makes the skill
non-conformant rather than merely thin.
The contract this skill is held to is `docs/consumption-contract.md`. Where this
@ -28,7 +28,7 @@ Step 1 is always the pre-pass. Run it, read its JSON payload, and judge that.
Check the payload before using it:
```sh
python3 tools/okf_contract_check.py --skill <SKILL_PATH> --payload <PAYLOAD_PATH>
okf check --skill <SKILL_PATH> --payload <PAYLOAD_PATH>
```
A non-zero exit is not a formatting complaint. It means the payload does not

8
skills/okf-consume/SKILL.md
View file

@ -1,6 +1,6 @@
---
name: okf-consume
description: Answer one question about the K2 procurement OKF bundle from a bounded payload assembled by the deterministic pre-pass tools/okf_consume.py, marking every claim with its source. Instantiated from okf-consume-template; every placeholder is filled with a measured value for this corpus.
description: Answer one question about the K2 procurement OKF bundle from a bounded payload assembled by the deterministic pre-pass okf consume, marking every claim with its source. Instantiated from okf-consume-template; every placeholder is filled with a measured value for this corpus.
---
# K2 procurement bundle consumption
@ -11,7 +11,7 @@ assembled, at one ref.
**This file is an instantiated copy of `skills/okf-consume-template/SKILL.md`.**
Every hole the template left is filled below with a value measured against this
corpus; the section headings are unchanged, because
`tools/okf_contract_check.py` reads them by name and a missing one makes the
`okf check` reads them by name and a missing one makes the
skill non-conformant rather than merely thin.
The contract this skill is held to is `docs/consumption-contract.md`. Where this
@ -22,7 +22,7 @@ file and the contract disagree, the contract binds.
Step 1 is always the pre-pass. Run it, read its JSON payload, and judge that.
```sh
python3 tools/okf_consume.py BUNDLE_ROOT --question "your question" --ref REF --out PAYLOAD_PATH
okf consume BUNDLE_ROOT --question "your question" --ref REF --out PAYLOAD_PATH
```
`BUNDLE_ROOT`, `REF` and `PAYLOAD_PATH` are runtime arguments a caller supplies,
@ -40,7 +40,7 @@ stays untouched.
Check the payload before using it:
```sh
python3 tools/okf_contract_check.py --skill skills/okf-consume/SKILL.md --payload PAYLOAD_PATH
okf check --skill skills/okf-consume/SKILL.md --payload PAYLOAD_PATH
```
A non-zero exit is not a formatting complaint. It means the payload does not

53
skills/okf-prosjekt/SKILL.md Normal file
View file

@ -0,0 +1,53 @@
---
name: okf-prosjekt
description: Gjør en mappe med prosjektdokumenter (PDF, DOCX, XLSX, PPTX, MD) om til en OKF-bundle OG en Claude Code-skill i den mappen, i én kommando, slik at neste Claude Code-økt der kan svare på spørsmål og hypoteser med kilde. Bruk når brukeren sier "/okf-prosjekt <mappe>", "lag en OKF-bundle av denne mappen", "koble disse dokumentene til Claude Code", "gjør prosjektdokumentene spørrbare", "bygg bundle fra <mappe>", eller peker på en mappe med prosjektdokumenter og vil stille spørsmål til dem.
---
# /okf-prosjekt — fra mappe til spørrbar bundle i én kommando
Kjør ÉN kommando og gjengi sammendraget. Ikke forklar OKF, ikke vis flagg, ikke
foreslå alternative kommandoer.
```sh
okf project <mappe>
```
- `<mappe>` er brukerens argument (`$ARGUMENTS`). Mangler den: spør hvilken
mappe, ingenting annet.
- Bundlen og skillen havner i **gjeldende mappe** (`.okf/<id>/` og
`.claude/skills/<id>-consume/`). Det er poenget: brukeren starter `claude` på
nytt der og spør.
- Bundle-id blir mappenavnet. Flere mapper i samme prosjekt: kjør kommandoen én
gang per mappe med `--id <navn>`; modellen velger riktig skill ut fra
spørsmålet.
Er `okf` ikke installert, sier skallet `command not found`. Da er ett steg
manglende, ikke feil: se README i `open/llm-ingestion-okf` for install-linjen,
og stopp.
## Etter kjøringen
Gjengi tallene ordrett — dokumenter lest, konsepter skrevet, hvor skillen ble
skrevet — og NESTE-linjen. To ting sier kommandoen som du IKKE skal utelate:
- Står det at dokumenter **ikke er i bundlen**: si i én setning at spørsmål om
dem bare kan besvares `[sourced-not-sufficient]`, fordi ingen utdrag kan
sitere dem.
- Står det at dokumenter **landet hele** (ett konsept hver): si i én setning at
et spørsmål om dem returnerer hele dokumentet som ett utdrag, og at svaret
ofte blir `[sourced-not-sufficient]` inntil dokumentet er delt opp.
Exit 1 = ett steg nektet, og kommandoen skriver koden. Vis den linjen og stopp.
Exit 2 = kjøringen skjedde ikke i det hele tatt (som regel feil mappe).
## Hva brukeren gjør så, i klartekst
1. `claude` på nytt i mappa.
2. Still spørsmålet eller hypotesen rett ut. For eksempel «Hva krever
dokumentene om prisutfylling?», «Hypotese: opplæring er priset inn i
mekanisk ferdigstillelse — bekreft eller avkreft med kilde», eller «Lag et
notat `krav-pris.md` med alle krav til pris, ett avsnitt per krav, med
dokument og kravnummer».
Skillen `<id>-consume` som ble skrevet i mappa kjører pre-passet og
kontraktsjekken selv, og merker hver påstand med kilde.

2
src/llm_ingestion_okf/__init__.py
View file

@ -76,7 +76,7 @@ from .manifest import (
)
from .materialize import IngestResult, materialize_bundle
__version__ = "0.5.0a2"
__version__ = "0.6.0"
__all__ = [
"BlockedFile",

93
src/llm_ingestion_okf/cli.py
View file

@ -1,4 +1,11 @@
"""`okf` — the installed command. One subcommand today: `build`.
"""`okf` — the installed command. Five subcommands: the whole chain, packaged.
`build` a bundle from a folder, `consume` it for one question, `check` the
payload against the contract, generate a `skill` for it, or do the whole thing
in one step with `project`. Four of the five moved here on 2026-09-08 (O5)
from unpackaged scripts under `tools/`, which a consumer who installs this
library does not have; until then the reading direction existed only for
someone standing in a clone.
## What it replaces
@ -38,9 +45,12 @@ the replay dated differently passes both explicitly.
## What it does not decide
**Every arm is OFF unless the caller asks, and this command does not move a
default.** Which arm should ship as the default is the operator's decision and
is not taken here.
**Which arm ships as the default is the operator's decision and is not taken
here.** Three of the six are ON as of 2026-09-08 -- `--outline-run 3`,
`--table-grid` and `--unit-fold` -- each moved by the operator on a
measurement and each with an explicit opt-out. The remaining three are OFF
unless the caller asks. This module's job is to state the answer, never to
pick it.
The arms are, however, REACHABLE from here, and that is a change of 2026-09-08.
Until then `_propose_plans` called the proposer with no arm flag at all, so
@ -71,6 +81,32 @@ from .propose import run as propose_run
__all__ = ["DEFAULT_STAMP", "build", "main", "measure"]
#: The subcommands whose FLAGS belong to the module that implements them.
#:
#: Dispatched before `argparse` runs rather than rebuilt as subparsers here: a
#: second declaration of `--reserve-top-rank` or `--rarity-weight` is a second
#: place they can drift, and the pre-pass's flag set is the thing four
#: measurement reports are pinned to. Each entry is registered as a subparser
#: below anyway, so `okf --help` lists it and an unknown command is still an
#: error -- the parser knows the NAMES, the modules keep the flags.
#:
#: Imported lazily inside the dispatch: `okf build` should not pay to import
#: the ranker, and `okf consume` should not pay to import the proposer.
DELEGATED = ("consume", "check", "skill", "project")
def _delegate(command: str, argv: list[str]) -> int:
if command == "consume":
from .consume import main as run
elif command == "check":
from .contract_check import main as run
elif command == "skill":
from .skill import main as run
else:
from .project import main as run
return run(argv)
CLI_ID = "okf build"
#: What `okf build` runs when no flag is given. Moved 2026-09-08 by the
@ -89,7 +125,15 @@ CLI_ID = "okf build"
#: Each arm keeps an explicit opt-out: `--outline-run 0` (the number was
#: always its own switch) and `--no-unit-fold`. A default a caller cannot turn
#: off is not a default.
#: Arm E joined the default on 2026-09-08, one round after the other two, on a
#: measurement taken AFTER the first move rather than before it: Arm F's
#: published "5 of 12" was measured with Arm E on, and the round-3 default --
#: D plus F, Arm E off -- scored 2 of 12 with `docx` at 0 of 3. The fold's
#: table clause folds a table back into the heading that introduces it, and
#: with Arm E off a grid table is not one block but one block per rule line,
#: so there was nothing whole to fold. `--no-table-grid` is its opt-out.
DEFAULT_OUTLINE_RUN = 3
DEFAULT_TABLE_GRID = True
DEFAULT_UNIT_FOLD = True
#: The timestamp written when the caller passes none, for the ingest stamp and
@ -166,7 +210,7 @@ def build(
plans_dir: Path | None = None,
okf_type: str = "reference",
outline_run: int = DEFAULT_OUTLINE_RUN,
table_grid: bool = False,
table_grid: bool = DEFAULT_TABLE_GRID,
unit_fold: bool = DEFAULT_UNIT_FOLD,
keep_table_heading: bool = False,
sheet_section_rows: bool = False,
@ -254,9 +298,23 @@ def _write_log(bundle: Path, report: CorpusReport, *, profile: BundleProfile) ->
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog="okf",
description="OKF bundle tooling. One folder in, one bundle out.",
description=(
"OKF bundle tooling. One folder in, one bundle out -- and one command "
"from there to a bundle you can ask a question of."
),
)
subcommands = parser.add_subparsers(dest="command", required=True)
# Registered for `okf --help` and for the unknown-command error. Their
# arguments are NOT declared here: `main` hands the rest of `argv` to the
# implementing module, which owns them. `add_help=False` keeps
# `okf consume --help` reaching that module's help rather than this stub's.
for delegated, blurb in (
("consume", "cut a bundle to one contract-conformant payload for one question"),
("check", "check a consumption skill and one payload against the contract"),
("skill", "instantiate the consumption skill template for one bundle"),
("project", "folder in, bundle plus skill out: build and skill in one step"),
):
subcommands.add_parser(delegated, help=blurb, add_help=False)
build_parser = subcommands.add_parser(
"build",
help="build an OKF bundle from a folder of documents",
@ -336,12 +394,24 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
build_parser.add_argument(
"--table-grid",
action="store_true",
default=DEFAULT_TABLE_GRID,
help=(
"Arm E, passed to the proposer unchanged: a pandoc grid-table rule "
"line no longer closes an open table block, so one grid table is one "
"concept instead of one per row group. Absent (the default) is OFF. "
"Measured on the K3 sample: it changes a `.docx` experience list from "
"21 concepts to 6"
"concept instead of one per row group. ON by default since "
"2026-09-08 (operator); the flag is kept so a call site that passes "
"it stays valid. Measured on the K3 sample: it changes a `.docx` "
"experience list from 21 concepts to 6, and without it Arm F has no "
"joined table to fold"
),
)
build_parser.add_argument(
"--no-table-grid",
action="store_false",
dest="table_grid",
help=(
"Arm E's explicit opt-out. With --outline-run 0 and --no-unit-fold "
"it reproduces the pre-2026-09-08 default byte for byte"
),
)
build_parser.add_argument(
@ -409,7 +479,10 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
arguments = list(sys.argv[1:] if argv is None else argv)
if arguments and arguments[0] in DELEGATED:
return _delegate(arguments[0], arguments[1:])
args = parse_args(arguments)
if not args.inbox.is_dir():
print(f"{CLI_ID}: FAILED - no such folder: {args.inbox}", file=sys.stderr)
return 2

1722
src/llm_ingestion_okf/consume.py Normal file
View file

File diff suppressed because it is too large Load diff

503
src/llm_ingestion_okf/contract_check.py Normal file
View file

@ -0,0 +1,503 @@
"""Check a consumption skill and one pre-pass payload against the contract.
`docs/consumption-contract.md` is normative prose. Prose has no test, so this
command reads the mechanically checkable half of it: the payload shape (SS 3, 5,
6, 7, 8) and the fixed vocabulary a conformant `SKILL.md` must carry (SS 4, 6).
**What it cannot see, said here rather than implied.** The division of labour
(SS 2) and the prohibitions (SS 9) are properties of a RUN -- whether an agent
went looking for context the pre-pass withheld, whether a retrieval tool was
pointed at the verdict layer. No static check reaches them. Conformance here is
the floor, never the proof, and a green run means the payload carries what a
claim would have to rest on -- not that the claim was made honestly.
**Every rule has its own code.** One "invalid" verdict over fourteen different
defects is a diagnostic no caller can act on. The codes are the contract's
paragraphs made addressable.
**The report carries denominators**, because the contract requires them of its
consumers and a checker exempting itself would be stating the rule it breaks:
how many rules ran, over how many excerpts and withheld entries.
Exit codes are three, not two: 0 conformant, 1 non-conformant, 2 the check did
not run. Collapsing 2 into 1 would report an unread file as a failed check.
**It moved into the package on 2026-09-08 (O5).** It lived in `tools/` on the
argument that staying out of the wheel left no consumer's install surface
changed. The generated consumption skill made that argument cost more than it
bought: the skill's own check step named this file by absolute path into a
checkout, so the one command that tells a reader whether their payload
conforms was unreachable from an install. `okf check` is that command, on
PATH. `tools/okf_contract_check.py` remains as a thin wrapper for the
published reproduction blocks.
"""
from __future__ import annotations
import argparse
import json
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass
from pathlib import Path
from typing import Any
#: SS 4.1. Five literals, spelled exactly. `[unverifiable-from-bundle]` is one
#: literal string -- no variants, no translations -- so this is a membership
#: test on the literal, never on a translated concept name.
REQUIRED_MARKINGS = (
"extracted",
"derived",
"[unverifiable-from-bundle]",
"[unread]",
"[sourced-not-sufficient]",
)
#: SS 6.1. A closed set of three, and the third is a real state: `unknown` says
#: the concept carries no `adjudication` key, which is what an older bundle
#: looks like. Collapsing it into `proposed` or into absence reports "not
#: judged" where the truth is "we cannot tell", and only one of those is a fact
#: about the concept.
ADJUDICATION_STATES = ("proposed", "adjudicated", "unknown")
#: SS 6.2, from SPEC SS 5.3: derived from `verified`, lowest to highest.
TRUST_TIERS = ("unverified", "machine-confirmed", "human-reviewed")
#: The level-2 headings a conformant skill carries. Checked by name because the
#: alternative -- searching the prose for the obligation -- passes on any
#: document that happens to use the words.
REQUIRED_SECTIONS = (
"Pre-pass",
"Division of labour",
"Markings",
"States",
"Budget",
"Denominators",
"Prohibitions",
)
@dataclass(frozen=True)
class Finding:
"""One contract paragraph, unmet, named by its code."""
code: str
message: str
@dataclass(frozen=True)
class Report:
"""Findings plus the denominators they were measured over."""
findings: tuple[Finding, ...]
rules_evaluated: int
excerpts_examined: int
withheld_examined: int
def render(self) -> str:
denominator = (
f"{self.rules_evaluated} rules over {self.excerpts_examined} excerpts "
f"and {self.withheld_examined} withheld entries"
)
if not self.findings:
return f"conformant: {denominator}, 0 findings"
lines = [f"NOT conformant: {denominator}, {len(self.findings)} findings"]
lines += [f" {finding.code}: {finding.message}" for finding in self.findings]
return "\n".join(lines)
@dataclass(frozen=True)
class Context:
"""What every rule reads. `payload` is empty when the payload is not a
mapping at all, so each rule stays a total function over its input."""
skill: str
payload: Mapping[str, Any]
payload_is_mapping: bool
def _mapping(value: object) -> Mapping[str, Any]:
return value if isinstance(value, Mapping) else {}
def _sequence(value: object) -> Sequence[Any]:
if isinstance(value, Sequence) and not isinstance(value, (str, bytes)):
return value
return ()
def _text(value: object) -> str:
return value if isinstance(value, str) else ""
def _whole(value: object) -> int | None:
# `bool` is an `int` in Python and would make `True` a budget. Excluded by
# name rather than trusted not to arrive.
return value if isinstance(value, int) and not isinstance(value, bool) else None
# --- Payload rules -----------------------------------------------------------
def rule_payload_shape(ctx: Context) -> list[Finding]:
if ctx.payload_is_mapping:
return []
return [Finding("payload_invalid", "the payload is not a JSON object (SS 8)")]
def rule_contract_version(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping or _text(ctx.payload.get("contract")):
return []
return [
Finding(
"contract_unversioned",
"the payload names no `contract` revision, so a reader cannot tell "
"which revision it is holding (SS 8.2)",
)
]
def rule_bundle_ref(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
bundle = _mapping(ctx.payload.get("bundle"))
findings = []
if not _text(bundle.get("ref")):
findings.append(
Finding(
"ref_missing",
"the payload names no bundle `ref`; a version is the producer's "
"assertion, a ref is a fact about bytes (SS 3.3)",
)
)
if not _text(bundle.get("bundle_id")):
findings.append(Finding("ref_missing", "the payload names no `bundle.bundle_id` (SS 3.1)"))
return findings
def rule_excerpt_source_marking(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
findings = []
for position, raw in enumerate(_sequence(ctx.payload.get("excerpts"))):
excerpt = _mapping(raw)
for key in ("bundle_id", "concept_id", "sha256"):
if not _text(excerpt.get(key)):
findings.append(
Finding(
"source_marking_incomplete",
f"excerpt {position} carries no {key!r}; identity across "
"bundles is the (bundle_id, concept_id) tuple with the "
"digest of the bytes it was taken from (SS 3.1, SS 3.2)",
)
)
return findings
def rule_excerpt_named(ctx: Context) -> list[Finding]:
"""SS 8: every excerpt carries a `title`.
Added 2026-09-08 on a measurement, not a preference: `portfolio-optimiser`
ran three paid arms in which the pre-pass delivered the gold concept at rank
1 of 8 on 3 of 3 bundles and the model answered correctly on 1 of 3, because
the excerpt carried `concept_id` and `text` and nothing a reader could name
the document by. A payload no answer can cite from is not conformant; the
identity fields are what SS 3.1's tuple is FOR.
"""
if not ctx.payload_is_mapping:
return []
return [
Finding(
"excerpt_unnamed",
f"excerpt {position} carries no 'title'; an excerpt a reader cannot "
"name is one an answer cannot cite, whatever its rank (SS 8)",
)
for position, raw in enumerate(_sequence(ctx.payload.get("excerpts")))
if not _text(_mapping(raw).get("title"))
]
def rule_excerpt_states(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
findings = []
closed: tuple[tuple[str, tuple[str, ...]], ...] = (
("adjudication", ADJUDICATION_STATES),
("trust_tier", TRUST_TIERS),
)
for position, raw in enumerate(_sequence(ctx.payload.get("excerpts"))):
excerpt = _mapping(raw)
for key, allowed in closed:
if key not in excerpt:
findings.append(
Finding(
"state_missing",
f"excerpt {position} carries no {key!r}; the third state is "
"written explicitly, never omitted -- omission collapses "
'"we cannot tell" into a fact about the concept (SS 6.1)',
)
)
continue
value = _text(excerpt.get(key))
if value not in allowed:
findings.append(
Finding(
"state_not_in_set",
f"excerpt {position} has {key}={value!r}, outside the closed "
f"set {allowed} (SS 6)",
)
)
return findings
def rule_denominator_identity(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
counts = _mapping(ctx.payload.get("denominators"))
values = {key: _whole(counts.get(key)) for key in ("considered", "withheld", "delivered")}
missing = sorted(key for key, value in values.items() if value is None)
if missing:
return [
Finding(
"denominator_identity",
f"the payload reports no whole-number {', '.join(missing)}; a count "
"that is not reported is unmeasured, not zero (SS 5.1)",
)
]
considered = values["considered"]
withheld = values["withheld"]
delivered = values["delivered"]
assert considered is not None and withheld is not None and delivered is not None
if considered != withheld + delivered:
return [
Finding(
"denominator_identity",
f"considered ({considered}) != withheld ({withheld}) + delivered "
f"({delivered}); a count that does not close is not a denominator "
"(SS 5.2)",
)
]
return []
def rule_denominator_lists(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
counts = _mapping(ctx.payload.get("denominators"))
findings = []
for key, member in (("delivered", "excerpts"), ("withheld", "withheld")):
declared = _whole(counts.get(key))
if declared is None:
continue
actual = len(_sequence(ctx.payload.get(member)))
if declared != actual:
findings.append(
Finding(
"denominator_list_mismatch",
f"denominators.{key} is {declared} but {member} holds {actual}; "
"the count and the list are two statements of one fact (SS 8.1)",
)
)
return findings
def rule_withheld_rules(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
findings = []
for position, raw in enumerate(_sequence(ctx.payload.get("withheld"))):
entry = _mapping(raw)
for key in ("concept_id", "rule"):
if not _text(entry.get(key)):
findings.append(
Finding(
"cut_undeclared",
f"withheld entry {position} carries no {key!r}; a visible "
"drop is worth more than a silent one (SS 5.3)",
)
)
return findings
def rule_budget_declared(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
budget = _mapping(ctx.payload.get("budget"))
findings = []
for key in ("unit", "instrument"):
if not _text(budget.get(key)):
findings.append(
Finding(
"budget_undeclared",
f"the budget names no {key!r}; a number without its instrument "
"and unit is not a measurement (SS 7.1)",
)
)
limit = _whole(budget.get("limit"))
if limit is None or limit <= 0:
findings.append(
Finding(
"budget_undeclared",
'"bounded" without a bound is a denominator failure in prose (SS 7.1)',
)
)
if _whole(budget.get("spent")) is None:
findings.append(Finding("budget_undeclared", "the budget reports no `spent` (SS 7.2)"))
return findings
def rule_budget_gate(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
budget = _mapping(ctx.payload.get("budget"))
limit = _whole(budget.get("limit"))
spent = _whole(budget.get("spent"))
if limit is None or spent is None or spent <= limit:
return []
return [
Finding(
"budget_exceeded",
f"spent ({spent}) exceeds limit ({limit}); the cut strategy is wrong "
"for this bundle, which is a finding requiring a decision and never "
"a retry with a narrower question (SS 7.3)",
)
]
def rule_instrument_validated(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
known = _mapping(_mapping(ctx.payload.get("budget")).get("known_positive"))
expected = _whole(known.get("expected"))
measured = _whole(known.get("measured"))
if expected is None or measured is None or expected <= 0 or not _text(known.get("case")):
return [
Finding(
"instrument_unvalidated",
"the budget carries no usable `known_positive` (case, expected, "
"measured); an instrument that has not reproduced a known figure "
"has not been shown to count (SS 7.4)",
)
]
if expected != measured:
return [
Finding(
"instrument_unvalidated",
f"the known-positive expected {expected} and the instrument "
f"measured {measured} (SS 7.4)",
)
]
return []
# --- Skill rules -------------------------------------------------------------
def rule_skill_sections(ctx: Context) -> list[Finding]:
return [
Finding(
"skill_section_missing",
f"the skill carries no `## {section}` section (SS 8 of this checker's "
"reading; the headings are fixed so they can be checked by name)",
)
for section in REQUIRED_SECTIONS
if f"## {section}" not in ctx.skill
]
def rule_skill_markings(ctx: Context) -> list[Finding]:
return [
Finding(
"skill_marking_missing",
f"the skill does not carry the required marking {marking!r} verbatim "
"(SS 4.1); one literal string, no variants, no translations",
)
for marking in REQUIRED_MARKINGS
if marking not in ctx.skill
]
def rule_skill_states(ctx: Context) -> list[Finding]:
return [
Finding(
"skill_state_missing",
f"the skill does not name the state {state!r}, so its consumer cannot "
"be held to the closed set (SS 6)",
)
for state in (*ADJUDICATION_STATES, *TRUST_TIERS)
if f"`{state}`" not in ctx.skill
]
#: Every rule, in report order. `len(RULES)` is the denominator the report
#: quotes: "how many rules ran" is the number that makes "0 findings" mean
#: something.
RULES: tuple[Callable[[Context], list[Finding]], ...] = (
rule_payload_shape,
rule_contract_version,
rule_bundle_ref,
rule_excerpt_source_marking,
rule_excerpt_named,
rule_excerpt_states,
rule_denominator_identity,
rule_denominator_lists,
rule_withheld_rules,
rule_budget_declared,
rule_budget_gate,
rule_instrument_validated,
rule_skill_sections,
rule_skill_markings,
rule_skill_states,
)
def check(skill_text: str, payload: object) -> Report:
"""Run every rule. No rule short-circuits another: a caller fixing one
defect should not discover a second only on the next run."""
is_mapping = isinstance(payload, Mapping)
ctx = Context(
skill=skill_text,
payload=payload if isinstance(payload, Mapping) else {},
payload_is_mapping=is_mapping,
)
findings: list[Finding] = []
for rule in RULES:
findings.extend(rule(ctx))
return Report(
findings=tuple(findings),
rules_evaluated=len(RULES),
excerpts_examined=len(_sequence(ctx.payload.get("excerpts"))),
withheld_examined=len(_sequence(ctx.payload.get("withheld"))),
)
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument("--skill", type=Path, required=True, help="the SKILL.md to check")
parser.add_argument("--payload", type=Path, required=True, help="one pre-pass payload (JSON)")
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
try:
skill_text = args.skill.read_text(encoding="utf-8")
except OSError as exc:
print(f"could not read the skill: {exc}")
return 2
try:
payload = json.loads(args.payload.read_text(encoding="utf-8"))
except OSError as exc:
print(f"could not read the payload: {exc}")
return 2
except json.JSONDecodeError as exc:
print(f"the payload is not readable JSON: {exc}")
return 2
report = check(skill_text, payload)
print(report.render())
return 1 if report.findings else 0
if __name__ == "__main__":
raise SystemExit(main())

263
src/llm_ingestion_okf/project.py Normal file
View file

@ -0,0 +1,263 @@
"""One folder of documents in, one questionable project out, in one command.
`okf project <folder>` is `okf build` followed by `okf skill`, plus the summary
a person needs in order to know what they just got. It adds no rule of its own
and owns no flag that changes a bundle's bytes: the build runs on THIS
package's default, so a project bundle and an `okf build` bundle of the same
folder at the same stamp are the same bytes.
**Why a third command rather than a documented three-step.** The three-step
existed and was measured on a reader: set `PYTHONPATH`, take a snapshot of a
checkout, run `python3 tools/okf_skill.py`, and know which of three directories
each artefact belongs in. The operator's word for it was "extremely cryptic",
and that is a measurement of the instructions, not of the reader. Everything
this module does was already reachable; what was missing was that it was one
thing.
**The two destinations are conventions, not choices this module invents.**
`<out>/.okf/<id>/` keeps the bundle out of the way of the documents it was
built from -- the drop folder is walked recursively, so a bundle written beside
the documents would be its own input on the next run. `<out>/.claude/skills/`
is where Claude Code looks, and the skill is the whole point: a bundle nobody
can ask a question of is a directory.
Exit codes are three, as everywhere in this chain: 0 the project was written,
1 the run happened and refused, 2 the run did not happen.
"""
from __future__ import annotations
import argparse
import re
import sys
import unicodedata
from pathlib import Path
from . import consume, skill
from .cli import DEFAULT_STAMP, build
from .corpus import CorpusReport
from .errors import IngestError
from .inbox import walk_inbox
from .profiles import SEGMENTED_OKF_V0_2
CLI_ID = "okf project"
#: Where the bundle and the skill go under `--out`. Constants rather than
#: flags: a project whose layout varies per run is one whose summary cannot
#: tell a reader where anything is.
BUNDLE_DIR = ".okf"
SKILLS_DIR = Path(".claude") / "skills"
#: What the bundle declares as its upstream version. A VALUE, and normally the
#: caller's (decision E1) -- but `okf project` has no catalog to ask, and a
#: required flag here would put the one-command form back behind a question
#: nobody standing in front of a folder of PDFs can answer. So it is stated:
#: the version the segmented profile this command builds under was written for.
PROJECT_OKF_VERSION = "0.2"
_ID_SAFE = re.compile(r"[^a-z0-9]+")
def slug(name: str) -> str:
"""A folder name reduced to `[a-z0-9-]`, or a coded refusal.
NFC first, for the reason `materialize.reduce_to_id_grammar` normalises:
macOS hands filenames over decomposed, so the same visible folder name
reduces two different ways depending on which form it arrived in.
"""
reduced = _ID_SAFE.sub("-", unicodedata.normalize("NFC", name).casefold()).strip("-")
if not reduced:
raise IngestError(
f"the folder name {name!r} reduces to nothing in the id grammar; pass --id",
code="manifest_invalid",
)
return reduced
def inventory(folder: Path, bundle: Path) -> tuple[tuple[str, ...], tuple[str, ...]]:
"""Two lists a reader needs and cannot get from a concept count.
The first is the dropped documents NO concept names as its source: they are
in the folder, they are not in the bundle, and no excerpt can quote them.
The second is the documents that landed WHOLE, as one flat concept at the
bundle root -- the ones the mechanical rules found no boundary in. They are
reachable, and reaching them returns the entire document as one excerpt,
which the budget will often refuse outright and which, when it does fit,
frequently does not carry the conclusion at the place a reader asked about.
That is the `[sourced-not-sufficient]` case, and it is the difference
between a bundle that has 15 concepts and a bundle that answers.
Read off the INDEX TREE and not off the directory, the same walk the
pre-pass uses: a summary computed by a rule the pre-pass does not share
could name a document as present that no question will ever reach.
"""
walked, _ = walk_inbox(folder, exclude=bundle)
dropped = {path.relative_to(folder).as_posix() for path in walked}
represented: set[str] = set()
whole: set[str] = set()
root_bundle_id = consume.root_bundle_id_of(bundle, profile=SEGMENTED_OKF_V0_2)
for concept_id in consume.enumerate_concepts(bundle, profile=SEGMENTED_OKF_V0_2):
path = bundle / f"{concept_id}{SEGMENTED_OKF_V0_2.paths.concept_suffix}"
concept = consume.read_concept(path, bundle_root=bundle, root_bundle_id=root_bundle_id)
represented.add(concept.source_file)
# A concept id with no `/` sits at the bundle root rather than under a
# per-document directory, which is what an unsegmented document
# produces. Measured on the artefact rather than read off the run's
# log: the log is a file a caller can delete.
if "/" not in concept_id:
whole.add(concept.source_file)
return tuple(sorted(dropped - represented)), tuple(sorted(whole))
def summarise(
folder: Path,
bundle: Path,
skill_path: Path,
out: Path,
report: CorpusReport,
concepts: int,
missing: tuple[str, ...],
whole: tuple[str, ...],
) -> str:
"""The plain-language summary, with a denominator on every number."""
lines = [
f"Read {report.n} document(s) from {folder}.",
f"Wrote {concepts} concept(s) to {bundle}.",
f"Wrote the skill to {skill_path}.",
"",
]
if missing:
lines.append(
f"{len(missing)} of {report.n} document(s) are in the folder and NOT in "
"the bundle, so no excerpt can quote them. A question about one of "
"these can only be answered `[sourced-not-sufficient]`:"
)
lines.extend(f" - {name}" for name in missing)
if report.codes:
lines.append(
" reason code(s): " + ", ".join(f"{code} x{count}" for code, count in report.codes)
)
else:
lines.append(
f"0 of {report.n} document(s) were left out of the bundle. Every "
"document in the folder is reachable to a question."
)
lines.append("")
if whole:
lines.append(
f"{len(whole)} of {report.n} document(s) landed WHOLE, as one concept "
"each: the rules found no heading, table or numbered outline to cut "
"them on. Asking about one of these returns the entire document as a "
"single excerpt, which is often refused for size and, when it fits, "
"often does not carry the answer at the place you asked about. Expect "
"`[sourced-not-sufficient]` there:"
)
lines.extend(f" - {name}" for name in whole)
else:
lines.append(
f"0 of {report.n} document(s) landed whole; every one was cut into "
"parts a question can reach separately."
)
lines.extend(
[
"",
f"NEXT: start claude again in {out} and ask your question.",
]
)
return "\n".join(lines)
def create(
folder: Path,
*,
out: Path,
bundle_id: str | None = None,
ingested_at: str = DEFAULT_STAMP,
force: bool = False,
) -> tuple[Path, Path, str]:
"""Build the bundle, generate the skill, return both paths and the summary.
Keyword-only with defaults, so a caller taking this as an API keeps a
source-compatible call when a parameter is added.
"""
identity = bundle_id if bundle_id is not None else slug(folder.resolve().name)
bundle = out / BUNDLE_DIR / identity
report = build(
folder,
bundle,
ingested_at=ingested_at,
bundle_id=identity,
okf_version=PROJECT_OKF_VERSION,
)
if report.unaccounted or report.merged + report.rejected != report.n:
raise IngestError(
f"K1b FAILED - merged ({report.merged}) + coded rejections "
f"({report.rejected}) != N ({report.n}). Unaccounted: "
f"{', '.join(report.unaccounted) or '(none named)'}",
code="conservation_failed",
)
skill_dir = out / SKILLS_DIR / f"{identity}-consume"
written = skill.generate(bundle, out=skill_dir, force=force)
concepts = len(consume.enumerate_concepts(bundle, profile=SEGMENTED_OKF_V0_2))
missing, whole = inventory(folder, bundle)
summary = summarise(folder, bundle, written, out, report, concepts, missing, whole)
return bundle, written, summary
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog=CLI_ID,
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("folder", type=Path, help="the folder of documents to make questionable")
parser.add_argument(
"--id",
dest="bundle_id",
default=None,
help="the bundle id. Defaults to the folder's name reduced to [a-z0-9-]",
)
parser.add_argument(
"--out",
type=Path,
default=None,
help="where the project is written. Defaults to the current directory",
)
parser.add_argument(
"--ingested-at",
default=DEFAULT_STAMP,
help=f"stamped verbatim. Default {DEFAULT_STAMP}: deterministic, never the clock",
)
parser.add_argument(
"--force", action="store_true", help="replace an existing SKILL.md at the destination"
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
if not args.folder.is_dir():
print(f"{CLI_ID}: the run did not happen - no such folder: {args.folder}", file=sys.stderr)
return 2
out = args.out if args.out is not None else Path.cwd()
try:
_, _, summary = create(
args.folder,
out=out,
bundle_id=args.bundle_id,
ingested_at=args.ingested_at,
force=args.force,
)
except (IngestError, consume.ConsumeError, skill.SkillError) as exc:
print(f"{CLI_ID}: refused ({exc.code}) - {exc}", file=sys.stderr)
return 1
except OSError as exc:
print(f"{CLI_ID}: the run did not happen - {exc}", file=sys.stderr)
return 2
print(summary)
return 0
if __name__ == "__main__":
raise SystemExit(main())

630
src/llm_ingestion_okf/skill.py Normal file
View file

@ -0,0 +1,630 @@
"""Instantiate the consumption skill template for ONE named OKF bundle.
`skills/okf-consume-template/SKILL.md` is a template whose own rule is that a
copy leaving a `<PLACEHOLDER>` unfilled "is not configured, it is unfinished".
Filling it by hand is what produced `skills/okf-consume/` for one corpus. This
command does the same thing for any bundle, from values it measures rather than
values someone remembered.
**Why a generator rather than one generic skill.** Measured 2026-09-08: the
contract checker passes the UNFILLED template against a real payload (exit 0, 15
rules, 0 findings), and passes a skill built for a different bundle against this
one's payload. So the checker cannot tell the two forms apart, and the choice
could not be made on conformance. It was made on what the skill has to state:
§ 5's denominators, § 7.6's breaking point and § 6.4's conditional-field list
are all per-bundle numbers. A generic skill can either leave them as holes -- the
template's own definition of unfinished -- or carry another corpus's numbers,
which is worse, because a stated cost that is false for this bundle is a
measurement failure and not merely a gap. Instantiating is what makes them true.
And with several bundles connected at once, a generic skill has nothing to
select on: each generated skill carries the bundle's id in its own name.
**Zero model calls, zero network, no clock.** The same bundle bytes produce the
same skill bytes.
**It moved into the package on 2026-09-08 (O5), and so did the reason it could
not before.** The old objection was exact: a wheel-installed `okf skill` would
emit a command pointing at `tools/okf_consume.py`, which the wheel does not
contain. That objection was about what the generated skill NAMES, and the
answer was to change what it names. The emitted commands are now `okf consume`
and `okf check` -- names on PATH after an install, resolved by the shell and
not by this repository's layout. The pre-pass and the checker moved into the
package in the same step, so both names exist wherever the generated skill
does. Measured before the move: a skill generated from a checkout carried four
lines with an absolute path into that checkout, two of them the commands a
reader is told to run.
Exit codes are three, as elsewhere in this chain: 0 the skill was written, 1 the
run happened and refused, 2 the run did not happen.
"""
from __future__ import annotations
import argparse
import json
import re
from collections import Counter
from pathlib import Path
from . import consume as okf_consume
from .profiles import BundleProfile
#: The template, resolved to a copy that exists wherever this module does.
#:
#: A wheel carries `_data/okf-consume-template.md`, force-included from the
#: authored file below at build time; a source tree carries only the authored
#: file. Preferring the packaged copy and falling back keeps ONE authored
#: template -- a committed second copy is the drift this generator exists to
#: prevent, restated one directory up.
PACKAGED_TEMPLATE = Path(__file__).resolve().parent / "_data" / "okf-consume-template.md"
AUTHORED_TEMPLATE = (
Path(__file__).resolve().parents[2] / "skills" / "okf-consume-template" / "SKILL.md"
)
#: What the generated skill tells a reader to RUN. Names on PATH after an
#: install, never paths into a checkout: a generated skill is meant to be moved,
#: shared and run by someone who does not have this repository.
PRE_PASS_COMMAND = "okf consume"
CHECKER_COMMAND = "okf check"
#: The contract is prose in this repository and is NOT shipped in the wheel, so
#: the generated skill names it as a document rather than as a path a reader
#: could open. Naming an absolute path here is what made a generated skill
#: unmovable; naming a file the reader may not have is honest about which it is.
CONTRACT = "docs/consumption-contract.md in open/llm-ingestion-okf"
#: The profile the pre-pass reads a bundle under, spelled so the generated skill
#: can name it in § 9.2's sentence. The pre-pass's own default; a bundle built
#: under another profile needs a copy of this tool that says so.
PROFILE_NAME = "SEGMENTED_OKF_V0_2"
#: The conditional frontmatter keys the generated skill reports a denominator
#: for. Every one of them is written by SOME producer and not by others, which
#: is exactly what § 6.4 says a consumer must be told about rather than left to
#: infer from an absence.
CONDITIONAL_FIELDS = (
"adjudication",
"bundle_id",
"verified",
"req_number",
"sources",
)
#: Tokens too short to carry a question. The same floor the pre-pass's own
#: matcher uses, so the derived example question cannot be shorter than what the
#: ranker can see.
MIN_QUESTION_TOKEN = 5
class SkillError(Exception):
"""The generator refused. Carries the code, like the rest of this chain."""
def __init__(self, message: str, *, code: str) -> None:
super().__init__(message)
self.code = code
def template_path() -> Path:
"""The template this generator instantiates, or a coded refusal.
Refuses rather than falling through to a default: a generator that
silently emitted a skill built from no template would produce a file
carrying this bundle's numbers and none of the contract's sections.
"""
for candidate in (PACKAGED_TEMPLATE, AUTHORED_TEMPLATE):
if candidate.is_file():
return candidate
raise SkillError(
f"the consumption skill template was not found at {PACKAGED_TEMPLATE} "
f"or {AUTHORED_TEMPLATE}",
code="template_missing",
)
# --- The blocks the template hands over verbatim ------------------------------
#: Every template block this generator rewrites WHOLE, by exact string. Held to
#: the template by a test: an edit that moves one of these would otherwise
#: produce a skill silently missing that rewrite, which is the drift the
#: instantiated copy exists to avoid.
TEMPLATE_HEADER = """**This file is a template.** Every `<PLACEHOLDER>` is a hole a per-corpus copy
fills; none of them has a default, and a copy that leaves one unfilled is not
configured, it is unfinished. The section headings are fixed:
`okf check` reads them, and a missing one makes the skill
non-conformant rather than merely thin."""
TEMPLATE_PRE_PASS = """```sh
<PRE_PASS_COMMAND> --bundle-root <BUNDLE_ROOT> --ref <REF> --out <PAYLOAD_PATH>
```"""
TEMPLATE_CHECK = """```sh
okf check --skill <SKILL_PATH> --payload <PAYLOAD_PATH>
```"""
TEMPLATE_CONTRACT_LINE = (
"The contract this skill is held to is `docs/consumption-contract.md`. Where this"
)
TEMPLATE_EXTENSIONS = """**Extensions, if this corpus needs any.** `<EXTENSION_MARKINGS: for each, the
literal, what it means here, and which of the five it would otherwise collapse
into. Write "none" if there are none.>`"""
TEMPLATE_CONDITIONAL = """**Conditionally-written fields in this corpus.** `<CONDITIONAL_FIELDS: each
field this profile writes only when a build-time condition held, and what its
absence does and does not mean. Absence is a measurement, not a fact.>`"""
TEMPLATE_SCALING = """**Scaling.** `<COST_SCALING: whether cost tracks the question or the corpus, what
the whole bundle at this ref costs by the same instrument, and the corpus size
at which this strategy stops fitting the budget.>`"""
TEMPLATE_DENOMINATORS = """The payload reports three counts — `considered`, `withheld`, `delivered` — and
`considered == withheld + delivered`. Carry them into your output."""
TEMPLATE_ENUMERATION = (
"- **No directory enumeration** unless `<PROFILE_NAME>` says the index is derived."
)
TEMPLATE_OUTPUT = "Write to `<OUT>`. It must carry: the bundle ref; the findings, each with a"
REPLACED_BLOCKS = (
TEMPLATE_HEADER,
TEMPLATE_PRE_PASS,
TEMPLATE_CHECK,
TEMPLATE_CONTRACT_LINE,
TEMPLATE_EXTENSIONS,
TEMPLATE_CONDITIONAL,
TEMPLATE_SCALING,
TEMPLATE_DENOMINATORS,
TEMPLATE_ENUMERATION,
TEMPLATE_OUTPUT,
)
# --- What the generator measures ----------------------------------------------
def slug(value: str) -> str:
"""A Claude Code skill name from a bundle id: lowercase, hyphen-joined."""
reduced = re.sub(r"[^a-z0-9]+", "-", value.lower()).strip("-")
return reduced or "okf"
def example_question(titles: list[str]) -> str:
"""A question this bundle really answers, derived rather than invented.
The most frequent long token across the concepts' own titles, byte-sorted on
a tie. Derived because the shipped payload has to be one this bundle
produces: a question sharing no token with any concept is withheld under
`no_lexical_match` and the pre-pass refuses, so a hand-picked constant would
fail on the first bundle that does not happen to contain it.
"""
counts: Counter[str] = Counter()
for title in titles:
counts.update(
{token for token in okf_consume.normalise(title) if len(token) >= MIN_QUESTION_TOKEN}
)
if not counts:
# Every title is short or empty. Fall back to the longest title as it
# stands, which by construction matches at least its own concept.
longest = max(titles, key=lambda title: (len(title), title), default="")
if not longest:
raise SkillError(
"no concept in this bundle carries a title, so no example "
"question can be derived from it; pass --example-question",
code="no_example_question",
)
return longest
top = min(counts.items(), key=lambda item: (-item[1], item[0]))[0]
return f"Hva sier denne bundelen om {top}?"
def field_counts(concepts: list[okf_consume.Concept]) -> dict[str, int]:
"""How many concepts carry each conditional field. Set membership, never a
guess from two equal totals.
The `source_*` rows are DISCOVERED from the bundle rather than listed here,
for the same reason the excerpt carries them by prefix: a fixed list reports
a denominator for the producers someone thought of, and says nothing about
the locator this producer actually chose.
"""
discovered = sorted(
{
key
for concept in concepts
for key in concept.frontmatter
if key.startswith(okf_consume.SOURCE_KEY_PREFIX)
}
)
fields = (*CONDITIONAL_FIELDS, *discovered)
counts = {field: 0 for field in fields}
for concept in concepts:
for field in fields:
if concept.frontmatter.get(field, "").strip() or (
field == "sources" and concept.sources_present
):
counts[field] += 1
return counts
def whole_bundle_cost(concepts: list[okf_consume.Concept]) -> int:
"""What every concept in this bundle would cost by the gate's own
instrument, if a single answer delivered all of them."""
total = 0
for concept in concepts:
excerpt = okf_consume.excerpt_for(concept)
if excerpt is not None:
total += okf_consume.excerpt_weight(excerpt)
return total
def directory_control(bundle_root: Path, *, profile: BundleProfile) -> tuple[int, int]:
"""The index walk against the method § 9.2 forbids the CONSUMER from using.
Run HERE, once, at generation time -- never on the question path. § 9.2
binds a consumer reaching for context at query time; a build-time control is
what turns "the walk loses nothing" from an assumption into a number the
generated skill can quote.
"""
walked = len(okf_consume.enumerate_concepts(bundle_root, profile=profile))
suffix = profile.paths.concept_suffix
reserved = {profile.index.name, "log.md"}
on_disk = len([path for path in bundle_root.rglob(f"*{suffix}") if path.name not in reserved])
return walked, on_disk
# --- The instantiation --------------------------------------------------------
def render(
bundle_root: Path,
*,
out: Path,
profile: BundleProfile = okf_consume.DEFAULT_PROFILE,
question: str | None = None,
) -> tuple[str, dict[str, object]]:
"""The skill text and the example payload that proves it, for one bundle."""
bundle_root = bundle_root.resolve()
bundle_id = okf_consume.root_bundle_id_of(bundle_root, profile=profile)
ref = okf_consume.bundle_ref(bundle_root, profile=profile)
concept_ids = okf_consume.enumerate_concepts(bundle_root, profile=profile)
concepts = [
okf_consume.read_concept(
bundle_root / f"{concept_id}{profile.paths.concept_suffix}",
bundle_root=bundle_root,
root_bundle_id=bundle_id,
)
for concept_id in concept_ids
]
if not concepts:
raise SkillError(
f"{bundle_root} has an index but no concept under it; a skill for an "
"empty bundle would state denominators of zero it never measured",
code="bundle_empty",
)
total = len(concepts)
asked = question or example_question([concept.title for concept in concepts])
payload = okf_consume.build_payload(bundle_root, question=asked, profile=profile)
counts = field_counts(concepts)
walked, on_disk = directory_control(bundle_root, profile=profile)
cost = whole_bundle_cost(concepts)
denominators = payload["denominators"]
assert isinstance(denominators, dict)
budget = payload["budget"]
assert isinstance(budget, dict)
withheld = payload["withheld"]
assert isinstance(withheld, list)
bookkeeping = okf_consume.measure(json.dumps(withheld, ensure_ascii=False))
per_withheld = bookkeeping / len(withheld) if withheld else 0.0
breaking = int(okf_consume.DEFAULT_LIMIT / per_withheld) if per_withheld else 0
name = f"{slug(bundle_id)}-consume"
text = template_path().read_text(encoding="utf-8")
text = text.split("---\n", 2)[2]
text = _rewrite(
text,
bundle_root=bundle_root,
skill_path=out.resolve() / "SKILL.md",
bundle_id=bundle_id,
ref=ref,
name=name,
total=total,
counts=counts,
walked=walked,
on_disk=on_disk,
cost=cost,
asked=asked,
spent=int(budget["spent"]),
delivered=int(denominators["delivered"]),
bookkeeping=bookkeeping,
breaking=breaking,
)
header = f"---\nname: {name}\ndescription: {_description(bundle_id, total, ref)}\n---\n"
return header + text, payload
def _description(bundle_id: str, total: int, ref: str) -> str:
return (
f"Answer one question about the OKF bundle `{bundle_id}` ({total} concepts, "
f"ref {ref}) from a bounded payload assembled by a deterministic pre-pass, "
"marking every claim with its source, its title and its provenance locator. "
"Use whenever a question is about what that bundle's documents require, say "
"or contain. Generated by `okf skill`; every value below is measured "
"against this bundle at this ref."
)
def _rewrite(
text: str,
*,
bundle_root: Path,
skill_path: Path,
bundle_id: str,
ref: str,
name: str,
total: int,
counts: dict[str, int],
walked: int,
on_disk: int,
cost: int,
asked: str,
spent: int,
delivered: int,
bookkeeping: int,
breaking: int,
) -> str:
replacements: list[tuple[str, str]] = [
(
TEMPLATE_HEADER,
"**This file is an instantiated copy of "
"`skills/okf-consume-template/SKILL.md`,** generated by "
f"`okf skill` for one bundle: `{bundle_id}` at ref\n"
f"`{ref}`. Every value below was measured against those bytes. If the\n"
"bundle moves, the ref moves with it and this file is stale — regenerate\n"
"it rather than editing a number here. The section headings are fixed:\n"
"the contract checker reads them by name.",
),
(
TEMPLATE_PRE_PASS,
"```sh\n"
f"{PRE_PASS_COMMAND} \\\n"
f" {bundle_root} \\\n"
' --question "your question" \\\n'
f" --ref {ref} \\\n"
" --out /tmp/payload.json\n"
"```\n\n"
"`--ref` is an **assertion**, never an override: the identity is computed\n"
"from the bytes either way, and a mismatch refuses. Read the pre-pass's\n"
"own exit status, which carries three values: **0** a payload was written,\n"
"**1** the run happened and refused, **2** the run did not happen at all.",
),
(
TEMPLATE_CHECK,
"```sh\n"
f"{CHECKER_COMMAND} \\\n"
f" --skill {skill_path} \\\n"
" --payload /tmp/payload.json\n"
"```",
),
(
TEMPLATE_CONTRACT_LINE,
f"The contract this skill is held to is `{CONTRACT}`. Where this",
),
(
TEMPLATE_EXTENSIONS,
"**Extensions, if this corpus needs any: none.** This generated skill adds\n"
"no marking to the required five. § 4.3 makes the undeclared extension the\n"
"defect, so the absence is stated rather than left to be inferred — and a\n"
"corpus that does need a sixth needs a hand-edited copy that declares it.",
),
(
TEMPLATE_CONDITIONAL,
_conditional_table(total, counts),
),
(
TEMPLATE_SCALING,
_scaling(
total=total,
cost=cost,
asked=asked,
spent=spent,
delivered=delivered,
bookkeeping=bookkeeping,
breaking=breaking,
),
),
(
TEMPLATE_DENOMINATORS,
_denominators(total, asked=asked, delivered=delivered),
),
(
TEMPLATE_ENUMERATION,
_enumeration(walked, on_disk),
),
(
TEMPLATE_OUTPUT,
"Write to the path the caller names, or to your answer if none was named.\n"
"It must carry: the bundle ref; the findings, each with a",
),
("`<CORPUS>` bundle", f"`{bundle_id}` bundle"),
("# <CORPUS> consumption", f"# {bundle_id} consumption"),
("<BUDGET_LIMIT>", str(okf_consume.DEFAULT_LIMIT)),
("<BUDGET_UNIT>", okf_consume.BUDGET_UNIT),
("<BUDGET_INSTRUMENT>", okf_consume.BUDGET_INSTRUMENT),
("<KNOWN_POSITIVE_CASE>", okf_consume.KNOWN_POSITIVE_CASE),
("<KNOWN_POSITIVE_EXPECTED>", str(okf_consume.KNOWN_POSITIVE_EXPECTED)),
]
for old, new in replacements:
if old not in text:
raise SkillError(
f"the template no longer carries the block this generator rewrites: {old[:70]!r}",
code="template_drift",
)
text = text.replace(old, new)
assert name # kept in the signature so a caller cannot forget to name the skill
return text
def _conditional_table(total: int, counts: dict[str, int]) -> str:
rows = "\n".join(
f"| `{field}` | **{count} of {total}** | "
f"{'the producer wrote none for that concept' if count else 'no concept in this bundle carries it'} | "
"that the source document lacks what the field asserts |"
for field, count in counts.items()
)
return (
"**Conditionally-written fields in this bundle, with what each absence does\n"
"and does not mean.** Every count is over the same denominator — "
f"**{total} concepts**, the set the index walk reaches. § 6.4: absence is a\n"
"measurement about the producer, never a fact about the source.\n\n"
"| Field | Present on | Absence means | Absence does NOT mean |\n"
"|---|---|---|---|\n"
f"{rows}\n\n"
"A field present on **0 of "
f"{total}** is a measured zero, not an unmeasured one: the count was taken\n"
"over every concept, and it is reported so a negative claim resting on it\n"
"carries its denominator."
)
def _scaling(
*,
total: int,
cost: int,
asked: str,
spent: int,
delivered: int,
bookkeeping: int,
breaking: int,
) -> str:
share = (spent / cost * 100) if cost else 0.0
return (
"**Scaling. Cost tracks the question, not the corpus.** Measured on this\n"
f"bundle at generation time, with the question `{asked}`: the delivered set\n"
f"was **{delivered} excerpts** costing **{spent} {okf_consume.BUDGET_UNIT}**,\n"
f"against a whole bundle that would cost **{cost}** by the same instrument if\n"
f"one answer delivered all {total} concepts — so that answer was about\n"
f"**{share:.1f} %** of the corpus. One question is one measurement: a\n"
"different question moves `spent` and this figure with it.\n\n"
"**The breaking point, stated so it can be observed to have been passed.**\n"
"The `withheld` list carries one entry per considered concept and grows\n"
f"linearly: here it is **{bookkeeping} bytes** for {total} concepts. At roughly\n"
f"**{breaking} concepts** the bookkeeping alone reaches the "
f"{okf_consume.DEFAULT_LIMIT}-byte\n"
"limit, and although it is not counted against `spent`, a payload whose\n"
"bookkeeping dwarfs its content has stopped being a cut. The pre-pass also\n"
"reads every concept body on every run, so the same growth is a wall-clock\n"
"cost with no precomputed index behind it."
)
def _denominators(total: int, *, asked: str, delivered: int) -> str:
return (
"The payload reports three counts — `considered`, `withheld`, `delivered` — and\n"
"`considered == withheld + delivered`. Carry them into your output.\n\n"
f"For this bundle `considered` is **{total}**, every concept the index walk\n"
"reaches, never the post-ranking shortlist. A concept dropped at the ranking\n"
"stage is `withheld` **with its rule**, not invisible, and the rules are a\n"
"closed set of six: `verdict_layer_excluded` (a verdict-layer file, § 9.1),\n"
"`verified_unreadable` (a `verified` value this reader cannot decode, so no\n"
"tier can be derived), `no_lexical_match` (the concept shares no token with\n"
"the question), `over_budget_alone` (one excerpt exceeds the whole limit),\n"
"`below_k` (ranked outside the shortlist the cut considers) and\n"
"`over_budget_after_knapsack` (it ranked inside the shortlist and the pack\n"
"had no room). Naming the rule is what makes a drop visible.\n\n"
"**One limitation to carry into every negative claim.** `no_lexical_match` is\n"
'a per-concept relevance drop, not a whole-question "this bundle has no\n'
f'answer" gate: on the generation question `{asked}` it still returned\n'
f"{delivered} excerpts. **An empty `excerpts` list is evidence of absence; a\n"
"full one is not evidence of presence.** When the delivered excerpts do not\n"
"actually answer the question, say `[sourced-not-sufficient]` and report that\n"
"the cut found nothing responsive."
)
def _enumeration(walked: int, on_disk: int) -> str:
agreement = (
f"which costs nothing here: the walk reaches **{walked}** concepts and a\n"
f" directory walk finds **{on_disk}**"
if walked == on_disk
else f"and the two disagree — the walk reaches **{walked}** concepts where a\n"
f" directory walk finds **{on_disk}**, so some concept is unreachable through\n"
" the index and the bundle's producer should be told"
)
return (
"- **No directory enumeration.** This bundle is read under the\n"
f" `{PROFILE_NAME}` profile, whose index policy declares\n"
" `entries_match_directory = False`, so § 9.2's permission does not apply.\n"
f" The pre-pass walks the **index tree** instead, {agreement}\n"
" (controlled once at generation time, never on the question path). Do not\n"
" enumerate a directory yourself either."
)
def generate(
bundle_root: Path,
*,
out: Path,
profile: BundleProfile = okf_consume.DEFAULT_PROFILE,
question: str | None = None,
force: bool = False,
) -> Path:
"""Write `out/SKILL.md` and its reference payload. Returns the skill path."""
target = out / "SKILL.md"
if target.exists() and not force:
raise SkillError(
f"{target} already exists; pass --force to replace it. A silent "
"overwrite would destroy a hand-edited copy whose extra measurements "
"this generator cannot reproduce",
code="target_occupied",
)
text, payload = render(bundle_root, out=out, profile=profile, question=question)
(out / "references").mkdir(parents=True, exist_ok=True)
target.write_text(text, encoding="utf-8")
(out / "references" / "example-payload.json").write_text(
okf_consume.serialise(payload), encoding="utf-8"
)
return target
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument("bundle", type=Path, help="the OKF bundle to instantiate a skill for")
parser.add_argument(
"--out", type=Path, required=True, help="the skill directory to write (SKILL.md inside)"
)
parser.add_argument(
"--example-question",
default=None,
help="the question the shipped reference payload answers. Derived from the "
"bundle's own titles when omitted",
)
parser.add_argument(
"--force", action="store_true", help="replace an existing SKILL.md at --out"
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
try:
written = generate(
args.bundle, out=args.out, question=args.example_question, force=args.force
)
except okf_consume.ConsumeError as exc:
print(f"refused ({exc.code}): {exc}")
return 1
except SkillError as exc:
print(f"refused ({exc.code}): {exc}")
return 1
except OSError as exc:
print(f"the run did not happen: {exc}")
return 2
print(f"wrote {written}")
return 0
if __name__ == "__main__":
raise SystemExit(main())

91
tests/test_cli_build.py
View file

@ -763,6 +763,7 @@ def test_the_build_default_is_now_arm_d_plus_arm_f(tmp_path: Path) -> None:
PROPOSED_AT,
"--outline-run",
"3",
"--table-grid",
"--unit-fold",
)
== 0
@ -787,6 +788,7 @@ def test_each_arm_in_the_default_has_an_explicit_opt_out(tmp_path: Path) -> None
PROPOSED_AT,
"--outline-run",
"0",
"--no-table-grid",
"--no-unit-fold",
)
== 0
@ -802,6 +804,7 @@ def test_each_arm_in_the_default_has_an_explicit_opt_out(tmp_path: Path) -> None
PROPOSED_AT,
"--outline-run",
"0",
"--no-table-grid",
"--no-unit-fold",
)
== 0
@ -831,7 +834,9 @@ def test_a_bundle_built_with_no_flags_is_byte_identical_to_the_shipped_one(
"""
inbox = inbox_with_subdirectories(tmp_path)
armed = two_script_bundle(
inbox, tmp_path / "reference-armed", proposer_flags=("--outline-run", "3", "--unit-fold")
inbox,
tmp_path / "reference-armed",
proposer_flags=("--outline-run", "3", "--table-grid", "--unit-fold"),
)
bundle = tmp_path / "cli-bundle-2"
assert build(inbox, bundle, "--ingested-at", INGESTED_AT, "--proposed-at", PROPOSED_AT) == 0
@ -849,8 +854,92 @@ def test_a_bundle_built_with_no_flags_is_byte_identical_to_the_shipped_one(
PROPOSED_AT,
"--outline-run",
"0",
"--no-table-grid",
"--no-unit-fold",
)
== 0
)
assert tree(opted_out) == tree(plain)
# --- the default moved again: Arm E joined it (2026-09-08, round 4) --------
#
# Round 3 moved the default to Arm D plus Arm F and left `--table-grid` a flag.
# Measured afterwards on the operator's five-document folder, that combination
# is Arm F WITHOUT a joined table to fold: the fold's table clause folds a
# table back into the heading that introduces it, and with Arm E off a grid
# table is not one block but one block per rule line, so there is nothing whole
# to fold. The published "Arm F matches 5 of 12" was measured with
# `--table-grid` ON; the shipped default scored 2 of 12, and `docx` 0 of 3.
#
# Arm E therefore joins the default, with the same explicit opt-out every arm
# in it has. `--keep-table-heading` does NOT join: measured on two K2 bundles
# it buys 35 bytes and zero rank positions.
GRID_SHEET_DOCUMENT = {
"theta.md": (
"## 7 Romskjema\n\nInnledende avsnitt.\n\n"
"+-------+-------+\n| Navn | Verdi |\n+=======+=======+\n"
"| Areal | 120 |\n+-------+-------+\n| Hoyde | 3 |\n+-------+-------+\n"
)
}
def inbox_for_grid_sheet(root: Path) -> Path:
inbox = root / "grid-inbox"
inbox.mkdir(parents=True, exist_ok=True)
for name, body in GRID_SHEET_DOCUMENT.items():
(inbox / name).write_text(body, encoding="utf-8", newline="")
return inbox
def test_the_build_default_is_now_arm_d_plus_arm_e_plus_arm_f(tmp_path: Path) -> None:
"""The red test for round 4's move: no flags must EQUAL the three flags.
Measured on a grid table, which is the one shape Arm E decides: with the
arm off, each rule line closes the block, so the sheet lands as one concept
per row group with a title naming a LINE NUMBER -- `Tabell linje 6` -- and
the fold has no whole table to fold back into the heading above it. That is
the defect the round-3 default shipped with, and it is asserted on titles
rather than a count because a wrong cut can still produce a right number.
"""
inbox = inbox_for_grid_sheet(tmp_path)
plans = tmp_path / "plans-grid-default"
assert build(inbox, tmp_path / "grid-default", "--plans-dir", str(plans)) == 0
assert plan_titles(plans) == ["7 Romskjema"]
explicit = tmp_path / "plans-grid-explicit"
assert (
build(
inbox,
tmp_path / "grid-explicit",
"--plans-dir",
str(explicit),
"--outline-run",
"3",
"--table-grid",
"--unit-fold",
)
== 0
)
assert plan_titles(explicit) == plan_titles(plans)
def test_arm_e_in_the_default_has_an_explicit_opt_out(tmp_path: Path) -> None:
"""The other half: `--no-table-grid` gets the pre-move cut back, and says so.
Same rule every other arm in the default follows -- a default a caller
cannot turn off is not a default. The titles asserted here are the ones the
round-3 default produced on this fixture.
"""
inbox = inbox_for_grid_sheet(tmp_path)
plans = tmp_path / "plans-grid-opt-out"
assert (
build(inbox, tmp_path / "grid-opt-out", "--plans-dir", str(plans), "--no-table-grid") == 0
)
assert plan_titles(plans) == [
"7 Romskjema",
"Tabell linje 6",
"Tabell linje 8",
"Tabell linje 10",
]

49
tests/test_okf_consume.py
View file

@ -1182,20 +1182,39 @@ def test_the_readme_consume_section_states_the_rule_count_the_code_emits() -> No
def test_the_readme_recipe_names_only_commands_this_repository_ships() -> None:
# Every command in the "Consume in Claude Code" section was run in the
# session that wrote it. This test cannot re-run them; what it can hold is
# that each script the recipe invokes still exists under the path it names.
"""Every command the recipe invokes must exist. What "exist" means MOVED.
Until 2026-09-08 (O5) the recipe told a reader to run `python3
tools/<script>.py`, and this test held that each of those three files was
on disk. The recipe now names `okf` subcommands, because a reader who
installed this library has no `tools/` directory at all -- so the check
that means the same thing is that each subcommand is one the CLI
registers. That is a stronger claim than a file existing: a script can be
present and unreachable from the installed command, which is exactly the
defect O5 fixed.
The section's own heading is the fixed point, not the form of the commands
inside it.
"""
readme = (PROJECT_ROOT / "README.md").read_text(encoding="utf-8")
recipe = readme.split("## Consume in Claude Code", 1)[1].split("\n## ", 1)[0]
scripts = set(re.findall(r"python3 (tools/\S+\.py)", recipe))
assert scripts == {
"tools/okf_skill.py",
"tools/okf_consume.py",
"tools/okf_contract_check.py",
}, scripts
for script in scripts:
assert (PROJECT_ROOT / script).is_file(), script
assert "okf build " in recipe
# No route back to the old form: a `python3 tools/...` line in the recipe
# is a line a reader without this repository cannot run.
assert re.findall(r"python3 (tools/\S+\.py)", recipe) == []
invoked = set(re.findall(r"^okf ([a-z]+)", recipe, flags=re.MULTILINE))
assert invoked == {"build", "consume", "check", "skill", "project"}, invoked
listed = subprocess.run(
[sys.executable, "-m", "llm_ingestion_okf.cli", "--help"],
capture_output=True,
text=True,
cwd=PROJECT_ROOT,
)
assert listed.returncode == 0
for command in invoked:
assert command in listed.stdout, command
# --- Step 11: the measurement scorer -----------------------------------------
@ -1340,7 +1359,11 @@ def test_every_vocabulary_member_is_long_enough_to_ever_match() -> None:
def test_the_vocabulary_is_one_list_and_names_no_corpus_document() -> None:
source = (PROJECT_ROOT / "tools" / "okf_consume.py").read_text(encoding="utf-8")
# The PATH moved on 2026-09-08 (O5) and nothing else in this test did. The
# pre-pass is `src/llm_ingestion_okf/consume.py` now; `tools/okf_consume.py`
# is an alias to it, so reading the old path would have measured a wrapper
# and passed on a file with no vocabulary in it at all.
source = (PROJECT_ROOT / "src" / "llm_ingestion_okf" / "consume.py").read_text(encoding="utf-8")
assert source.count("COST_VOCABULARY = (") == 1
leak = re.compile(r"del-ii-bilag|del-i-vedlegg|prisskjema|prissammenstilling|stange", re.I)
assert leak.findall(source) == []

187
tests/test_okf_project.py Normal file
View file

@ -0,0 +1,187 @@
"""`okf project`: one folder in, one bundle plus one skill out.
The command adds no rule and owns no flag that changes a bundle's bytes, so
these tests are mostly about that: the project bundle must be the SAME bytes
`okf build` writes for the same folder at the same stamp, or there are two
build paths and the reports are pinned to one of them.
"""
from __future__ import annotations
import hashlib
import subprocess
import sys
from pathlib import Path
import pytest
PROJECT_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(PROJECT_ROOT / "src"))
from llm_ingestion_okf import project # noqa: E402
from llm_ingestion_okf.cli import build # noqa: E402
from llm_ingestion_okf.cli import main as okf_main # noqa: E402
from llm_ingestion_okf.errors import IngestError # noqa: E402
DOCUMENTS = {
"krav.md": (
"## 4 Grunnforhold\n\nGrunnen er morene over berg.\n\n"
"### 4.1 Loesmasser\n\nLoesmassene er telefarlige.\n"
),
"notat.md": "Et notat uten overskrift, uten tabell og uten nummerering.\n",
}
@pytest.fixture
def folder(tmp_path: Path) -> Path:
target = tmp_path / "Mine Dokumenter"
target.mkdir()
for name, body in DOCUMENTS.items():
(target / name).write_text(body, encoding="utf-8", newline="")
return target
def tree(root: Path) -> dict[str, str]:
return {
path.relative_to(root).as_posix(): hashlib.sha256(path.read_bytes()).hexdigest()
for path in sorted(root.rglob("*"))
if path.is_file()
}
def test_the_project_bundle_is_the_bytes_okf_build_writes(folder: Path, tmp_path: Path) -> None:
"""The invariant the whole command rests on: ONE build path, not two.
`okf project` runs `okf build` with this package's default and no flag list
of its own. If it ever grew one, a project bundle and a build bundle of the
same folder would differ, and every measurement report pinned to the build
path would be describing a bundle nobody produces.
"""
out = tmp_path / "project"
bundle, _, _ = project.create(folder, out=out)
reference = tmp_path / "reference"
build(folder, reference, bundle_id="mine-dokumenter", okf_version="0.2")
assert tree(bundle) == tree(reference)
def test_the_id_defaults_to_the_folder_name_in_the_id_grammar(folder: Path, tmp_path: Path) -> None:
out = tmp_path / "project"
bundle, skill_path, _ = project.create(folder, out=out)
assert bundle == out / ".okf" / "mine-dokumenter"
assert skill_path == out / ".claude" / "skills" / "mine-dokumenter-consume" / "SKILL.md"
assert bundle.is_dir() and skill_path.is_file()
def test_a_named_id_is_used_verbatim(folder: Path, tmp_path: Path) -> None:
out = tmp_path / "project"
bundle, skill_path, _ = project.create(folder, out=out, bundle_id="anbud-2026")
assert bundle.name == "anbud-2026"
assert skill_path.parent.name == "anbud-2026-consume"
def test_a_folder_name_that_reduces_to_nothing_refuses_by_code(tmp_path: Path) -> None:
"""A refusal with a code, not a bundle called `""`.
A folder named only in punctuation reduces to the empty string, and an
empty bundle id would produce a bundle whose concepts join on nothing.
"""
weird = tmp_path / "..."
weird.mkdir()
(weird / "a.md").write_text("# A\n\nKropp.\n", encoding="utf-8", newline="")
with pytest.raises(IngestError) as caught:
project.create(weird, out=tmp_path / "project")
assert caught.value.code == "manifest_invalid"
def test_the_folder_name_is_normalised_before_it_is_reduced(tmp_path: Path) -> None:
"""NFC first, for the reason the rest of this package normalises first.
macOS hands a filename over decomposed, so `é` arrives as `e` plus a
combining acute. Reduced without normalising, the same visible folder name
produces two different bundle ids depending on which form it arrived in.
"""
assert project.slug("Prosjekt É") == project.slug("Prosjekt É")
def test_the_summary_names_the_documents_that_landed_whole(folder: Path, tmp_path: Path) -> None:
"""SS 6.4 discipline applied to a summary: the number carries its denominator.
`notat.md` has no heading, no table and no numbered outline, so the rules
find no boundary and it lands as one concept. A reader who is told only
"3 concepts" cannot tell that asking about that document returns the whole
of it as one excerpt.
"""
out = tmp_path / "project"
_, _, summary = project.create(folder, out=out)
assert "Read 2 document(s)" in summary
assert "1 of 2 document(s) landed WHOLE" in summary
assert "notat.md" in summary
assert "krav.md" not in summary
assert "[sourced-not-sufficient]" in summary
assert f"NEXT: start claude again in {out}" in summary
def test_a_document_that_is_in_the_bundle_is_not_reported_as_missing(
folder: Path, tmp_path: Path
) -> None:
"""The known-positive for `inventory`'s first list.
Its own control: with every document ingested the list must be empty, and
with the bundle read against a DIFFERENT folder every document must appear.
A search that cannot find would report an empty list either way.
"""
out = tmp_path / "project"
bundle, _, _ = project.create(folder, out=out)
missing, whole = project.inventory(folder, bundle)
assert missing == ()
assert whole == ("notat.md",)
other = tmp_path / "other"
other.mkdir()
(other / "fremmed.md").write_text("# Fremmed\n\nKropp.\n", encoding="utf-8", newline="")
stranger, _ = project.inventory(other, bundle)
assert stranger == ("fremmed.md",)
def test_the_generated_skill_names_no_path_into_this_repository(
folder: Path, tmp_path: Path
) -> None:
"""O5's whole point, asserted where a user actually meets it.
The known-positive runs first: the string this searches for occurs in the
environment running the test, so a zero means the generator kept it out.
"""
out = tmp_path / "project"
_, skill_path, _ = project.create(folder, out=out)
text = skill_path.read_text(encoding="utf-8")
assert str(PROJECT_ROOT) in str(Path(__file__).resolve())
assert str(PROJECT_ROOT) not in text
assert "okf consume" in text
assert "okf check" in text
def test_the_subcommand_exists_and_reports_zero(folder: Path, tmp_path: Path) -> None:
assert okf_main(["project", str(folder), "--out", str(tmp_path / "project")]) == 0
def test_a_missing_folder_is_two_and_not_one(tmp_path: Path) -> None:
"""Three exit codes, not two: an unread folder is not a refused build."""
assert okf_main(["project", str(tmp_path / "nope"), "--out", str(tmp_path / "p")]) == 2
def test_the_installed_command_reaches_every_subcommand() -> None:
"""`okf --help` must LIST them, or a reader has to be told they exist.
The dispatch happens before argparse, so without the registration in
`parse_args` these four would work and be invisible.
"""
listed = subprocess.run(
[sys.executable, "-m", "llm_ingestion_okf.cli", "--help"],
capture_output=True,
text=True,
cwd=PROJECT_ROOT,
)
assert listed.returncode == 0
for command in ("build", "consume", "check", "skill", "project"):
assert command in listed.stdout, command

46
tests/test_okf_skill.py
View file

@ -89,17 +89,47 @@ def test_a_skill_generated_for_one_bundle_does_not_name_the_other(tmp_path: Path
@pytest.mark.parametrize("bundle", BUNDLES, ids=lambda path: path.name)
def test_the_generated_commands_are_absolute_so_a_caller_outside_this_repo_can_run_them(
bundle: Path, tmp_path: Path
) -> None:
# The skill is copied into someone else's `.claude/skills/`, where a
# relative `tools/okf_consume.py` resolves to nothing.
def test_the_generated_commands_name_this_repository_nowhere(bundle: Path, tmp_path: Path) -> None:
"""O5's red measurement, made a test: no path into a checkout, anywhere.
**This assertion REPLACES its own opposite, and the replacement is the
point.** The test here until 2026-09-08 required the emitted commands to be
ABSOLUTE, on the reasoning that a skill copied into someone else's
`.claude/skills/` cannot resolve a relative `tools/okf_consume.py`. Both
halves of that were true and the conclusion was still wrong: an absolute
path into THIS clone is not portable either, it is merely portable-looking.
Measured before the move, a skill generated from a checkout carried four
lines naming this checkout by absolute path, two of them the commands a
reader is told to run -- so the skill could not be moved, shared, or run by
anyone without that clone at that exact path.
It also caught nothing by then: the regex it looped over matched zero lines
once the commands stopped being `python3 <file>.py`, so it was green over
an empty set. The assertion below has a denominator that cannot go to zero.
"""
written = _generate(bundle, tmp_path / bundle.name)
text = written.read_text(encoding="utf-8")
for command in re.findall(r"^\S*python3? (\S+\.py)", text, flags=re.MULTILINE):
assert Path(command).is_absolute(), command
assert Path(command).is_file(), command
# The BUNDLE root is the one absolute path that belongs here: it points at
# the caller's data. These fixtures happen to live inside this repository,
# so it is removed before the search -- otherwise the search would find the
# repository root inside the one path allowed to carry it.
assert str(bundle.resolve()) in text
rest = text.replace(str(bundle.resolve()), "<BUNDLE>")
# The known-positive for the search: the string it hunts for DOES occur in
# the environment running it, so the zero below means the generator kept it
# out rather than the search being unable to find it.
assert str(PROJECT_ROOT) in str(Path(__file__).resolve())
assert str(PROJECT_ROOT) not in rest
# No file under `tools/` at all, and not only the two commands: the
# attribution lines named the generator by path too, which is a file the
# reader does not have either.
assert "tools/" not in rest
# And what it names instead: commands resolved by PATH after an install.
assert "\nokf consume \\\n" in text
assert "\nokf check \\\n" in text
@pytest.mark.parametrize("bundle", BUNDLES, ids=lambda path: path.name)

1693
tools/okf_consume.py
View file

File diff suppressed because it is too large Load diff

507
tools/okf_contract_check.py
View file

@ -1,497 +1,34 @@
"""Check a consumption skill and one pre-pass payload against the contract.
#!/usr/bin/env python3
"""Thin entry point. The implementation is `llm_ingestion_okf.contract_check`.
`docs/consumption-contract.md` is normative prose. Prose has no test, so this
command reads the mechanically checkable half of it: the payload shape (SS 3, 5,
6, 7, 8) and the fixed vocabulary a conformant `SKILL.md` must carry (SS 4, 6).
It moved into the package on 2026-09-08 (O5): the generated consumption skill's
check step named this file by absolute path into a checkout, so the one command
that tells a reader whether their payload conforms was unreachable from an
install. The command is now `okf check`, on PATH.
**What it cannot see, said here rather than implied.** The division of labour
(SS 2) and the prohibitions (SS 9) are properties of a RUN -- whether an agent
went looking for context the pre-pass withheld, whether a retrieval tool was
pointed at the verdict layer. No static check reaches them. Conformance here is
the floor, never the proof, and a green run means the payload carries what a
claim would have to rest on -- not that the claim was made honestly.
This file stays because the published reproduction blocks name it, and a
measurement whose command no longer runs is a measurement nobody can repeat.
**Every rule has its own code.** One "invalid" verdict over fourteen different
defects is a diagnostic no caller can act on. The codes are the contract's
paragraphs made addressable.
**The report carries denominators**, because the contract requires them of its
consumers and a checker exempting itself would be stating the rule it breaks:
how many rules ran, over how many excerpts and withheld entries.
Exit codes are three, not two: 0 conformant, 1 non-conformant, 2 the check did
not run. Collapsing 2 into 1 would report an unread file as a failed check.
It lives outside `src/`, so it never enters a wheel and no consumer's install
surface changes because it exists.
No logic here, deliberately: a second copy of a contract rule is a second thing
that can pass while the shipped one fails.
"""
from __future__ import annotations
import argparse
import json
from collections.abc import Callable, Mapping, Sequence
from dataclasses import dataclass
import sys
from pathlib import Path
from typing import Any
#: SS 4.1. Five literals, spelled exactly. `[unverifiable-from-bundle]` is one
#: literal string -- no variants, no translations -- so this is a membership
#: test on the literal, never on a translated concept name.
REQUIRED_MARKINGS = (
"extracted",
"derived",
"[unverifiable-from-bundle]",
"[unread]",
"[sourced-not-sufficient]",
)
#: SS 6.1. A closed set of three, and the third is a real state: `unknown` says
#: the concept carries no `adjudication` key, which is what an older bundle
#: looks like. Collapsing it into `proposed` or into absence reports "not
#: judged" where the truth is "we cannot tell", and only one of those is a fact
#: about the concept.
ADJUDICATION_STATES = ("proposed", "adjudicated", "unknown")
#: SS 6.2, from SPEC SS 5.3: derived from `verified`, lowest to highest.
TRUST_TIERS = ("unverified", "machine-confirmed", "human-reviewed")
#: The level-2 headings a conformant skill carries. Checked by name because the
#: alternative -- searching the prose for the obligation -- passes on any
#: document that happens to use the words.
REQUIRED_SECTIONS = (
"Pre-pass",
"Division of labour",
"Markings",
"States",
"Budget",
"Denominators",
"Prohibitions",
)
@dataclass(frozen=True)
class Finding:
"""One contract paragraph, unmet, named by its code."""
code: str
message: str
@dataclass(frozen=True)
class Report:
"""Findings plus the denominators they were measured over."""
findings: tuple[Finding, ...]
rules_evaluated: int
excerpts_examined: int
withheld_examined: int
def render(self) -> str:
denominator = (
f"{self.rules_evaluated} rules over {self.excerpts_examined} excerpts "
f"and {self.withheld_examined} withheld entries"
)
if not self.findings:
return f"conformant: {denominator}, 0 findings"
lines = [f"NOT conformant: {denominator}, {len(self.findings)} findings"]
lines += [f" {finding.code}: {finding.message}" for finding in self.findings]
return "\n".join(lines)
@dataclass(frozen=True)
class Context:
"""What every rule reads. `payload` is empty when the payload is not a
mapping at all, so each rule stays a total function over its input."""
skill: str
payload: Mapping[str, Any]
payload_is_mapping: bool
def _mapping(value: object) -> Mapping[str, Any]:
return value if isinstance(value, Mapping) else {}
def _sequence(value: object) -> Sequence[Any]:
if isinstance(value, Sequence) and not isinstance(value, (str, bytes)):
return value
return ()
def _text(value: object) -> str:
return value if isinstance(value, str) else ""
def _whole(value: object) -> int | None:
# `bool` is an `int` in Python and would make `True` a budget. Excluded by
# name rather than trusted not to arrive.
return value if isinstance(value, int) and not isinstance(value, bool) else None
# --- Payload rules -----------------------------------------------------------
def rule_payload_shape(ctx: Context) -> list[Finding]:
if ctx.payload_is_mapping:
return []
return [Finding("payload_invalid", "the payload is not a JSON object (SS 8)")]
def rule_contract_version(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping or _text(ctx.payload.get("contract")):
return []
return [
Finding(
"contract_unversioned",
"the payload names no `contract` revision, so a reader cannot tell "
"which revision it is holding (SS 8.2)",
)
]
def rule_bundle_ref(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
bundle = _mapping(ctx.payload.get("bundle"))
findings = []
if not _text(bundle.get("ref")):
findings.append(
Finding(
"ref_missing",
"the payload names no bundle `ref`; a version is the producer's "
"assertion, a ref is a fact about bytes (SS 3.3)",
)
)
if not _text(bundle.get("bundle_id")):
findings.append(Finding("ref_missing", "the payload names no `bundle.bundle_id` (SS 3.1)"))
return findings
def rule_excerpt_source_marking(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
findings = []
for position, raw in enumerate(_sequence(ctx.payload.get("excerpts"))):
excerpt = _mapping(raw)
for key in ("bundle_id", "concept_id", "sha256"):
if not _text(excerpt.get(key)):
findings.append(
Finding(
"source_marking_incomplete",
f"excerpt {position} carries no {key!r}; identity across "
"bundles is the (bundle_id, concept_id) tuple with the "
"digest of the bytes it was taken from (SS 3.1, SS 3.2)",
)
)
return findings
def rule_excerpt_named(ctx: Context) -> list[Finding]:
"""SS 8: every excerpt carries a `title`.
Added 2026-09-08 on a measurement, not a preference: `portfolio-optimiser`
ran three paid arms in which the pre-pass delivered the gold concept at rank
1 of 8 on 3 of 3 bundles and the model answered correctly on 1 of 3, because
the excerpt carried `concept_id` and `text` and nothing a reader could name
the document by. A payload no answer can cite from is not conformant; the
identity fields are what SS 3.1's tuple is FOR.
"""
if not ctx.payload_is_mapping:
return []
return [
Finding(
"excerpt_unnamed",
f"excerpt {position} carries no 'title'; an excerpt a reader cannot "
"name is one an answer cannot cite, whatever its rank (SS 8)",
)
for position, raw in enumerate(_sequence(ctx.payload.get("excerpts")))
if not _text(_mapping(raw).get("title"))
]
def rule_excerpt_states(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
findings = []
closed: tuple[tuple[str, tuple[str, ...]], ...] = (
("adjudication", ADJUDICATION_STATES),
("trust_tier", TRUST_TIERS),
)
for position, raw in enumerate(_sequence(ctx.payload.get("excerpts"))):
excerpt = _mapping(raw)
for key, allowed in closed:
if key not in excerpt:
findings.append(
Finding(
"state_missing",
f"excerpt {position} carries no {key!r}; the third state is "
"written explicitly, never omitted -- omission collapses "
'"we cannot tell" into a fact about the concept (SS 6.1)',
)
)
continue
value = _text(excerpt.get(key))
if value not in allowed:
findings.append(
Finding(
"state_not_in_set",
f"excerpt {position} has {key}={value!r}, outside the closed "
f"set {allowed} (SS 6)",
)
)
return findings
def rule_denominator_identity(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
counts = _mapping(ctx.payload.get("denominators"))
values = {key: _whole(counts.get(key)) for key in ("considered", "withheld", "delivered")}
missing = sorted(key for key, value in values.items() if value is None)
if missing:
return [
Finding(
"denominator_identity",
f"the payload reports no whole-number {', '.join(missing)}; a count "
"that is not reported is unmeasured, not zero (SS 5.1)",
)
]
considered = values["considered"]
withheld = values["withheld"]
delivered = values["delivered"]
assert considered is not None and withheld is not None and delivered is not None
if considered != withheld + delivered:
return [
Finding(
"denominator_identity",
f"considered ({considered}) != withheld ({withheld}) + delivered "
f"({delivered}); a count that does not close is not a denominator "
"(SS 5.2)",
)
]
return []
def rule_denominator_lists(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
counts = _mapping(ctx.payload.get("denominators"))
findings = []
for key, member in (("delivered", "excerpts"), ("withheld", "withheld")):
declared = _whole(counts.get(key))
if declared is None:
continue
actual = len(_sequence(ctx.payload.get(member)))
if declared != actual:
findings.append(
Finding(
"denominator_list_mismatch",
f"denominators.{key} is {declared} but {member} holds {actual}; "
"the count and the list are two statements of one fact (SS 8.1)",
)
)
return findings
def rule_withheld_rules(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
findings = []
for position, raw in enumerate(_sequence(ctx.payload.get("withheld"))):
entry = _mapping(raw)
for key in ("concept_id", "rule"):
if not _text(entry.get(key)):
findings.append(
Finding(
"cut_undeclared",
f"withheld entry {position} carries no {key!r}; a visible "
"drop is worth more than a silent one (SS 5.3)",
)
)
return findings
def rule_budget_declared(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
budget = _mapping(ctx.payload.get("budget"))
findings = []
for key in ("unit", "instrument"):
if not _text(budget.get(key)):
findings.append(
Finding(
"budget_undeclared",
f"the budget names no {key!r}; a number without its instrument "
"and unit is not a measurement (SS 7.1)",
)
)
limit = _whole(budget.get("limit"))
if limit is None or limit <= 0:
findings.append(
Finding(
"budget_undeclared",
'"bounded" without a bound is a denominator failure in prose (SS 7.1)',
)
)
if _whole(budget.get("spent")) is None:
findings.append(Finding("budget_undeclared", "the budget reports no `spent` (SS 7.2)"))
return findings
def rule_budget_gate(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
budget = _mapping(ctx.payload.get("budget"))
limit = _whole(budget.get("limit"))
spent = _whole(budget.get("spent"))
if limit is None or spent is None or spent <= limit:
return []
return [
Finding(
"budget_exceeded",
f"spent ({spent}) exceeds limit ({limit}); the cut strategy is wrong "
"for this bundle, which is a finding requiring a decision and never "
"a retry with a narrower question (SS 7.3)",
)
]
def rule_instrument_validated(ctx: Context) -> list[Finding]:
if not ctx.payload_is_mapping:
return []
known = _mapping(_mapping(ctx.payload.get("budget")).get("known_positive"))
expected = _whole(known.get("expected"))
measured = _whole(known.get("measured"))
if expected is None or measured is None or expected <= 0 or not _text(known.get("case")):
return [
Finding(
"instrument_unvalidated",
"the budget carries no usable `known_positive` (case, expected, "
"measured); an instrument that has not reproduced a known figure "
"has not been shown to count (SS 7.4)",
)
]
if expected != measured:
return [
Finding(
"instrument_unvalidated",
f"the known-positive expected {expected} and the instrument "
f"measured {measured} (SS 7.4)",
)
]
return []
# --- Skill rules -------------------------------------------------------------
def rule_skill_sections(ctx: Context) -> list[Finding]:
return [
Finding(
"skill_section_missing",
f"the skill carries no `## {section}` section (SS 8 of this checker's "
"reading; the headings are fixed so they can be checked by name)",
)
for section in REQUIRED_SECTIONS
if f"## {section}" not in ctx.skill
]
def rule_skill_markings(ctx: Context) -> list[Finding]:
return [
Finding(
"skill_marking_missing",
f"the skill does not carry the required marking {marking!r} verbatim "
"(SS 4.1); one literal string, no variants, no translations",
)
for marking in REQUIRED_MARKINGS
if marking not in ctx.skill
]
def rule_skill_states(ctx: Context) -> list[Finding]:
return [
Finding(
"skill_state_missing",
f"the skill does not name the state {state!r}, so its consumer cannot "
"be held to the closed set (SS 6)",
)
for state in (*ADJUDICATION_STATES, *TRUST_TIERS)
if f"`{state}`" not in ctx.skill
]
#: Every rule, in report order. `len(RULES)` is the denominator the report
#: quotes: "how many rules ran" is the number that makes "0 findings" mean
#: something.
RULES: tuple[Callable[[Context], list[Finding]], ...] = (
rule_payload_shape,
rule_contract_version,
rule_bundle_ref,
rule_excerpt_source_marking,
rule_excerpt_named,
rule_excerpt_states,
rule_denominator_identity,
rule_denominator_lists,
rule_withheld_rules,
rule_budget_declared,
rule_budget_gate,
rule_instrument_validated,
rule_skill_sections,
rule_skill_markings,
rule_skill_states,
)
def check(skill_text: str, payload: object) -> Report:
"""Run every rule. No rule short-circuits another: a caller fixing one
defect should not discover a second only on the next run."""
is_mapping = isinstance(payload, Mapping)
ctx = Context(
skill=skill_text,
payload=payload if isinstance(payload, Mapping) else {},
payload_is_mapping=is_mapping,
)
findings: list[Finding] = []
for rule in RULES:
findings.extend(rule(ctx))
return Report(
findings=tuple(findings),
rules_evaluated=len(RULES),
excerpts_examined=len(_sequence(ctx.payload.get("excerpts"))),
withheld_examined=len(_sequence(ctx.payload.get("withheld"))),
)
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument("--skill", type=Path, required=True, help="the SKILL.md to check")
parser.add_argument("--payload", type=Path, required=True, help="one pre-pass payload (JSON)")
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
try:
skill_text = args.skill.read_text(encoding="utf-8")
except OSError as exc:
print(f"could not read the skill: {exc}")
return 2
try:
payload = json.loads(args.payload.read_text(encoding="utf-8"))
except OSError as exc:
print(f"could not read the payload: {exc}")
return 2
except json.JSONDecodeError as exc:
print(f"the payload is not readable JSON: {exc}")
return 2
report = check(skill_text, payload)
print(report.render())
return 1 if report.findings else 0
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
from llm_ingestion_okf import contract_check as _impl # noqa: E402
if __name__ == "__main__":
raise SystemExit(main())
raise SystemExit(_impl.main())
# ALIASED, not re-exported. `import okf_contract_check` must hand back the packaged module
# ITSELF: a re-export binds copies of the names into a second module object, so
# a caller patching one of them patches a binding the implementation never
# reads. Measured on the move: two tests that monkeypatch `okf_contract_check` went green
# again only under the alias. Guarded by the `__main__` branch above, because
# aliasing `sys.modules["__main__"]` would replace the running script.
sys.modules[__name__] = _impl

598
tools/okf_skill.py
View file

@ -1,589 +1,33 @@
"""Instantiate the consumption skill template for ONE named OKF bundle.
#!/usr/bin/env python3
"""Thin entry point. The implementation is `llm_ingestion_okf.skill`.
`skills/okf-consume-template/SKILL.md` is a template whose own rule is that a
copy leaving a `<PLACEHOLDER>` unfilled "is not configured, it is unfinished".
Filling it by hand is what produced `skills/okf-consume/` for one corpus. This
command does the same thing for any bundle, from values it measures rather than
values someone remembered.
It moved into the package on 2026-09-08 (O5). The objection that kept it out --
a wheel-installed `okf skill` would emit a command pointing at a file the wheel
does not carry -- was about what the GENERATED skill names, and it was answered
by changing that: the emitted commands are `okf consume` and `okf check`, names
on PATH, and both moved into the package in the same step.
**Why a generator rather than one generic skill.** Measured 2026-09-08: the
contract checker passes the UNFILLED template against a real payload (exit 0, 15
rules, 0 findings), and passes a skill built for a different bundle against this
one's payload. So the checker cannot tell the two forms apart, and the choice
could not be made on conformance. It was made on what the skill has to state:
§ 5's denominators, § 7.6's breaking point and § 6.4's conditional-field list
are all per-bundle numbers. A generic skill can either leave them as holes -- the
template's own definition of unfinished -- or carry another corpus's numbers,
which is worse, because a stated cost that is false for this bundle is a
measurement failure and not merely a gap. Instantiating is what makes them true.
And with several bundles connected at once, a generic skill has nothing to
select on: each generated skill carries the bundle's id in its own name.
**Zero model calls, zero network, no clock.** The same bundle bytes produce the
same skill bytes. It lives outside `src/`, so it never enters a wheel and no
consumer's install surface changes because it exists -- and a wheel-installed
`okf skill` would emit a command pointing at `tools/okf_consume.py`, which the
wheel does not contain.
Exit codes are three, as elsewhere in this chain: 0 the skill was written, 1 the
run happened and refused, 2 the run did not happen.
This file stays because `docs/2026-09-07-okf-konsumskill-maaling.md` names it,
and a measurement whose command no longer runs is a measurement nobody can
repeat.
"""
from __future__ import annotations
import argparse
import json
import re
import sys
from collections import Counter
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
import okf_consume # noqa: E402
from llm_ingestion_okf.profiles import BundleProfile # noqa: E402
PROJECT_ROOT = Path(__file__).resolve().parents[1]
TEMPLATE = PROJECT_ROOT / "skills" / "okf-consume-template" / "SKILL.md"
CONTRACT = PROJECT_ROOT / "docs" / "consumption-contract.md"
PRE_PASS = PROJECT_ROOT / "tools" / "okf_consume.py"
CHECKER = PROJECT_ROOT / "tools" / "okf_contract_check.py"
#: The profile the pre-pass reads a bundle under, spelled so the generated skill
#: can name it in § 9.2's sentence. The pre-pass's own default; a bundle built
#: under another profile needs a copy of this tool that says so.
PROFILE_NAME = "SEGMENTED_OKF_V0_2"
#: The conditional frontmatter keys the generated skill reports a denominator
#: for. Every one of them is written by SOME producer and not by others, which
#: is exactly what § 6.4 says a consumer must be told about rather than left to
#: infer from an absence.
CONDITIONAL_FIELDS = (
"adjudication",
"bundle_id",
"verified",
"req_number",
"sources",
)
#: Tokens too short to carry a question. The same floor the pre-pass's own
#: matcher uses, so the derived example question cannot be shorter than what the
#: ranker can see.
MIN_QUESTION_TOKEN = 5
class SkillError(Exception):
"""The generator refused. Carries the code, like the rest of this chain."""
def __init__(self, message: str, *, code: str) -> None:
super().__init__(message)
self.code = code
# --- The blocks the template hands over verbatim ------------------------------
#: Every template block this generator rewrites WHOLE, by exact string. Held to
#: the template by a test: an edit that moves one of these would otherwise
#: produce a skill silently missing that rewrite, which is the drift the
#: instantiated copy exists to avoid.
TEMPLATE_HEADER = """**This file is a template.** Every `<PLACEHOLDER>` is a hole a per-corpus copy
fills; none of them has a default, and a copy that leaves one unfilled is not
configured, it is unfinished. The section headings are fixed:
`tools/okf_contract_check.py` reads them, and a missing one makes the skill
non-conformant rather than merely thin."""
TEMPLATE_PRE_PASS = """```sh
<PRE_PASS_COMMAND> --bundle-root <BUNDLE_ROOT> --ref <REF> --out <PAYLOAD_PATH>
```"""
TEMPLATE_CHECK = """```sh
python3 tools/okf_contract_check.py --skill <SKILL_PATH> --payload <PAYLOAD_PATH>
```"""
TEMPLATE_CONTRACT_LINE = (
"The contract this skill is held to is `docs/consumption-contract.md`. Where this"
)
TEMPLATE_EXTENSIONS = """**Extensions, if this corpus needs any.** `<EXTENSION_MARKINGS: for each, the
literal, what it means here, and which of the five it would otherwise collapse
into. Write "none" if there are none.>`"""
TEMPLATE_CONDITIONAL = """**Conditionally-written fields in this corpus.** `<CONDITIONAL_FIELDS: each
field this profile writes only when a build-time condition held, and what its
absence does and does not mean. Absence is a measurement, not a fact.>`"""
TEMPLATE_SCALING = """**Scaling.** `<COST_SCALING: whether cost tracks the question or the corpus, what
the whole bundle at this ref costs by the same instrument, and the corpus size
at which this strategy stops fitting the budget.>`"""
TEMPLATE_DENOMINATORS = """The payload reports three counts — `considered`, `withheld`, `delivered` — and
`considered == withheld + delivered`. Carry them into your output."""
TEMPLATE_ENUMERATION = (
"- **No directory enumeration** unless `<PROFILE_NAME>` says the index is derived."
)
TEMPLATE_OUTPUT = "Write to `<OUT>`. It must carry: the bundle ref; the findings, each with a"
REPLACED_BLOCKS = (
TEMPLATE_HEADER,
TEMPLATE_PRE_PASS,
TEMPLATE_CHECK,
TEMPLATE_CONTRACT_LINE,
TEMPLATE_EXTENSIONS,
TEMPLATE_CONDITIONAL,
TEMPLATE_SCALING,
TEMPLATE_DENOMINATORS,
TEMPLATE_ENUMERATION,
TEMPLATE_OUTPUT,
)
# --- What the generator measures ----------------------------------------------
def slug(value: str) -> str:
"""A Claude Code skill name from a bundle id: lowercase, hyphen-joined."""
reduced = re.sub(r"[^a-z0-9]+", "-", value.lower()).strip("-")
return reduced or "okf"
def example_question(titles: list[str]) -> str:
"""A question this bundle really answers, derived rather than invented.
The most frequent long token across the concepts' own titles, byte-sorted on
a tie. Derived because the shipped payload has to be one this bundle
produces: a question sharing no token with any concept is withheld under
`no_lexical_match` and the pre-pass refuses, so a hand-picked constant would
fail on the first bundle that does not happen to contain it.
"""
counts: Counter[str] = Counter()
for title in titles:
counts.update(
{token for token in okf_consume.normalise(title) if len(token) >= MIN_QUESTION_TOKEN}
)
if not counts:
# Every title is short or empty. Fall back to the longest title as it
# stands, which by construction matches at least its own concept.
longest = max(titles, key=lambda title: (len(title), title), default="")
if not longest:
raise SkillError(
"no concept in this bundle carries a title, so no example "
"question can be derived from it; pass --example-question",
code="no_example_question",
)
return longest
top = min(counts.items(), key=lambda item: (-item[1], item[0]))[0]
return f"Hva sier denne bundelen om {top}?"
def field_counts(concepts: list[okf_consume.Concept]) -> dict[str, int]:
"""How many concepts carry each conditional field. Set membership, never a
guess from two equal totals.
The `source_*` rows are DISCOVERED from the bundle rather than listed here,
for the same reason the excerpt carries them by prefix: a fixed list reports
a denominator for the producers someone thought of, and says nothing about
the locator this producer actually chose.
"""
discovered = sorted(
{
key
for concept in concepts
for key in concept.frontmatter
if key.startswith(okf_consume.SOURCE_KEY_PREFIX)
}
)
fields = (*CONDITIONAL_FIELDS, *discovered)
counts = {field: 0 for field in fields}
for concept in concepts:
for field in fields:
if concept.frontmatter.get(field, "").strip() or (
field == "sources" and concept.sources_present
):
counts[field] += 1
return counts
def whole_bundle_cost(concepts: list[okf_consume.Concept]) -> int:
"""What every concept in this bundle would cost by the gate's own
instrument, if a single answer delivered all of them."""
total = 0
for concept in concepts:
excerpt = okf_consume.excerpt_for(concept)
if excerpt is not None:
total += okf_consume.excerpt_weight(excerpt)
return total
def directory_control(bundle_root: Path, *, profile: BundleProfile) -> tuple[int, int]:
"""The index walk against the method § 9.2 forbids the CONSUMER from using.
Run HERE, once, at generation time -- never on the question path. § 9.2
binds a consumer reaching for context at query time; a build-time control is
what turns "the walk loses nothing" from an assumption into a number the
generated skill can quote.
"""
walked = len(okf_consume.enumerate_concepts(bundle_root, profile=profile))
suffix = profile.paths.concept_suffix
reserved = {profile.index.name, "log.md"}
on_disk = len([path for path in bundle_root.rglob(f"*{suffix}") if path.name not in reserved])
return walked, on_disk
# --- The instantiation --------------------------------------------------------
def render(
bundle_root: Path,
*,
out: Path,
profile: BundleProfile = okf_consume.DEFAULT_PROFILE,
question: str | None = None,
) -> tuple[str, dict[str, object]]:
"""The skill text and the example payload that proves it, for one bundle."""
bundle_root = bundle_root.resolve()
bundle_id = okf_consume.root_bundle_id_of(bundle_root, profile=profile)
ref = okf_consume.bundle_ref(bundle_root, profile=profile)
concept_ids = okf_consume.enumerate_concepts(bundle_root, profile=profile)
concepts = [
okf_consume.read_concept(
bundle_root / f"{concept_id}{profile.paths.concept_suffix}",
bundle_root=bundle_root,
root_bundle_id=bundle_id,
)
for concept_id in concept_ids
]
if not concepts:
raise SkillError(
f"{bundle_root} has an index but no concept under it; a skill for an "
"empty bundle would state denominators of zero it never measured",
code="bundle_empty",
)
total = len(concepts)
asked = question or example_question([concept.title for concept in concepts])
payload = okf_consume.build_payload(bundle_root, question=asked, profile=profile)
counts = field_counts(concepts)
walked, on_disk = directory_control(bundle_root, profile=profile)
cost = whole_bundle_cost(concepts)
denominators = payload["denominators"]
assert isinstance(denominators, dict)
budget = payload["budget"]
assert isinstance(budget, dict)
withheld = payload["withheld"]
assert isinstance(withheld, list)
bookkeeping = okf_consume.measure(json.dumps(withheld, ensure_ascii=False))
per_withheld = bookkeeping / len(withheld) if withheld else 0.0
breaking = int(okf_consume.DEFAULT_LIMIT / per_withheld) if per_withheld else 0
name = f"{slug(bundle_id)}-consume"
text = TEMPLATE.read_text(encoding="utf-8")
text = text.split("---\n", 2)[2]
text = _rewrite(
text,
bundle_root=bundle_root,
skill_path=out.resolve() / "SKILL.md",
bundle_id=bundle_id,
ref=ref,
name=name,
total=total,
counts=counts,
walked=walked,
on_disk=on_disk,
cost=cost,
asked=asked,
spent=int(budget["spent"]),
delivered=int(denominators["delivered"]),
bookkeeping=bookkeeping,
breaking=breaking,
)
header = f"---\nname: {name}\ndescription: {_description(bundle_id, total, ref)}\n---\n"
return header + text, payload
def _description(bundle_id: str, total: int, ref: str) -> str:
return (
f"Answer one question about the OKF bundle `{bundle_id}` ({total} concepts, "
f"ref {ref}) from a bounded payload assembled by a deterministic pre-pass, "
"marking every claim with its source, its title and its provenance locator. "
"Use whenever a question is about what that bundle's documents require, say "
"or contain. Generated by tools/okf_skill.py; every value below is measured "
"against this bundle at this ref."
)
def _rewrite(
text: str,
*,
bundle_root: Path,
skill_path: Path,
bundle_id: str,
ref: str,
name: str,
total: int,
counts: dict[str, int],
walked: int,
on_disk: int,
cost: int,
asked: str,
spent: int,
delivered: int,
bookkeeping: int,
breaking: int,
) -> str:
replacements: list[tuple[str, str]] = [
(
TEMPLATE_HEADER,
"**This file is an instantiated copy of "
"`skills/okf-consume-template/SKILL.md`,** generated by "
f"`tools/okf_skill.py` for one bundle: `{bundle_id}` at ref\n"
f"`{ref}`. Every value below was measured against those bytes. If the\n"
"bundle moves, the ref moves with it and this file is stale — regenerate\n"
"it rather than editing a number here. The section headings are fixed:\n"
"the contract checker reads them by name.",
),
(
TEMPLATE_PRE_PASS,
"```sh\n"
f"python3 {PRE_PASS} \\\n"
f" {bundle_root} \\\n"
' --question "your question" \\\n'
f" --ref {ref} \\\n"
" --out /tmp/payload.json\n"
"```\n\n"
"`--ref` is an **assertion**, never an override: the identity is computed\n"
"from the bytes either way, and a mismatch refuses. Read the pre-pass's\n"
"own exit status, which carries three values: **0** a payload was written,\n"
"**1** the run happened and refused, **2** the run did not happen at all.",
),
(
TEMPLATE_CHECK,
"```sh\n"
f"python3 {CHECKER} \\\n"
f" --skill {skill_path} \\\n"
" --payload /tmp/payload.json\n"
"```",
),
(
TEMPLATE_CONTRACT_LINE,
f"The contract this skill is held to is `{CONTRACT}`. Where this",
),
(
TEMPLATE_EXTENSIONS,
"**Extensions, if this corpus needs any: none.** This generated skill adds\n"
"no marking to the required five. § 4.3 makes the undeclared extension the\n"
"defect, so the absence is stated rather than left to be inferred — and a\n"
"corpus that does need a sixth needs a hand-edited copy that declares it.",
),
(
TEMPLATE_CONDITIONAL,
_conditional_table(total, counts),
),
(
TEMPLATE_SCALING,
_scaling(
total=total,
cost=cost,
asked=asked,
spent=spent,
delivered=delivered,
bookkeeping=bookkeeping,
breaking=breaking,
),
),
(
TEMPLATE_DENOMINATORS,
_denominators(total, asked=asked, delivered=delivered),
),
(
TEMPLATE_ENUMERATION,
_enumeration(walked, on_disk),
),
(
TEMPLATE_OUTPUT,
"Write to the path the caller names, or to your answer if none was named.\n"
"It must carry: the bundle ref; the findings, each with a",
),
("`<CORPUS>` bundle", f"`{bundle_id}` bundle"),
("# <CORPUS> consumption", f"# {bundle_id} consumption"),
("<BUDGET_LIMIT>", str(okf_consume.DEFAULT_LIMIT)),
("<BUDGET_UNIT>", okf_consume.BUDGET_UNIT),
("<BUDGET_INSTRUMENT>", okf_consume.BUDGET_INSTRUMENT),
("<KNOWN_POSITIVE_CASE>", okf_consume.KNOWN_POSITIVE_CASE),
("<KNOWN_POSITIVE_EXPECTED>", str(okf_consume.KNOWN_POSITIVE_EXPECTED)),
]
for old, new in replacements:
if old not in text:
raise SkillError(
f"the template no longer carries the block this generator rewrites: {old[:70]!r}",
code="template_drift",
)
text = text.replace(old, new)
assert name # kept in the signature so a caller cannot forget to name the skill
return text
def _conditional_table(total: int, counts: dict[str, int]) -> str:
rows = "\n".join(
f"| `{field}` | **{count} of {total}** | "
f"{'the producer wrote none for that concept' if count else 'no concept in this bundle carries it'} | "
"that the source document lacks what the field asserts |"
for field, count in counts.items()
)
return (
"**Conditionally-written fields in this bundle, with what each absence does\n"
"and does not mean.** Every count is over the same denominator — "
f"**{total} concepts**, the set the index walk reaches. § 6.4: absence is a\n"
"measurement about the producer, never a fact about the source.\n\n"
"| Field | Present on | Absence means | Absence does NOT mean |\n"
"|---|---|---|---|\n"
f"{rows}\n\n"
"A field present on **0 of "
f"{total}** is a measured zero, not an unmeasured one: the count was taken\n"
"over every concept, and it is reported so a negative claim resting on it\n"
"carries its denominator."
)
def _scaling(
*,
total: int,
cost: int,
asked: str,
spent: int,
delivered: int,
bookkeeping: int,
breaking: int,
) -> str:
share = (spent / cost * 100) if cost else 0.0
return (
"**Scaling. Cost tracks the question, not the corpus.** Measured on this\n"
f"bundle at generation time, with the question `{asked}`: the delivered set\n"
f"was **{delivered} excerpts** costing **{spent} {okf_consume.BUDGET_UNIT}**,\n"
f"against a whole bundle that would cost **{cost}** by the same instrument if\n"
f"one answer delivered all {total} concepts — so that answer was about\n"
f"**{share:.1f} %** of the corpus. One question is one measurement: a\n"
"different question moves `spent` and this figure with it.\n\n"
"**The breaking point, stated so it can be observed to have been passed.**\n"
"The `withheld` list carries one entry per considered concept and grows\n"
f"linearly: here it is **{bookkeeping} bytes** for {total} concepts. At roughly\n"
f"**{breaking} concepts** the bookkeeping alone reaches the "
f"{okf_consume.DEFAULT_LIMIT}-byte\n"
"limit, and although it is not counted against `spent`, a payload whose\n"
"bookkeeping dwarfs its content has stopped being a cut. The pre-pass also\n"
"reads every concept body on every run, so the same growth is a wall-clock\n"
"cost with no precomputed index behind it."
)
def _denominators(total: int, *, asked: str, delivered: int) -> str:
return (
"The payload reports three counts — `considered`, `withheld`, `delivered` — and\n"
"`considered == withheld + delivered`. Carry them into your output.\n\n"
f"For this bundle `considered` is **{total}**, every concept the index walk\n"
"reaches, never the post-ranking shortlist. A concept dropped at the ranking\n"
"stage is `withheld` **with its rule**, not invisible, and the rules are a\n"
"closed set of six: `verdict_layer_excluded` (a verdict-layer file, § 9.1),\n"
"`verified_unreadable` (a `verified` value this reader cannot decode, so no\n"
"tier can be derived), `no_lexical_match` (the concept shares no token with\n"
"the question), `over_budget_alone` (one excerpt exceeds the whole limit),\n"
"`below_k` (ranked outside the shortlist the cut considers) and\n"
"`over_budget_after_knapsack` (it ranked inside the shortlist and the pack\n"
"had no room). Naming the rule is what makes a drop visible.\n\n"
"**One limitation to carry into every negative claim.** `no_lexical_match` is\n"
'a per-concept relevance drop, not a whole-question "this bundle has no\n'
f'answer" gate: on the generation question `{asked}` it still returned\n'
f"{delivered} excerpts. **An empty `excerpts` list is evidence of absence; a\n"
"full one is not evidence of presence.** When the delivered excerpts do not\n"
"actually answer the question, say `[sourced-not-sufficient]` and report that\n"
"the cut found nothing responsive."
)
def _enumeration(walked: int, on_disk: int) -> str:
agreement = (
f"which costs nothing here: the walk reaches **{walked}** concepts and a\n"
f" directory walk finds **{on_disk}**"
if walked == on_disk
else f"and the two disagree — the walk reaches **{walked}** concepts where a\n"
f" directory walk finds **{on_disk}**, so some concept is unreachable through\n"
" the index and the bundle's producer should be told"
)
return (
"- **No directory enumeration.** This bundle is read under the\n"
f" `{PROFILE_NAME}` profile, whose index policy declares\n"
" `entries_match_directory = False`, so § 9.2's permission does not apply.\n"
f" The pre-pass walks the **index tree** instead, {agreement}\n"
" (controlled once at generation time, never on the question path). Do not\n"
" enumerate a directory yourself either."
)
def generate(
bundle_root: Path,
*,
out: Path,
profile: BundleProfile = okf_consume.DEFAULT_PROFILE,
question: str | None = None,
force: bool = False,
) -> Path:
"""Write `out/SKILL.md` and its reference payload. Returns the skill path."""
target = out / "SKILL.md"
if target.exists() and not force:
raise SkillError(
f"{target} already exists; pass --force to replace it. A silent "
"overwrite would destroy a hand-edited copy whose extra measurements "
"this generator cannot reproduce",
code="target_occupied",
)
text, payload = render(bundle_root, out=out, profile=profile, question=question)
(out / "references").mkdir(parents=True, exist_ok=True)
target.write_text(text, encoding="utf-8")
(out / "references" / "example-payload.json").write_text(
okf_consume.serialise(payload), encoding="utf-8"
)
return target
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument("bundle", type=Path, help="the OKF bundle to instantiate a skill for")
parser.add_argument(
"--out", type=Path, required=True, help="the skill directory to write (SKILL.md inside)"
)
parser.add_argument(
"--example-question",
default=None,
help="the question the shipped reference payload answers. Derived from the "
"bundle's own titles when omitted",
)
parser.add_argument(
"--force", action="store_true", help="replace an existing SKILL.md at --out"
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
try:
written = generate(
args.bundle, out=args.out, question=args.example_question, force=args.force
)
except okf_consume.ConsumeError as exc:
print(f"refused ({exc.code}): {exc}")
return 1
except SkillError as exc:
print(f"refused ({exc.code}): {exc}")
return 1
except OSError as exc:
print(f"the run did not happen: {exc}")
return 2
print(f"wrote {written}")
return 0
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
from llm_ingestion_okf import skill as _impl # noqa: E402
if __name__ == "__main__":
raise SystemExit(main())
raise SystemExit(_impl.main())
# ALIASED, not re-exported. `import okf_skill` must hand back the packaged module
# ITSELF: a re-export binds copies of the names into a second module object, so
# a caller patching one of them patches a binding the implementation never
# reads. Measured on the move: two tests that monkeypatch `okf_skill` went green
# again only under the alias. Guarded by the `__main__` branch above, because
# aliasing `sys.modules["__main__"]` would replace the running script.
sys.modules[__name__] = _impl