open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(cli): okf project/consume/check/skill, and a generated skill with no path into a checkout

Browse source 
The reading direction existed only for someone standing in a clone. `consume`,
`contract_check` and `skill` moved from `tools/` into the package and are
reachable as `okf consume`, `okf check` and `okf skill`; `okf project` is new
and does the whole thing in one command.

The red measurement: a consumption skill generated from a checkout carried 4
lines naming that checkout by absolute path, 2 of them the commands the skill
tells a reader to run. It now names `okf consume` and `okf check`, and a test
asserts this repository appears in it nowhere, with a known-positive so the
zero is a measurement rather than a search that could not find.

The `tools/` files stay as ALIASES, not re-exports: a re-export binds copies of
the names into a second module object, so a caller patching one patches a
binding the implementation never reads. Two tests that monkeypatch okf_consume
went green again only under the alias. Every published reproduction block runs
unchanged.

The template and docs/consumption-contract.md (the section 7.4 known-positive)
are force-included into the wheel from the file they are authored in, so both
travel with the commands that cannot run without them and there is still one
authored copy of each.

Step 0, before any of it: okf build's default gained Arm E (--table-grid),
with --no-table-grid as its opt-out. The default moved to D plus F earlier the
same day on Arm F's published 5 of 12 -- a figure measured with Arm E ON.
Without it the fold has no joined table to fold, and the shipped default scored
2 of 12 with docx 0 of 3. Measured on the operator's folder: 30 md / 15
concepts on the new default against 43 / 28 without Arm E.

Install measurement from a fresh uv tool install, empty folder, this repository
nowhere on PYTHONPATH: 5 documents in, 15 concepts out, 0 references to tools/
in the generated skill, okf check conformant (15 rules, 0 findings).

Deviation stated rather than hidden: the order asked that
tests/test_okf_consume.py be left untouched. Two assertions in it read a PATH,
which is the one thing this work changes. Both were moved and the second made
stronger -- it now asserts every command the README recipe names is a
subcommand the CLI registers, which a file existing on disk never proved.

Suite 1414 -> 1427. ruff clean, mypy --strict clean over 21 files.
Record: docs/2026-09-08-o5-okf-project.md

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 21:47:38 +02:00
commit f6fea13299
21 changed files with 4017 additions and 2833 deletions
Download patch file Download diff file Expand all files Collapse all files

112
README.md
View file

@ -48,9 +48,15 @@ do not. `main` has since moved its own pin to `>=1.2,<2.0` (see
not in the commands above. Reading a pin off this branch and installing it
against `v0.4.0` is the one combination that fails.
`v0.4.0` is the current stable tag. `v0.5.0a2` is a pre-release for the named
OKF v0.2 pilot set only; pin it only if you are one of them (see
[Upstream OKF versions](#upstream-okf-versions)).
`v0.6.0` is the current tag and the one the three-line form under [Consume in
Claude Code](#consume-in-claude-code) installs: it is the first tag carrying the
`okf project`, `okf consume`, `okf check` and `okf skill` subcommands, without
which that form does not exist. `v0.4.0` is the last tag before the OKF v0.2
work. `v0.5.0a2` is a pre-release for the named OKF v0.2 pilot set only.
**`v0.6.0` does not make OKF v0.2 generally available.** `OKF_LATEST` is
unchanged and still points at `DEFAULT`; flipping that alias is the GA event and
this tag is not it (see [Upstream OKF versions](#upstream-okf-versions)).
## Build
@ -235,65 +241,71 @@ hit@8 was **5 of 6** questions at rank 1 against a chance baseline of **1.35 of
## Consume in Claude Code
A folder of documents to an answer a model can cite, in three commands. Every
command below was run end to end on 2026-09-08 against a nine-document folder
and a 270-concept third-party bundle; nothing here is untested.
A folder of documents to an answer a model can cite, in **three lines**. You do
not need this repository — the first line installs the command, the second
builds the bundle and writes a skill beside it, the third asks.
```sh
SRC=/tmp/c1-fresh-src # the folder of documents
BUNDLE=/tmp/c1-fresh-bundle # where the OKF bundle goes
PROJECT=/tmp/c1-scratch # the project you will ask the question from
uv tool install "llm-ingestion-okf[extract] @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.6.0"
okf project ~/my-documents
claude
```
**1. Build the bundle.**
`okf project` writes the bundle to `.okf/<id>/` and a skill to
`.claude/skills/<id>-consume/` in the **current directory**, then prints what it
read, what it wrote, and which documents a question cannot reach. Start `claude`
in that directory and ask in plain language; the generated skill runs the
pre-pass and the contract check itself and marks every claim with its source.
`<id>` is the folder's name reduced to `[a-z0-9-]`. Run it once per folder with
`--id <name>` to have several bundles reachable at once — each skill carries its
own `bundle_id`, which is what lets a model pick between them. `--out <dir>`
puts the project somewhere other than the current directory.
Measured 2026-09-08 from a fresh `uv tool install` with this repository nowhere
on the path: 5 documents in, 15 concepts out, a skill carrying **0** paths into
any checkout, and `okf check` conformant on its own payload (15 rules, 0
findings). Before that day the same result took a `PYTHONPATH`, a snapshot of a
clone, and a generated skill that named that clone by absolute path on four
lines — so it could not be moved, shared, or run by anyone else.
### The same thing in steps, if you want to see the payload
```sh
okf build "$SRC" --bundle "$BUNDLE" --bundle-id c1-fresh-20260908 --okf-version 0.2 --ingested-at 2026-09-08T00:00:00Z
okf build ./documents --bundle ./bundle --bundle-id my-bundle --okf-version 0.2
okf skill ./bundle --out ./project/.claude/skills/my-bundle-consume
okf consume ./bundle --question "your question" --out /tmp/payload.json
okf check --skill ./project/.claude/skills/my-bundle-consume/SKILL.md --payload /tmp/payload.json
```
**2. Generate a skill for that bundle**, straight into the project's skill
directory. The skill is instantiated for these bytes: its id, ref, concept
count, per-field denominators, whole-bundle cost and breaking point are all
measured from the bundle, and it ships a reference payload the checker accepts.
A bundle you only have read access to is fine — the generator only reads it.
```sh
python3 tools/okf_skill.py "$BUNDLE" --out "$PROJECT/.claude/skills/c1-fresh-20260908-consume"
```
### The honest limits
Repeat for every bundle you want reachable; each one gets its own skill named
after its `bundle_id`, which is what lets a model pick between them. A bundle
you only have read access to is fine — the generator only reads it.
**3. Ask.** From `$PROJECT`, in Claude Code:
```sh
claude -p "Hvordan skal prisene fylles ut?"
```
Measured with two bundles installed side by side: the model selected the right
skill from the question alone, ran the pre-pass and the contract check itself,
quoted the requirement verbatim, and named the document, the requirement number,
the source resource and the locator inside it. Across three questions, **0**
numbers or identifiers appeared in an answer that were not in the delivered set
or in the payload's own identities. On a question the bundle does not cover it
answered `[sourced-not-sufficient]` and reported the denominator rather than
inventing an answer.
The pre-pass and the checker are the same two commands the skill runs for you,
if you want to see the payload first:
```sh
python3 tools/okf_consume.py "$BUNDLE" --question "your question" --out /tmp/payload.json
python3 tools/okf_contract_check.py --skill "$PROJECT/.claude/skills/c1-fresh-20260908-consume/SKILL.md" --payload /tmp/payload.json
```
The honest limits: this was measured on **four questions** across two bundles,
which is a demonstration and not a hit rate. The ranking is lexical, and one of
the four found a topic the bundle **does** cover and did not rank it into the
cut — the skill then said so with its denominator instead of answering, which is
the behaviour the contract asks for, but a miss is still a miss.
Measured on **four questions** across two bundles, which is a demonstration and
not a hit rate. The ranking is lexical, and one of the four found a topic the
bundle **does** cover and did not rank it into the cut — the skill then said so
with its denominator instead of answering, which is the behaviour the contract
asks for, but a miss is still a miss.
`docs/2026-09-08-claude-code-skill-vilkaarlig-bundle.md` has the runs.
Two more things the summary tells you and this paragraph will not repeat: a
document that landed **whole** (no heading, table or numbered outline to cut it
on) comes back as one excerpt, which the budget often refuses and which often
does not carry the answer at the place you asked about; and a document that is
in the folder but **not** in the bundle cannot be quoted at all. Both cases are
answered `[sourced-not-sufficient]`, and `okf project` names the documents.
### The skill that runs this for you
`skills/okf-prosjekt/` in this repository is a Claude Code skill (Norwegian)
that wraps the command above: it takes a folder, runs `okf project`, and reads
the summary back. Install it for your user account after cloning:
```sh
mkdir -p ~/.claude/skills && cp -R skills/okf-prosjekt ~/.claude/skills/
```
## Implemented scope (v1)
The library provides three entry points for getting content into an OKF