open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(readme,skill,cli): the first screen an agent reads, three modes, and one flag that made two builds

Browse source 
`okf project` built a bundle two rules behind `okf build`. `cli.build`'s
signature defaulted `keep_table_heading` and `sheet_section_rows` to `False`
while argparse defaulted both to `True`, and `project.create` calls `build()`
as a function. Measured on a five-document folder: 15 concepts / 30 files
where `okf build` wrote 26 / 52, the whole difference in the priced sheet --
the document a question about price has to reach. The invariant test could not
see it: it compared `project.create` against the same function, and its two
fixture documents had neither a table nor a sheet. Both gaps are tests now,
and the two paths are byte-equal on that folder (`diff -rq`, 0 differences).

README opens with what / one install line / two commands / the three shapes of
request; the phase-status paragraph moved down, nothing deleted. One tag is
pinned everywhere: README pinned v0.4.0 on its install lines and v0.6.0 below,
llms.txt pinned v0.4.0, so an agent reading from the top installed a tag
without `okf project`.

The skill states three modes -- question, hypothesis (per premise, `confirmed`
/ `refuted` / `undecidable-from-bundle`), and a task producing a document
(source per claim in the artefact, an ungrounded paragraph written and marked
rather than dropped, the cut declared inside the document). The five markings
are untouched.

Generated skills state relative paths in the project layout: `okf consume
.okf/<id>` and `okf check --skill .claude/skills/<id>-consume/SKILL.md`,
runnable from where `okf project` tells the reader to start `claude`. Two
absolute paths to zero, measured with a query shown capable of finding first --
O5's published "4 -> 0" used `grep -c "^/"` against paths indented by two
spaces.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-09 18:12:02 +02:00
commit 1260fac154
12 changed files with 532 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

98
README.md
View file

@ -1,6 +1,41 @@
# llm-ingestion-okf
Shared OKF (Open Knowledge Format) ingestion library: spec-based connectors, bundle inbox, and external-bundle import. Security delegated to llm-ingestion-guard.
Turn a folder of documents (PDF, DOCX, XLSX, PPTX, MD) into a bundle a model can
answer from **with a source on every claim** — offline, deterministic, no model
call anywhere in the run path.
## Install
Python 3.10+ and [uv](https://docs.astral.sh/uv/). One line:
```sh
uv tool install "llm-ingestion-okf[extract] @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.7.0"
```
## Use it
```sh
okf project ~/my-documents # folder in: bundle + Claude Code skill, in this directory
claude # start Claude Code here
```
Then ask in plain language. Three shapes of request work, and the skill states
the rules for each:
- **a question** — "hva er kravene til pris?"
- **a hypothesis** — "stemmer det at leverandoeren baerer risikoen for grunnforhold?"
Answered per premise as `confirmed` / `refuted` / `undecidable-from-bundle`.
- **a task whose answer is a document** — "lag `krav-pris.md` med alle krav til
pris, ett avsnitt per krav, med dokument og kravnummer." Every claim in the
written file carries its source; a paragraph with no ground is written
and marked, never dropped.
Everything below is detail: [Consume in Claude
Code](#consume-in-claude-code) for the same thing in steps and with several
bundles at once, [Build](#build) for the flags, [Requirements](#requirements)
for the pip fallback and the guard pairing.
## What this library is
Status: phases 1–3 are implemented. Phase 1 (spec-based ingestion) covers
manifest validation, the `file`/`sql`/`http` connectors, deterministic
@ -17,46 +52,53 @@ formats through a vendored document converter. Three of those five office
rows are **unmeasured** — see [Binary extraction](#binary-extraction). Phase 4
(the Node half) is planned (see `docs/plan/`).
## Install
## Install in detail
Python 3.10+. Neither this package nor the guard it depends on is on a package
index yet. With uv, one command is enough:
Neither this package nor the guard it depends on is on a package index yet, so
both install by direct reference. With uv, one command resolves both:
```
uv pip install "llm-ingestion-okf @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.4.0"
```sh
uv pip install "llm-ingestion-okf @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.7.0"
```
uv resolves the guard on its own, because it reads the `[tool.uv.sources]`
entry in the `pyproject.toml` **of the tag it is installing**, and `v0.4.0`
points that entry at the guard tag below. Measured 2026-07-25 and re-measured
2026-08-20 with an empty `uv` cache; both runs installed
`llm-ingestion-guard==0.2.0` + `llm-ingestion-okf==0.4.0` and imported clean.
entry in the `pyproject.toml` **of the tag it is installing**, and `v0.7.0`
points that entry at `llm-ingestion-guard` `v1.3.0`. Use `uv tool install`
instead of `uv pip install` when you want the `okf` command on `PATH` without an
active virtualenv — that is the form the first screen shows.
With plain pip, the transitive git dependency does not resolve on its own —
**install the guard first**, or installing this package fails with
`No matching distribution found for llm-ingestion-guard`:
```
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v0.2.0"
pip install "llm-ingestion-okf @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.4.0"
```sh
pip install "llm-ingestion-guard @ git+https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git@v1.3.0"
pip install "llm-ingestion-okf @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.7.0"
```
The guard tag is paired to the okf tag, not to this branch: `v0.4.0` declares
`llm-ingestion-guard>=0.2,<0.3`, which `v0.2.0` satisfies and later guard tags
do not. `main` has since moved its own pin to `>=1.2,<2.0` (see
[Requirements](#requirements)); that pin reaches you in the next stable tag,
not in the commands above. Reading a pin off this branch and installing it
against `v0.4.0` is the one combination that fails.
The guard tag is paired to the okf tag, not to this branch. `v0.7.0` declares
`llm-ingestion-guard>=1.2,<2.0`, which `v1.3.0` satisfies; the pairing above is
read off that tag's own `[tool.uv.sources]`, not off this branch. Reading a pin
off `main` and installing it against an older okf tag is the one combination
that fails.
`v0.6.0` is the current tag and the one the three-line form under [Consume in
Claude Code](#consume-in-claude-code) installs: it is the first tag carrying the
`okf project`, `okf consume`, `okf check` and `okf skill` subcommands, without
which that form does not exist. `v0.4.0` is the last tag before the OKF v0.2
work. `v0.5.0a2` is a pre-release for the named OKF v0.2 pilot set only.
### Earlier tags, as history
**`v0.6.0` does not make OKF v0.2 generally available.** `OKF_LATEST` is
unchanged and still points at `DEFAULT`; flipping that alias is the GA event and
this tag is not it (see [Upstream OKF versions](#upstream-okf-versions)).
These are not install lines. They record what each earlier tag was, so a reader
who meets one in an older document knows what they are looking at.
- `v0.7.0` — the current tag: `okf project` builds the bundle `okf build`
builds (they were one flag apart before it), and the generated skill states
the question, hypothesis and document-task modes with relative paths.
- `v0.6.0` — the first tag carrying the `okf project`, `okf consume`,
`okf check` and `okf skill` subcommands.
- `v0.5.0a2` — a pre-release for the named OKF v0.2 pilot set only.
- `v0.4.0` — the last tag before the OKF v0.2 work; it declares
`llm-ingestion-guard>=0.2,<0.3`, which only guard `v0.2.0` satisfies.
**No tag yet makes OKF v0.2 generally available.** `OKF_LATEST` is unchanged and
still points at `DEFAULT`; flipping that alias is the GA event and none of the
tags above is it (see [Upstream OKF versions](#upstream-okf-versions)).
## Build
@ -317,7 +359,7 @@ not need this repository — the first line installs the command, the second
builds the bundle and writes a skill beside it, the third asks.
```sh
uv tool install "llm-ingestion-okf[extract] @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.6.0"
uv tool install "llm-ingestion-okf[extract] @ git+https://git.fromaitochitta.com/open/llm-ingestion-okf.git@v0.7.0"
okf project ~/my-documents
claude
```