open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(cli): okf project/consume/check/skill, and a generated skill with no path into a checkout

Browse source 
The reading direction existed only for someone standing in a clone. `consume`,
`contract_check` and `skill` moved from `tools/` into the package and are
reachable as `okf consume`, `okf check` and `okf skill`; `okf project` is new
and does the whole thing in one command.

The red measurement: a consumption skill generated from a checkout carried 4
lines naming that checkout by absolute path, 2 of them the commands the skill
tells a reader to run. It now names `okf consume` and `okf check`, and a test
asserts this repository appears in it nowhere, with a known-positive so the
zero is a measurement rather than a search that could not find.

The `tools/` files stay as ALIASES, not re-exports: a re-export binds copies of
the names into a second module object, so a caller patching one patches a
binding the implementation never reads. Two tests that monkeypatch okf_consume
went green again only under the alias. Every published reproduction block runs
unchanged.

The template and docs/consumption-contract.md (the section 7.4 known-positive)
are force-included into the wheel from the file they are authored in, so both
travel with the commands that cannot run without them and there is still one
authored copy of each.

Step 0, before any of it: okf build's default gained Arm E (--table-grid),
with --no-table-grid as its opt-out. The default moved to D plus F earlier the
same day on Arm F's published 5 of 12 -- a figure measured with Arm E ON.
Without it the fold has no joined table to fold, and the shipped default scored
2 of 12 with docx 0 of 3. Measured on the operator's folder: 30 md / 15
concepts on the new default against 43 / 28 without Arm E.

Install measurement from a fresh uv tool install, empty folder, this repository
nowhere on PYTHONPATH: 5 documents in, 15 concepts out, 0 references to tools/
in the generated skill, okf check conformant (15 rules, 0 findings).

Deviation stated rather than hidden: the order asked that
tests/test_okf_consume.py be left untouched. Two assertions in it read a PATH,
which is the one thing this work changes. Both were moved and the second made
stronger -- it now asserts every command the README recipe names is a
subcommand the CLI registers, which a file existing on disk never proved.

Suite 1414 -> 1427. ruff clean, mypy --strict clean over 21 files.
Record: docs/2026-09-08-o5-okf-project.md

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 21:47:38 +02:00
commit f6fea13299
21 changed files with 4017 additions and 2833 deletions
Download patch file Download diff file Expand all files Collapse all files

91
tests/test_cli_build.py
View file

@ -763,6 +763,7 @@ def test_the_build_default_is_now_arm_d_plus_arm_f(tmp_path: Path) -> None:
PROPOSED_AT,
"--outline-run",
"3",
"--table-grid",
"--unit-fold",
)
== 0
@ -787,6 +788,7 @@ def test_each_arm_in_the_default_has_an_explicit_opt_out(tmp_path: Path) -> None
PROPOSED_AT,
"--outline-run",
"0",
"--no-table-grid",
"--no-unit-fold",
)
== 0
@ -802,6 +804,7 @@ def test_each_arm_in_the_default_has_an_explicit_opt_out(tmp_path: Path) -> None
PROPOSED_AT,
"--outline-run",
"0",
"--no-table-grid",
"--no-unit-fold",
)
== 0
@ -831,7 +834,9 @@ def test_a_bundle_built_with_no_flags_is_byte_identical_to_the_shipped_one(
"""
inbox = inbox_with_subdirectories(tmp_path)
armed = two_script_bundle(
inbox, tmp_path / "reference-armed", proposer_flags=("--outline-run", "3", "--unit-fold")
inbox,
tmp_path / "reference-armed",
proposer_flags=("--outline-run", "3", "--table-grid", "--unit-fold"),
)
bundle = tmp_path / "cli-bundle-2"
assert build(inbox, bundle, "--ingested-at", INGESTED_AT, "--proposed-at", PROPOSED_AT) == 0
@ -849,8 +854,92 @@ def test_a_bundle_built_with_no_flags_is_byte_identical_to_the_shipped_one(
PROPOSED_AT,
"--outline-run",
"0",
"--no-table-grid",
"--no-unit-fold",
)
== 0
)
assert tree(opted_out) == tree(plain)
# --- the default moved again: Arm E joined it (2026-09-08, round 4) --------
#
# Round 3 moved the default to Arm D plus Arm F and left `--table-grid` a flag.
# Measured afterwards on the operator's five-document folder, that combination
# is Arm F WITHOUT a joined table to fold: the fold's table clause folds a
# table back into the heading that introduces it, and with Arm E off a grid
# table is not one block but one block per rule line, so there is nothing whole
# to fold. The published "Arm F matches 5 of 12" was measured with
# `--table-grid` ON; the shipped default scored 2 of 12, and `docx` 0 of 3.
#
# Arm E therefore joins the default, with the same explicit opt-out every arm
# in it has. `--keep-table-heading` does NOT join: measured on two K2 bundles
# it buys 35 bytes and zero rank positions.
GRID_SHEET_DOCUMENT = {
"theta.md": (
"## 7 Romskjema\n\nInnledende avsnitt.\n\n"
"+-------+-------+\n| Navn | Verdi |\n+=======+=======+\n"
"| Areal | 120 |\n+-------+-------+\n| Hoyde | 3 |\n+-------+-------+\n"
)
}
def inbox_for_grid_sheet(root: Path) -> Path:
inbox = root / "grid-inbox"
inbox.mkdir(parents=True, exist_ok=True)
for name, body in GRID_SHEET_DOCUMENT.items():
(inbox / name).write_text(body, encoding="utf-8", newline="")
return inbox
def test_the_build_default_is_now_arm_d_plus_arm_e_plus_arm_f(tmp_path: Path) -> None:
"""The red test for round 4's move: no flags must EQUAL the three flags.
Measured on a grid table, which is the one shape Arm E decides: with the
arm off, each rule line closes the block, so the sheet lands as one concept
per row group with a title naming a LINE NUMBER -- `Tabell linje 6` -- and
the fold has no whole table to fold back into the heading above it. That is
the defect the round-3 default shipped with, and it is asserted on titles
rather than a count because a wrong cut can still produce a right number.
"""
inbox = inbox_for_grid_sheet(tmp_path)
plans = tmp_path / "plans-grid-default"
assert build(inbox, tmp_path / "grid-default", "--plans-dir", str(plans)) == 0
assert plan_titles(plans) == ["7 Romskjema"]
explicit = tmp_path / "plans-grid-explicit"
assert (
build(
inbox,
tmp_path / "grid-explicit",
"--plans-dir",
str(explicit),
"--outline-run",
"3",
"--table-grid",
"--unit-fold",
)
== 0
)
assert plan_titles(explicit) == plan_titles(plans)
def test_arm_e_in_the_default_has_an_explicit_opt_out(tmp_path: Path) -> None:
"""The other half: `--no-table-grid` gets the pre-move cut back, and says so.
Same rule every other arm in the default follows -- a default a caller
cannot turn off is not a default. The titles asserted here are the ones the
round-3 default produced on this fixture.
"""
inbox = inbox_for_grid_sheet(tmp_path)
plans = tmp_path / "plans-grid-opt-out"
assert (
build(inbox, tmp_path / "grid-opt-out", "--plans-dir", str(plans), "--no-table-grid") == 0
)
assert plan_titles(plans) == [
"7 Romskjema",
"Tabell linje 6",
"Tabell linje 8",
"Tabell linje 10",
]

49
tests/test_okf_consume.py
View file

@ -1182,20 +1182,39 @@ def test_the_readme_consume_section_states_the_rule_count_the_code_emits() -> No
def test_the_readme_recipe_names_only_commands_this_repository_ships() -> None:
# Every command in the "Consume in Claude Code" section was run in the
# session that wrote it. This test cannot re-run them; what it can hold is
# that each script the recipe invokes still exists under the path it names.
"""Every command the recipe invokes must exist. What "exist" means MOVED.
Until 2026-09-08 (O5) the recipe told a reader to run `python3
tools/<script>.py`, and this test held that each of those three files was
on disk. The recipe now names `okf` subcommands, because a reader who
installed this library has no `tools/` directory at all -- so the check
that means the same thing is that each subcommand is one the CLI
registers. That is a stronger claim than a file existing: a script can be
present and unreachable from the installed command, which is exactly the
defect O5 fixed.
The section's own heading is the fixed point, not the form of the commands
inside it.
"""
readme = (PROJECT_ROOT / "README.md").read_text(encoding="utf-8")
recipe = readme.split("## Consume in Claude Code", 1)[1].split("\n## ", 1)[0]
scripts = set(re.findall(r"python3 (tools/\S+\.py)", recipe))
assert scripts == {
"tools/okf_skill.py",
"tools/okf_consume.py",
"tools/okf_contract_check.py",
}, scripts
for script in scripts:
assert (PROJECT_ROOT / script).is_file(), script
assert "okf build " in recipe
# No route back to the old form: a `python3 tools/...` line in the recipe
# is a line a reader without this repository cannot run.
assert re.findall(r"python3 (tools/\S+\.py)", recipe) == []
invoked = set(re.findall(r"^okf ([a-z]+)", recipe, flags=re.MULTILINE))
assert invoked == {"build", "consume", "check", "skill", "project"}, invoked
listed = subprocess.run(
[sys.executable, "-m", "llm_ingestion_okf.cli", "--help"],
capture_output=True,
text=True,
cwd=PROJECT_ROOT,
)
assert listed.returncode == 0
for command in invoked:
assert command in listed.stdout, command
# --- Step 11: the measurement scorer -----------------------------------------
@ -1340,7 +1359,11 @@ def test_every_vocabulary_member_is_long_enough_to_ever_match() -> None:
def test_the_vocabulary_is_one_list_and_names_no_corpus_document() -> None:
source = (PROJECT_ROOT / "tools" / "okf_consume.py").read_text(encoding="utf-8")
# The PATH moved on 2026-09-08 (O5) and nothing else in this test did. The
# pre-pass is `src/llm_ingestion_okf/consume.py` now; `tools/okf_consume.py`
# is an alias to it, so reading the old path would have measured a wrapper
# and passed on a file with no vocabulary in it at all.
source = (PROJECT_ROOT / "src" / "llm_ingestion_okf" / "consume.py").read_text(encoding="utf-8")
assert source.count("COST_VOCABULARY = (") == 1
leak = re.compile(r"del-ii-bilag|del-i-vedlegg|prisskjema|prissammenstilling|stange", re.I)
assert leak.findall(source) == []

187
tests/test_okf_project.py Normal file
View file

@ -0,0 +1,187 @@
"""`okf project`: one folder in, one bundle plus one skill out.
The command adds no rule and owns no flag that changes a bundle's bytes, so
these tests are mostly about that: the project bundle must be the SAME bytes
`okf build` writes for the same folder at the same stamp, or there are two
build paths and the reports are pinned to one of them.
"""
from __future__ import annotations
import hashlib
import subprocess
import sys
from pathlib import Path
import pytest
PROJECT_ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(PROJECT_ROOT / "src"))
from llm_ingestion_okf import project # noqa: E402
from llm_ingestion_okf.cli import build # noqa: E402
from llm_ingestion_okf.cli import main as okf_main # noqa: E402
from llm_ingestion_okf.errors import IngestError # noqa: E402
DOCUMENTS = {
"krav.md": (
"## 4 Grunnforhold\n\nGrunnen er morene over berg.\n\n"
"### 4.1 Loesmasser\n\nLoesmassene er telefarlige.\n"
),
"notat.md": "Et notat uten overskrift, uten tabell og uten nummerering.\n",
}
@pytest.fixture
def folder(tmp_path: Path) -> Path:
target = tmp_path / "Mine Dokumenter"
target.mkdir()
for name, body in DOCUMENTS.items():
(target / name).write_text(body, encoding="utf-8", newline="")
return target
def tree(root: Path) -> dict[str, str]:
return {
path.relative_to(root).as_posix(): hashlib.sha256(path.read_bytes()).hexdigest()
for path in sorted(root.rglob("*"))
if path.is_file()
}
def test_the_project_bundle_is_the_bytes_okf_build_writes(folder: Path, tmp_path: Path) -> None:
"""The invariant the whole command rests on: ONE build path, not two.
`okf project` runs `okf build` with this package's default and no flag list
of its own. If it ever grew one, a project bundle and a build bundle of the
same folder would differ, and every measurement report pinned to the build
path would be describing a bundle nobody produces.
"""
out = tmp_path / "project"
bundle, _, _ = project.create(folder, out=out)
reference = tmp_path / "reference"
build(folder, reference, bundle_id="mine-dokumenter", okf_version="0.2")
assert tree(bundle) == tree(reference)
def test_the_id_defaults_to_the_folder_name_in_the_id_grammar(folder: Path, tmp_path: Path) -> None:
out = tmp_path / "project"
bundle, skill_path, _ = project.create(folder, out=out)
assert bundle == out / ".okf" / "mine-dokumenter"
assert skill_path == out / ".claude" / "skills" / "mine-dokumenter-consume" / "SKILL.md"
assert bundle.is_dir() and skill_path.is_file()
def test_a_named_id_is_used_verbatim(folder: Path, tmp_path: Path) -> None:
out = tmp_path / "project"
bundle, skill_path, _ = project.create(folder, out=out, bundle_id="anbud-2026")
assert bundle.name == "anbud-2026"
assert skill_path.parent.name == "anbud-2026-consume"
def test_a_folder_name_that_reduces_to_nothing_refuses_by_code(tmp_path: Path) -> None:
"""A refusal with a code, not a bundle called `""`.
A folder named only in punctuation reduces to the empty string, and an
empty bundle id would produce a bundle whose concepts join on nothing.
"""
weird = tmp_path / "..."
weird.mkdir()
(weird / "a.md").write_text("# A\n\nKropp.\n", encoding="utf-8", newline="")
with pytest.raises(IngestError) as caught:
project.create(weird, out=tmp_path / "project")
assert caught.value.code == "manifest_invalid"
def test_the_folder_name_is_normalised_before_it_is_reduced(tmp_path: Path) -> None:
"""NFC first, for the reason the rest of this package normalises first.
macOS hands a filename over decomposed, so `é` arrives as `e` plus a
combining acute. Reduced without normalising, the same visible folder name
produces two different bundle ids depending on which form it arrived in.
"""
assert project.slug("Prosjekt É") == project.slug("Prosjekt É")
def test_the_summary_names_the_documents_that_landed_whole(folder: Path, tmp_path: Path) -> None:
"""SS 6.4 discipline applied to a summary: the number carries its denominator.
`notat.md` has no heading, no table and no numbered outline, so the rules
find no boundary and it lands as one concept. A reader who is told only
"3 concepts" cannot tell that asking about that document returns the whole
of it as one excerpt.
"""
out = tmp_path / "project"
_, _, summary = project.create(folder, out=out)
assert "Read 2 document(s)" in summary
assert "1 of 2 document(s) landed WHOLE" in summary
assert "notat.md" in summary
assert "krav.md" not in summary
assert "[sourced-not-sufficient]" in summary
assert f"NEXT: start claude again in {out}" in summary
def test_a_document_that_is_in_the_bundle_is_not_reported_as_missing(
folder: Path, tmp_path: Path
) -> None:
"""The known-positive for `inventory`'s first list.
Its own control: with every document ingested the list must be empty, and
with the bundle read against a DIFFERENT folder every document must appear.
A search that cannot find would report an empty list either way.
"""
out = tmp_path / "project"
bundle, _, _ = project.create(folder, out=out)
missing, whole = project.inventory(folder, bundle)
assert missing == ()
assert whole == ("notat.md",)
other = tmp_path / "other"
other.mkdir()
(other / "fremmed.md").write_text("# Fremmed\n\nKropp.\n", encoding="utf-8", newline="")
stranger, _ = project.inventory(other, bundle)
assert stranger == ("fremmed.md",)
def test_the_generated_skill_names_no_path_into_this_repository(
folder: Path, tmp_path: Path
) -> None:
"""O5's whole point, asserted where a user actually meets it.
The known-positive runs first: the string this searches for occurs in the
environment running the test, so a zero means the generator kept it out.
"""
out = tmp_path / "project"
_, skill_path, _ = project.create(folder, out=out)
text = skill_path.read_text(encoding="utf-8")
assert str(PROJECT_ROOT) in str(Path(__file__).resolve())
assert str(PROJECT_ROOT) not in text
assert "okf consume" in text
assert "okf check" in text
def test_the_subcommand_exists_and_reports_zero(folder: Path, tmp_path: Path) -> None:
assert okf_main(["project", str(folder), "--out", str(tmp_path / "project")]) == 0
def test_a_missing_folder_is_two_and_not_one(tmp_path: Path) -> None:
"""Three exit codes, not two: an unread folder is not a refused build."""
assert okf_main(["project", str(tmp_path / "nope"), "--out", str(tmp_path / "p")]) == 2
def test_the_installed_command_reaches_every_subcommand() -> None:
"""`okf --help` must LIST them, or a reader has to be told they exist.
The dispatch happens before argparse, so without the registration in
`parse_args` these four would work and be invisible.
"""
listed = subprocess.run(
[sys.executable, "-m", "llm_ingestion_okf.cli", "--help"],
capture_output=True,
text=True,
cwd=PROJECT_ROOT,
)
assert listed.returncode == 0
for command in ("build", "consume", "check", "skill", "project"):
assert command in listed.stdout, command

46
tests/test_okf_skill.py
View file

@ -89,17 +89,47 @@ def test_a_skill_generated_for_one_bundle_does_not_name_the_other(tmp_path: Path
@pytest.mark.parametrize("bundle", BUNDLES, ids=lambda path: path.name)
def test_the_generated_commands_are_absolute_so_a_caller_outside_this_repo_can_run_them(
bundle: Path, tmp_path: Path
) -> None:
# The skill is copied into someone else's `.claude/skills/`, where a
# relative `tools/okf_consume.py` resolves to nothing.
def test_the_generated_commands_name_this_repository_nowhere(bundle: Path, tmp_path: Path) -> None:
"""O5's red measurement, made a test: no path into a checkout, anywhere.
**This assertion REPLACES its own opposite, and the replacement is the
point.** The test here until 2026-09-08 required the emitted commands to be
ABSOLUTE, on the reasoning that a skill copied into someone else's
`.claude/skills/` cannot resolve a relative `tools/okf_consume.py`. Both
halves of that were true and the conclusion was still wrong: an absolute
path into THIS clone is not portable either, it is merely portable-looking.
Measured before the move, a skill generated from a checkout carried four
lines naming this checkout by absolute path, two of them the commands a
reader is told to run -- so the skill could not be moved, shared, or run by
anyone without that clone at that exact path.
It also caught nothing by then: the regex it looped over matched zero lines
once the commands stopped being `python3 <file>.py`, so it was green over
an empty set. The assertion below has a denominator that cannot go to zero.
"""
written = _generate(bundle, tmp_path / bundle.name)
text = written.read_text(encoding="utf-8")
for command in re.findall(r"^\S*python3? (\S+\.py)", text, flags=re.MULTILINE):
assert Path(command).is_absolute(), command
assert Path(command).is_file(), command
# The BUNDLE root is the one absolute path that belongs here: it points at
# the caller's data. These fixtures happen to live inside this repository,
# so it is removed before the search -- otherwise the search would find the
# repository root inside the one path allowed to carry it.
assert str(bundle.resolve()) in text
rest = text.replace(str(bundle.resolve()), "<BUNDLE>")
# The known-positive for the search: the string it hunts for DOES occur in
# the environment running it, so the zero below means the generator kept it
# out rather than the search being unable to find it.
assert str(PROJECT_ROOT) in str(Path(__file__).resolve())
assert str(PROJECT_ROOT) not in rest
# No file under `tools/` at all, and not only the two commands: the
# attribution lines named the generator by path too, which is a file the
# reader does not have either.
assert "tools/" not in rest
# And what it names instead: commands resolved by PATH after an install.
assert "\nokf consume \\\n" in text
assert "\nokf check \\\n" in text
@pytest.mark.parametrize("bundle", BUNDLES, ids=lambda path: path.name)