open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(cli): okf project/consume/check/skill, and a generated skill with no path into a checkout

Browse source 
The reading direction existed only for someone standing in a clone. `consume`,
`contract_check` and `skill` moved from `tools/` into the package and are
reachable as `okf consume`, `okf check` and `okf skill`; `okf project` is new
and does the whole thing in one command.

The red measurement: a consumption skill generated from a checkout carried 4
lines naming that checkout by absolute path, 2 of them the commands the skill
tells a reader to run. It now names `okf consume` and `okf check`, and a test
asserts this repository appears in it nowhere, with a known-positive so the
zero is a measurement rather than a search that could not find.

The `tools/` files stay as ALIASES, not re-exports: a re-export binds copies of
the names into a second module object, so a caller patching one patches a
binding the implementation never reads. Two tests that monkeypatch okf_consume
went green again only under the alias. Every published reproduction block runs
unchanged.

The template and docs/consumption-contract.md (the section 7.4 known-positive)
are force-included into the wheel from the file they are authored in, so both
travel with the commands that cannot run without them and there is still one
authored copy of each.

Step 0, before any of it: okf build's default gained Arm E (--table-grid),
with --no-table-grid as its opt-out. The default moved to D plus F earlier the
same day on Arm F's published 5 of 12 -- a figure measured with Arm E ON.
Without it the fold has no joined table to fold, and the shipped default scored
2 of 12 with docx 0 of 3. Measured on the operator's folder: 30 md / 15
concepts on the new default against 43 / 28 without Arm E.

Install measurement from a fresh uv tool install, empty folder, this repository
nowhere on PYTHONPATH: 5 documents in, 15 concepts out, 0 references to tools/
in the generated skill, okf check conformant (15 rules, 0 findings).

Deviation stated rather than hidden: the order asked that
tests/test_okf_consume.py be left untouched. Two assertions in it read a PATH,
which is the one thing this work changes. Both were moved and the second made
stronger -- it now asserts every command the README recipe names is a
subcommand the CLI registers, which a file existing on disk never proved.

Suite 1414 -> 1427. ruff clean, mypy --strict clean over 21 files.
Record: docs/2026-09-08-o5-okf-project.md

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 21:47:38 +02:00
commit f6fea13299
21 changed files with 4017 additions and 2833 deletions
Download patch file Download diff file Expand all files Collapse all files

93
src/llm_ingestion_okf/cli.py
View file

@ -1,4 +1,11 @@
"""`okf` — the installed command. One subcommand today: `build`.
"""`okf` — the installed command. Five subcommands: the whole chain, packaged.
`build` a bundle from a folder, `consume` it for one question, `check` the
payload against the contract, generate a `skill` for it, or do the whole thing
in one step with `project`. Four of the five moved here on 2026-09-08 (O5)
from unpackaged scripts under `tools/`, which a consumer who installs this
library does not have; until then the reading direction existed only for
someone standing in a clone.
## What it replaces
@ -38,9 +45,12 @@ the replay dated differently passes both explicitly.
## What it does not decide
**Every arm is OFF unless the caller asks, and this command does not move a
default.** Which arm should ship as the default is the operator's decision and
is not taken here.
**Which arm ships as the default is the operator's decision and is not taken
here.** Three of the six are ON as of 2026-09-08 -- `--outline-run 3`,
`--table-grid` and `--unit-fold` -- each moved by the operator on a
measurement and each with an explicit opt-out. The remaining three are OFF
unless the caller asks. This module's job is to state the answer, never to
pick it.
The arms are, however, REACHABLE from here, and that is a change of 2026-09-08.
Until then `_propose_plans` called the proposer with no arm flag at all, so
@ -71,6 +81,32 @@ from .propose import run as propose_run
__all__ = ["DEFAULT_STAMP", "build", "main", "measure"]
#: The subcommands whose FLAGS belong to the module that implements them.
#:
#: Dispatched before `argparse` runs rather than rebuilt as subparsers here: a
#: second declaration of `--reserve-top-rank` or `--rarity-weight` is a second
#: place they can drift, and the pre-pass's flag set is the thing four
#: measurement reports are pinned to. Each entry is registered as a subparser
#: below anyway, so `okf --help` lists it and an unknown command is still an
#: error -- the parser knows the NAMES, the modules keep the flags.
#:
#: Imported lazily inside the dispatch: `okf build` should not pay to import
#: the ranker, and `okf consume` should not pay to import the proposer.
DELEGATED = ("consume", "check", "skill", "project")
def _delegate(command: str, argv: list[str]) -> int:
if command == "consume":
from .consume import main as run
elif command == "check":
from .contract_check import main as run
elif command == "skill":
from .skill import main as run
else:
from .project import main as run
return run(argv)
CLI_ID = "okf build"
#: What `okf build` runs when no flag is given. Moved 2026-09-08 by the
@ -89,7 +125,15 @@ CLI_ID = "okf build"
#: Each arm keeps an explicit opt-out: `--outline-run 0` (the number was
#: always its own switch) and `--no-unit-fold`. A default a caller cannot turn
#: off is not a default.
#: Arm E joined the default on 2026-09-08, one round after the other two, on a
#: measurement taken AFTER the first move rather than before it: Arm F's
#: published "5 of 12" was measured with Arm E on, and the round-3 default --
#: D plus F, Arm E off -- scored 2 of 12 with `docx` at 0 of 3. The fold's
#: table clause folds a table back into the heading that introduces it, and
#: with Arm E off a grid table is not one block but one block per rule line,
#: so there was nothing whole to fold. `--no-table-grid` is its opt-out.
DEFAULT_OUTLINE_RUN = 3
DEFAULT_TABLE_GRID = True
DEFAULT_UNIT_FOLD = True
#: The timestamp written when the caller passes none, for the ingest stamp and
@ -166,7 +210,7 @@ def build(
plans_dir: Path | None = None,
okf_type: str = "reference",
outline_run: int = DEFAULT_OUTLINE_RUN,
table_grid: bool = False,
table_grid: bool = DEFAULT_TABLE_GRID,
unit_fold: bool = DEFAULT_UNIT_FOLD,
keep_table_heading: bool = False,
sheet_section_rows: bool = False,
@ -254,9 +298,23 @@ def _write_log(bundle: Path, report: CorpusReport, *, profile: BundleProfile) ->
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog="okf",
description="OKF bundle tooling. One folder in, one bundle out.",
description=(
"OKF bundle tooling. One folder in, one bundle out -- and one command "
"from there to a bundle you can ask a question of."
),
)
subcommands = parser.add_subparsers(dest="command", required=True)
# Registered for `okf --help` and for the unknown-command error. Their
# arguments are NOT declared here: `main` hands the rest of `argv` to the
# implementing module, which owns them. `add_help=False` keeps
# `okf consume --help` reaching that module's help rather than this stub's.
for delegated, blurb in (
("consume", "cut a bundle to one contract-conformant payload for one question"),
("check", "check a consumption skill and one payload against the contract"),
("skill", "instantiate the consumption skill template for one bundle"),
("project", "folder in, bundle plus skill out: build and skill in one step"),
):
subcommands.add_parser(delegated, help=blurb, add_help=False)
build_parser = subcommands.add_parser(
"build",
help="build an OKF bundle from a folder of documents",
@ -336,12 +394,24 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
build_parser.add_argument(
"--table-grid",
action="store_true",
default=DEFAULT_TABLE_GRID,
help=(
"Arm E, passed to the proposer unchanged: a pandoc grid-table rule "
"line no longer closes an open table block, so one grid table is one "
"concept instead of one per row group. Absent (the default) is OFF. "
"Measured on the K3 sample: it changes a `.docx` experience list from "
"21 concepts to 6"
"concept instead of one per row group. ON by default since "
"2026-09-08 (operator); the flag is kept so a call site that passes "
"it stays valid. Measured on the K3 sample: it changes a `.docx` "
"experience list from 21 concepts to 6, and without it Arm F has no "
"joined table to fold"
),
)
build_parser.add_argument(
"--no-table-grid",
action="store_false",
dest="table_grid",
help=(
"Arm E's explicit opt-out. With --outline-run 0 and --no-unit-fold "
"it reproduces the pre-2026-09-08 default byte for byte"
),
)
build_parser.add_argument(
@ -409,7 +479,10 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
arguments = list(sys.argv[1:] if argv is None else argv)
if arguments and arguments[0] in DELEGATED:
return _delegate(arguments[0], arguments[1:])
args = parse_args(arguments)
if not args.inbox.is_dir():
print(f"{CLI_ID}: FAILED - no such folder: {args.inbox}", file=sys.stderr)
return 2