feat(cli): okf project/consume/check/skill, and a generated skill with no path into a checkout

The reading direction existed only for someone standing in a clone. `consume`,
`contract_check` and `skill` moved from `tools/` into the package and are
reachable as `okf consume`, `okf check` and `okf skill`; `okf project` is new
and does the whole thing in one command.

The red measurement: a consumption skill generated from a checkout carried 4
lines naming that checkout by absolute path, 2 of them the commands the skill
tells a reader to run. It now names `okf consume` and `okf check`, and a test
asserts this repository appears in it nowhere, with a known-positive so the
zero is a measurement rather than a search that could not find.

The `tools/` files stay as ALIASES, not re-exports: a re-export binds copies of
the names into a second module object, so a caller patching one patches a
binding the implementation never reads. Two tests that monkeypatch okf_consume
went green again only under the alias. Every published reproduction block runs
unchanged.

The template and docs/consumption-contract.md (the section 7.4 known-positive)
are force-included into the wheel from the file they are authored in, so both
travel with the commands that cannot run without them and there is still one
authored copy of each.

Step 0, before any of it: okf build's default gained Arm E (--table-grid),
with --no-table-grid as its opt-out. The default moved to D plus F earlier the
same day on Arm F's published 5 of 12 -- a figure measured with Arm E ON.
Without it the fold has no joined table to fold, and the shipped default scored
2 of 12 with docx 0 of 3. Measured on the operator's folder: 30 md / 15
concepts on the new default against 43 / 28 without Arm E.

Install measurement from a fresh uv tool install, empty folder, this repository
nowhere on PYTHONPATH: 5 documents in, 15 concepts out, 0 references to tools/
in the generated skill, okf check conformant (15 rules, 0 findings).

Deviation stated rather than hidden: the order asked that
tests/test_okf_consume.py be left untouched. Two assertions in it read a PATH,
which is the one thing this work changes. Both were moved and the second made
stronger -- it now asserts every command the README recipe names is a
subcommand the CLI registers, which a file existing on disk never proved.

Suite 1414 -> 1427. ruff clean, mypy --strict clean over 21 files.
Record: docs/2026-09-08-o5-okf-project.md

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 21:47:38 +02:00
commit f6fea13299
21 changed files with 4017 additions and 2833 deletions

View file

@ -0,0 +1,263 @@
"""One folder of documents in, one questionable project out, in one command.
`okf project <folder>` is `okf build` followed by `okf skill`, plus the summary
a person needs in order to know what they just got. It adds no rule of its own
and owns no flag that changes a bundle's bytes: the build runs on THIS
package's default, so a project bundle and an `okf build` bundle of the same
folder at the same stamp are the same bytes.
**Why a third command rather than a documented three-step.** The three-step
existed and was measured on a reader: set `PYTHONPATH`, take a snapshot of a
checkout, run `python3 tools/okf_skill.py`, and know which of three directories
each artefact belongs in. The operator's word for it was "extremely cryptic",
and that is a measurement of the instructions, not of the reader. Everything
this module does was already reachable; what was missing was that it was one
thing.
**The two destinations are conventions, not choices this module invents.**
`<out>/.okf/<id>/` keeps the bundle out of the way of the documents it was
built from -- the drop folder is walked recursively, so a bundle written beside
the documents would be its own input on the next run. `<out>/.claude/skills/`
is where Claude Code looks, and the skill is the whole point: a bundle nobody
can ask a question of is a directory.
Exit codes are three, as everywhere in this chain: 0 the project was written,
1 the run happened and refused, 2 the run did not happen.
"""
from __future__ import annotations
import argparse
import re
import sys
import unicodedata
from pathlib import Path
from . import consume, skill
from .cli import DEFAULT_STAMP, build
from .corpus import CorpusReport
from .errors import IngestError
from .inbox import walk_inbox
from .profiles import SEGMENTED_OKF_V0_2
CLI_ID = "okf project"
#: Where the bundle and the skill go under `--out`. Constants rather than
#: flags: a project whose layout varies per run is one whose summary cannot
#: tell a reader where anything is.
BUNDLE_DIR = ".okf"
SKILLS_DIR = Path(".claude") / "skills"
#: What the bundle declares as its upstream version. A VALUE, and normally the
#: caller's (decision E1) -- but `okf project` has no catalog to ask, and a
#: required flag here would put the one-command form back behind a question
#: nobody standing in front of a folder of PDFs can answer. So it is stated:
#: the version the segmented profile this command builds under was written for.
PROJECT_OKF_VERSION = "0.2"
_ID_SAFE = re.compile(r"[^a-z0-9]+")
def slug(name: str) -> str:
"""A folder name reduced to `[a-z0-9-]`, or a coded refusal.
NFC first, for the reason `materialize.reduce_to_id_grammar` normalises:
macOS hands filenames over decomposed, so the same visible folder name
reduces two different ways depending on which form it arrived in.
"""
reduced = _ID_SAFE.sub("-", unicodedata.normalize("NFC", name).casefold()).strip("-")
if not reduced:
raise IngestError(
f"the folder name {name!r} reduces to nothing in the id grammar; pass --id",
code="manifest_invalid",
)
return reduced
def inventory(folder: Path, bundle: Path) -> tuple[tuple[str, ...], tuple[str, ...]]:
"""Two lists a reader needs and cannot get from a concept count.
The first is the dropped documents NO concept names as its source: they are
in the folder, they are not in the bundle, and no excerpt can quote them.
The second is the documents that landed WHOLE, as one flat concept at the
bundle root -- the ones the mechanical rules found no boundary in. They are
reachable, and reaching them returns the entire document as one excerpt,
which the budget will often refuse outright and which, when it does fit,
frequently does not carry the conclusion at the place a reader asked about.
That is the `[sourced-not-sufficient]` case, and it is the difference
between a bundle that has 15 concepts and a bundle that answers.
Read off the INDEX TREE and not off the directory, the same walk the
pre-pass uses: a summary computed by a rule the pre-pass does not share
could name a document as present that no question will ever reach.
"""
walked, _ = walk_inbox(folder, exclude=bundle)
dropped = {path.relative_to(folder).as_posix() for path in walked}
represented: set[str] = set()
whole: set[str] = set()
root_bundle_id = consume.root_bundle_id_of(bundle, profile=SEGMENTED_OKF_V0_2)
for concept_id in consume.enumerate_concepts(bundle, profile=SEGMENTED_OKF_V0_2):
path = bundle / f"{concept_id}{SEGMENTED_OKF_V0_2.paths.concept_suffix}"
concept = consume.read_concept(path, bundle_root=bundle, root_bundle_id=root_bundle_id)
represented.add(concept.source_file)
# A concept id with no `/` sits at the bundle root rather than under a
# per-document directory, which is what an unsegmented document
# produces. Measured on the artefact rather than read off the run's
# log: the log is a file a caller can delete.
if "/" not in concept_id:
whole.add(concept.source_file)
return tuple(sorted(dropped - represented)), tuple(sorted(whole))
def summarise(
folder: Path,
bundle: Path,
skill_path: Path,
out: Path,
report: CorpusReport,
concepts: int,
missing: tuple[str, ...],
whole: tuple[str, ...],
) -> str:
"""The plain-language summary, with a denominator on every number."""
lines = [
f"Read {report.n} document(s) from {folder}.",
f"Wrote {concepts} concept(s) to {bundle}.",
f"Wrote the skill to {skill_path}.",
"",
]
if missing:
lines.append(
f"{len(missing)} of {report.n} document(s) are in the folder and NOT in "
"the bundle, so no excerpt can quote them. A question about one of "
"these can only be answered `[sourced-not-sufficient]`:"
)
lines.extend(f" - {name}" for name in missing)
if report.codes:
lines.append(
" reason code(s): " + ", ".join(f"{code} x{count}" for code, count in report.codes)
)
else:
lines.append(
f"0 of {report.n} document(s) were left out of the bundle. Every "
"document in the folder is reachable to a question."
)
lines.append("")
if whole:
lines.append(
f"{len(whole)} of {report.n} document(s) landed WHOLE, as one concept "
"each: the rules found no heading, table or numbered outline to cut "
"them on. Asking about one of these returns the entire document as a "
"single excerpt, which is often refused for size and, when it fits, "
"often does not carry the answer at the place you asked about. Expect "
"`[sourced-not-sufficient]` there:"
)
lines.extend(f" - {name}" for name in whole)
else:
lines.append(
f"0 of {report.n} document(s) landed whole; every one was cut into "
"parts a question can reach separately."
)
lines.extend(
[
"",
f"NEXT: start claude again in {out} and ask your question.",
]
)
return "\n".join(lines)
def create(
folder: Path,
*,
out: Path,
bundle_id: str | None = None,
ingested_at: str = DEFAULT_STAMP,
force: bool = False,
) -> tuple[Path, Path, str]:
"""Build the bundle, generate the skill, return both paths and the summary.
Keyword-only with defaults, so a caller taking this as an API keeps a
source-compatible call when a parameter is added.
"""
identity = bundle_id if bundle_id is not None else slug(folder.resolve().name)
bundle = out / BUNDLE_DIR / identity
report = build(
folder,
bundle,
ingested_at=ingested_at,
bundle_id=identity,
okf_version=PROJECT_OKF_VERSION,
)
if report.unaccounted or report.merged + report.rejected != report.n:
raise IngestError(
f"K1b FAILED - merged ({report.merged}) + coded rejections "
f"({report.rejected}) != N ({report.n}). Unaccounted: "
f"{', '.join(report.unaccounted) or '(none named)'}",
code="conservation_failed",
)
skill_dir = out / SKILLS_DIR / f"{identity}-consume"
written = skill.generate(bundle, out=skill_dir, force=force)
concepts = len(consume.enumerate_concepts(bundle, profile=SEGMENTED_OKF_V0_2))
missing, whole = inventory(folder, bundle)
summary = summarise(folder, bundle, written, out, report, concepts, missing, whole)
return bundle, written, summary
def parse_args(argv: list[str] | None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
prog=CLI_ID,
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("folder", type=Path, help="the folder of documents to make questionable")
parser.add_argument(
"--id",
dest="bundle_id",
default=None,
help="the bundle id. Defaults to the folder's name reduced to [a-z0-9-]",
)
parser.add_argument(
"--out",
type=Path,
default=None,
help="where the project is written. Defaults to the current directory",
)
parser.add_argument(
"--ingested-at",
default=DEFAULT_STAMP,
help=f"stamped verbatim. Default {DEFAULT_STAMP}: deterministic, never the clock",
)
parser.add_argument(
"--force", action="store_true", help="replace an existing SKILL.md at the destination"
)
return parser.parse_args(argv)
def main(argv: list[str] | None = None) -> int:
args = parse_args(argv)
if not args.folder.is_dir():
print(f"{CLI_ID}: the run did not happen - no such folder: {args.folder}", file=sys.stderr)
return 2
out = args.out if args.out is not None else Path.cwd()
try:
_, _, summary = create(
args.folder,
out=out,
bundle_id=args.bundle_id,
ingested_at=args.ingested_at,
force=args.force,
)
except (IngestError, consume.ConsumeError, skill.SkillError) as exc:
print(f"{CLI_ID}: refused ({exc.code}) - {exc}", file=sys.stderr)
return 1
except OSError as exc:
print(f"{CLI_ID}: the run did not happen - {exc}", file=sys.stderr)
return 2
print(summary)
return 0
if __name__ == "__main__":
raise SystemExit(main())