open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

fix(skill): regenerate skills/okf-consume from the golden bundle it ships a payload for

Browse source 
The hand-filled skill predated `okf skill`, declared no bundle identity,
and was refused by the checker it tells its reader to run, against the
payload shipped beside it:

  NOT conformant: 16 rules over 3 excerpts and 0 withheld entries, 1 findings
    bundle_mismatch: the skill declares no readable bundle identity, ...

It is now the generator's output for
examples/ingest-golden-segmented-okf-v0-2/expected-bundle, the bundle the
payload always came from:

  conformant: 16 rules over 3 excerpts and 0 withheld entries, 0 findings

The payload's bytes do not move. --force and --example-question are both
required: without the question the payload is a different one (spent
2125 -> 1420, delivered 3 -> 2, withheld 0 -> 1) and the byte test fails.
`okf skill` writes the bundle root and the skill path absolute when --out
is not under .claude/skills/, so the checkout prefix is stripped in one
mechanical step that a test holds; the generator is unchanged.

The frontmatter name becomes b-golden-segmented-okf-v0-2-consume. Claude
Code takes a project skill's command from its directory, which stays
okf-consume, and nothing in this repository named the skill by name.
references/README.md is rewritten by hand with the full regeneration
command; living prose in README, CLAUDE.md, the test comment and the
skill.py docstring is corrected, dated journals are left as written.

Report: docs/2026-09-11-k3-runde18-konsumskillen-regenerert.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-11 02:21:35 +02:00
commit 6858ff2d35
8 changed files with 397 additions and 164 deletions
Download patch file Download diff file Expand all files Collapse all files

200
docs/2026-09-11-k3-runde18-konsumskillen-regenerert.md Normal file
View file

@ -0,0 +1,200 @@
# K3 round 18 — the shipped consumption skill is regenerated from the bundle it ships a payload for
**Date:** 2026-09-11 · **HEAD at start:** `f5a002e` · **Lock commit:** `07df6a0` ·
**Red test:** `c7d662e` · No push, no tag, no version bump, no new checker rule,
no new `okf skill` flag.
## 0 Deviations from the order, stated before anything is built on them
- **HEAD at start was `f5a002e`, not `7cca9e0`.** K3-17 landed three commits on
top. It touched none of `skills/`, `src/llm_ingestion_okf/skill.py`,
`contract_check.py` or `tests/test_okf_consume.py` (`git diff --stat
7cca9e0..HEAD` over those paths is empty), but it touched `consume.py` and
`README.md`, so every figure below was measured again on `f5a002e`. The README
lines the order cites moved by 16: `README.md:486` is now `:502`, and
`:489–495` is now `:505–510`.
- **` M uv.lock` was not present at session start**; the working tree was clean
and the tracked lock still said `0.7.0`. The one-line change was produced by
`uv lock` in a separate `git archive` export and copied in. Its diff against
the tracked lock is exactly the line the order names, and nothing else.
- **`okf skill` writes two ABSOLUTE paths into this checkout**, which the order
did not mention. `project_root_of(out)` recognises only `<root>/.claude/skills/<name>`,
so for `--out skills/okf-consume` the bundle root and the skill's own path are
written absolute. Measured in the working tree, lines 26 and 41 of the
generated file carried `/Users/…/llm-ingestion-okf/`. Shipped as generated,
the two commands in a public file would name one checkout on one machine. The
generator was not changed (§ 2 says how this was handled).
- **One red in the working-tree suite that the order does not know about**, and
it is the invocation, not the code: run from a scratch cwd,
`tests/test_guard_adapter.py::test_the_guard_parses_the_flow_form_sources_our_goldens_emit`
fails with `FileNotFoundError`, because it opens
`Path("examples/ingest-golden-okf-v0-2/…")` relative to the cwd. From the
repository root it passes (1 passed). The order asks for a scratch cwd, and
that test cannot pass from one, so the working-tree suite was run from the
repository root.
## 1 Reproduced first, on `f5a002e`
Frozen export (`git archive` + `uv sync --frozen --extra extract`, uv 0.9.8,
CPython 3.11.14), with the ignored gold set copied in so the K2 pin runs rather
than skips. Exit codes captured directly (`out=$(…); rc=$?`), never through a pipe.
| what | order (PM) | this round |
|---|---|---|
| `okf check` on the shipped pair | rc 1, 1 finding | rc 1: `NOT conformant: 16 rules over 3 excerpts and 0 withheld entries, 1 findings` / `bundle_mismatch: the skill declares no readable bundle identity, so no payload can be shown to belong to it; a <PLACEHOLDER> left unfilled is not an identity, and neither is its absence (SS 3.1, SS 3.3)` |
| `len(RULES)` | 16 | 16 (`contract_check.py` line 519) |
| `uv lock --check`, tracked lock | rc 1 | rc 1, "needs to be updated" |
| `uv lock --check`, synced lock | rc 0 | rc 0, "Resolved 49 packages" |
| lock diff | 1 line | 1 line (`0.7.0` → `0.8.1`, line 550) |
| `okf skill` flags | `--out`, `--example-question`, `--force` | the same three; no `--name` |
| without `--force` | `refused (target_occupied)` | the same, rc 1, in scratch and in the working tree |
| without `--example-question` | `spent` 2125 → 1420, `delivered` 3 → 2, `withheld` 0 → 1 | identical |
| with `--example-question` | payload diff empty | empty |
| suite (export) | one known red | 1599 passed, 1 skipped, 1 failed (`test_the_four_existing_goldens_are_untouched`, no `.git` in an export) |
| ruff 0.16.6 / format / mypy | clean | clean / 120 files / 21 source files, no issues |
| K2 pin | 7 passed | 7 passed (`EXPECTED_CONCEPTS` 453, `EXPECTED_RANKS` `(1, 1, 1, 1, 1, 5)`) |
The one skipped test is `tests/test_extract.py:197`, which needs
`OKF_HTML_CORPUS` and is a skip by design.
## 2 What was built
**Red first (`c7d662e`), two tests in `tests/test_okf_consume.py`:**
1. `test_the_shipped_skill_passes_the_checker_against_its_own_payload` reads the
shipped `SKILL.md` and `references/example-payload.json` from disk and
requires `contract_check.check(...).findings == ()`. No existing test did
this: the checker tests build a skill in memory, so the file on disk was never
read. Red on HEAD with the `bundle_mismatch` finding above.
2. `test_the_shipped_skill_is_the_generator_output_with_the_checkout_made_relative`
requires the shipped file to equal `okf_skill.render(GOLDEN, out=SKILL.parent,
question="Hva sier veiledningen om krav?")` with `f"{PROJECT_ROOT}/"` removed,
and first asserts that the prefix occurs in the generated text, so the
equality is not a comparison of two texts that never carried it. Red on HEAD
(`name: okf-consume` against the generated file).
**The regeneration**, from the repository root with this checkout's own
`.venv/bin/okf` and never an `okf` on PATH:
```sh
okf skill examples/ingest-golden-segmented-okf-v0-2/expected-bundle \
--out skills/okf-consume --force --example-question "Hva sier veiledningen om krav?"
python3 -c 'import os, pathlib; p = pathlib.Path("skills/okf-consume/SKILL.md"); p.write_text(p.read_text(encoding="utf-8").replace(os.path.realpath(".") + "/", ""), encoding="utf-8")'
```
After the strip, the only absolute paths left are the two `/tmp/payload.json`
scratch targets the generator writes for every skill. `git diff --stat` before
the prose edits: `skills/okf-consume/SKILL.md | 224 +++---` (94 insertions, 130
deletions) and nothing else, so **the payload's bytes did not move**.
`references/README.md` is not written by `okf skill` and was rewritten by hand
with the three commands above plus `okf check`, and why each one is there.
**After:** `okf check --skill skills/okf-consume/SKILL.md --payload
skills/okf-consume/references/example-payload.json` →
`conformant: 16 rules over 3 excerpts and 0 withheld entries, 0 findings`, rc 0.
## 3 The frontmatter name
The regenerated file carries `name: b-golden-segmented-okf-v0-2-consume`; the
directory stays `skills/okf-consume/`. Three measurements decided it:
1. **Who names the skill by name.** `grep -rnE 'okf-consume([^/a-z-]|$)'` over
`README.md CLAUDE.md llms.txt docs/ tests/ src/ skills/ CHANGELOG.md` finds
the old file's own `name: okf-consume` (the known-positive) and one Python
path constant. **0 lines name the skill `okf-consume` as a name**; every other
mention is the path `skills/okf-consume/…`, and those stand.
2. **How Claude Code identifies a skill**, from
`https://code.claude.com/docs/en/skills` fetched 2026-09-11: "In a personal
or project skill, `name` sets only the display label shown in skill listings,
and the command still comes from the directory name"; in a plugin skill,
"`name` sets the last segment of the command". This repository has no plugin
manifest (no `.claude-plugin/`, no `plugin.json` outside `.venv`), so Claude
Code loads nothing from `skills/` here. A copy placed at
`.claude/skills/okf-consume/` is still `/okf-consume`; only its label moves.
3. **`okf skill` has no `--name` flag** (`okf skill --help`), and none was added.
So the generated name was accepted, and no line needed correcting as a skill name.
## 4 The command lines that name the skill, and this round's denominator
`grep -rn 'skills/okf-consume/SKILL.md'` over `README.md CLAUDE.md llms.txt docs/
skills/ src/ tests/` finds **4 command lines in code blocks** outside the skill's
own directory, the same four the order counted. **1 of the 4 runs exactly as
written**; the other three name a placeholder payload (`payload.json` twice,
`<a K2 payload>` once), so each was run with the payload it describes:
| line | payload | result | rc |
|---|---|---|---|
| `README.md:502` | the shipped example payload | `conformant: 16 rules over 3 excerpts and 0 withheld entries, 0 findings` | 0 |
| `docs/2026-09-07-okf-konsumskill-maaling.md:256` | a golden payload from `tools/okf_consume.py` | the same line | 0 |
| `docs/2026-09-08-claude-code-skill-vilkaarlig-bundle.md:38` | a K2 payload, built to scratch from the corpus, read only | `NOT conformant: 16 rules over 8 excerpts and 621 withheld entries, 1 findings`, `bundle_mismatch` naming both halves | 1 |
| `docs/2026-09-08-claude-code-skill-vilkaarlig-bundle.md:171` | `/tmp/c1-fresh-payload.json`, **as written** | `NOT conformant: 16 rules over 8 excerpts and 46 withheld entries, 1 findings`, `bundle_mismatch` | 1 |
The skill's own two blocks, run from the repository root: `okf consume
examples/ingest-golden-segmented-okf-v0-2/expected-bundle … --ref <its ref>` rc
0, and `okf check --skill skills/okf-consume/SKILL.md --payload <that payload>`
rc 0 with 0 findings.
**Green is 2 of 4, not 4 of 4, and should not be 4 of 4.** The two rc 1 rows are
the rule doing its job: the skill now names the golden bundle, and those
payloads come from two other bundles. Both lines are in dated journals, were
true of the hand-filled K2 copy on the day they were written, and were not
rewritten.
**Those two rows did not start failing this round.** The hand-filled copy
against the same K2 payload, measured on the `f5a002e` export, is already rc 1
(`the skill declares no readable bundle identity`), and `skills/okf-consume/SKILL.md`
is byte-identical from `113b3f8` to `f5a002e`: the lines have exited 1 since the
rule landed in `7cca9e0`, and this round changes only the wording of the
finding. The same holds for the one consumer measured: `portfolio-optimiser`
cites `tools/okf_contract_check.py --skill skills/okf-consume/SKILL.md` with K2
payloads in two dated documents (the rest of its hits are copies of this
repository's own tree in a scratch directory), the path it cites did not move,
and nothing there names the skill. No coordination message was sent.
## 5 Living prose corrected, dated prose left
Corrected: `README.md` (the paragraph after the check command),
`CLAUDE.md` (the paragraph on the first instantiated skill, and the
`--withheld-titles` sentence that cited the K2 copy's breaking point),
the comment in `test_the_skill_and_a_real_payload_pass_the_checker_together`,
the `skill.py` module docstring (printed by `okf skill --help`),
`skills/okf-consume/references/README.md`, and `CHANGELOG.md` under
`[Unreleased]`. Left as written: `docs/2026-09-08-o5-okf-project.md:153`,
`docs/2026-09-10-k3-runde15-bundle-mismatch.md:90`,
`docs/2026-09-09-o6-forsteskjerm-og-skillmodi.md:169`,
`docs/2026-09-08-rangeringsbom-sammensatte-ord.md:246`, and the four dated
command lines in § 4.
## 6 Honesty limits
- **"Green" means ONE skill/payload pair gives 0 findings over 16 rules.** It
says nothing about whether the skill answers well. The generated numbers
describe a three-concept bundle and nothing larger, and conformance is the
floor and never the proof; `okf check --help` says so itself.
- **The command-line denominator is mine**: 4, from one `grep` over the tracked
paths named in § 4. Lines in untracked files and in other repositories are not
in it, and 1 of the 4 was runnable as written.
- **The name choice rests on what was measured, not on what reads well**: one
documentation page as fetched on 2026-09-11 and one `grep`. Claude Code's
behaviour can change, and no copy of the skill was loaded in a Claude Code
session to observe the command.
- **The strip step departs from "regenerate rather than edit".** It is
mechanical and a test holds it, but the generator alone does not produce the
shipped bytes. A generator that wrote paths relative to the directory it runs
in would remove the step, and it would change `okf skill`'s output for every
caller, so it is not this round's to take.
- **The tests hold the bytes under the interpreter that runs them.** The
working tree runs CPython 3.14.0 and the export CPython 3.11.14; both suites
are in § 7.
## 7 After
Working tree, repository root, CPython 3.14.0, every file staged by name:
**1602 passed, 1 skipped** (the 1600 of `f5a002e` plus the two tests of § 2),
ruff clean, 120 files formatted, mypy clean over 21 source files. The K2 pin is
7 passed, and `test_no_corpus_document_name_reaches_any_file_this_work_tracks`
ran rather than skipped and passed, over the regenerated `SKILL.md` and the
rewritten `references/README.md`. The export of the delivery commit is measured
after the commit, because an export of uncommitted work is not an export; its
figures are in the session's report and in `STATE.md`.