llm-ingestion-okf/docs/2026-09-09-o6-forsteskjerm-og-skillmodi.md
Kjell Tore Guttormsen 6ff18fd703 feat(propose,extract,cli): a title that ends in a number, and a converter's own anchor in a concept id
Round 9: the four rests in STATE's NESTE that needed no operator decision.

CLAUSE 1 CLASSIFIED BY THE NUMBER, NOT THE TITLE. `_TRAILING_PAGE_NUMBER`
admitted a candidate into a contents run by asking whether the title ended in
an integer -- a question about the number. A drawing's dimension chain, a
schematic's labels, a door schedule, a coordinate column and a soil-layer
table all end in integers and name nothing. Measured over the 43-document
corpus: 68 candidates discarded over 11 of 39 readable documents, of which
19 over 5 documents are data rows.

That corrects round 8's own decomposition. Its "four misclassified numeric
tables and seven real contents listings" needs each document on one side, and
two of the eleven are both. Read across all 68 titles rather than the
three-title sample: 5 documents carry a data row, 8 carry a real entry.

`--contents-name` requires a NAME to survive stripping the page number. The
threshold is SWEPT, not chosen, and collapses at both ends: at an alphabetic
run of 1 a door schedule keeps a stray `V` and 13 of 19 are rescued; at 3 the
two-letter section name `VA` stops being a name, falls out of run membership,
and takes `RIB`, `MMI` and `Tittelfelt` below `CONTENTS_RUN` with it -- one
acronym costing four REAL entries. At 2: 16 of 19 rescued, 0 of 49 regressed.
The three not rescued carry a real word and are named rather than rounded off.

THE CONVERTER'S ANCHOR WAS IN THE CONCEPT ID. Pandoc writes a sheet as
`## <name> {#sheet-N}` and a titled slide as `## <title> {#slide-N}`. Because
a filename is reduced FROM the title, the anchor reached both. Operator
authorised the strip 2026-09-09 after the exposure was counted: 2 of 810
concepts on the previous default bundle, 2 of 1108 on Arm B, 1 of 26 on the
operator's folder. Two ids renamed, one of which `portfolio-optimiser` has
cited in writing; both are in the report so that message can be sent.

One rule in one function, read by BOTH title-forming sites -- a rule in only
one would leave the id and the title naming the same concept differently. The
known-negative is the point: `Mal for {kundenavn}` is a title an author wrote.

odt/rtf/pptx MEASURED END TO END FOR THE FIRST TIME, on hand-built documents,
because the corpus denominator is genuinely zero (86 files: 66 pdf, 10 docx,
4 xlsx, 2 zip, 2 smc, 2 doc). `_EVIDENCE` gains a third class rather than
stretching an existing one: `constructed` means the row has met a document,
but not one anyone wrote for their own purposes. odt 1 of 1 declared headings;
pptx 2 of 2 on a deck that declares slide titles and 0 of 2 on one that does
not -- round 7's reading of pptx was a fixture property, not the format; rtf
0 segments, because the container has no heading style and the author's title
is bold text. rtf is the one open finding.

ACCEPTANCE, all four. The 12-position reference is label-identical in BOTH
readings (pdf 7/8, docx 3/3, xlsx 0/1 or 1/1, sheet 10/12 or 11/12). One K2
bundle carrying both changes: 453 concepts / 865 md, hit@8 [1,1,1,1,1,None]
on it AND on Arm B, with the known-negative still reproducing on the new
bytes. `okf project` byte-equal to `okf build`, `diff -r` empty. Consumer
cost is a re-run: 436/832 -> 453/865, digest 21af4a1aa98315cf.

Three published numbers corrected: README's 596 tests (1515), README's "15
concepts out" for `okf project` (that was the O6 defect; it is 26), and O6's
print-mode method, which does not reproduce without --allowedTools.

Report: docs/2026-09-09-k3-runde9-restene.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-09 21:45:08 +02:00

16 KiB
Raw Blame History

O6: the first screen an agent reads, three modes, and one flag that made two builds

Session 119, order 20260909T155135Z-8080812956-from-.claude, on 9567145. Commit 1260fac, tag v0.7.0 (local, unpushed).

Everything below is a measurement with the command beside it. Where a figure this repository published before is wrong, the correction is stated rather than quietly replaced. § 7 states what was not measured.

1 The red measurement, and the query that could not find

1.1 The order's own premises, restated on HEAD

The order's PM figures (i)(iii) were measured on an export of a364ef4, one commit behind HEAD. Restated on 9567145:

Premise PM on a364ef4 Measured on 9567145 Command
suite 1498 1498 pytest -q
ruff clean clean ruff check src tests tools
mypy clean, 21 files clean, 21 files mypy --strict src
template SKILL.md 164 lines 164 wc -l < skills/okf-consume-template/SKILL.md
a GENERATED skill 231 lines 231 okf skill …/expected-bundle --out <scratch>
README.md 676 lines 679 wc -l < README.md
okf project install line line 317 line 320 grep -n "@v0\." README.md
~/okf-test/dokumenter default 52 md 30 md — see § 2 okf project ~/okf-test/dokumenter

The last row is not a restatement. It is a defect, and it is § 2.

1.2 The pins disagreed

$ grep -n "@v0\." README.md
26:  … llm-ingestion-okf.git@v0.4.0
40:  … llm-ingestion-pipeline-security.git@v0.2.0
41:  … llm-ingestion-okf.git@v0.4.0
320: … llm-ingestion-okf.git@v0.6.0
$ grep -n "@v0\." llms.txt
10:  … llm-ingestion-okf.git@v0.4.0

An agent reading either file from the top installed v0.4.0, which has no okf project. llms.txt — the file an agent fetches first — named neither okf project nor Claude Code at all.

1.3 The query that could not find (face 4)

docs/2026-09-08-o5-okf-project.md lines 16 and 177 measure absolute paths in a generated skill with grep -c "^/" and report 4 → 0. The generator indents those paths by two spaces, so ^/ cannot match one. Control, run before any zero was read:

$ printf 'prose with no path\n  /Users/x/bundle\nokf consume /Users/x/other\n' > kp.txt
$ grep -c "^/" kp.txt                      # O5's query
0                                          # <- cannot find a known positive
$ grep -cE '(^|[[:space:]])/[A-Za-z]' kp.txt
2                                          # <- can

Applied to a generated skill on 9567145:

$ grep -nE '(^|[[:space:]])/[A-Za-z]' <skill>/SKILL.md
26:  /Users/ktg/repos/llm-ingestion-okf/examples/…/expected-bundle \
29:  --out /tmp/payload.json
41:  --skill /private/tmp/o6-red/skillgen/SKILL.md \
42:  --payload /tmp/payload.json
$ grep -n "$HOME" <skill>/SKILL.md
26:  /Users/ktg/repos/…

Two paths into the machine the skill was generated on (26 and 41), confirming the order's (iii). The other two are /tmp/payload.json, a scratch write target that names no checkout. O5's "0" was never a measurement — the query it used could not have produced anything else.

1.4 The modes, counted per term

The order's (ii) is confirmed in substance and corrected in form. Counted with grep -ic <term> skills/okf-consume-template/SKILL.md:

term template generated
hypothes 0 0
task 0 0
oppgave 0 0
notat 0 0
write 4 2
produce 7 15
confirm 1 1
refut 0 0

Every write and produce hit is unrelated ("the producer wrote none", "Write to <OUT>"). The modes were absent; the claim "0 occurrences of hypothes/task/write/produce" was still false as a measurement.

2 The defect the reproduction found: two builds, one command

okf project ~/okf-test/dokumenter wrote 15 concepts / 30 files. okf build on the same folder wrote 26 / 52. CLAUDE.md states the opposite — "it owns NO flag that moves a bundle's bytes and a test holds it byte-equal to okf build" — and a test by that name exists and was green.

Mechanism. cli.build's Python signature carried keep_table_heading: bool = False and sheet_section_rows: bool = False while argparse carried DEFAULT_KEEP_TABLE_HEADING = True and DEFAULT_SHEET_SECTION_ROWS = True. Every other flag reads its DEFAULT_ constant in both places; these two did not. project.create calls build() as a function and passes no flag list, so it read the signature. The whole 22-file difference was inside del-ii-bilag-7-prisskjema/ — the priced spreadsheet, which is the document a question about price has to reach.

Why the test could not see it. test_the_project_bundle_is_the_bytes_okf_build_writes compares project.create against the same build() function, so both sides carried the same wrong value; and its two fixture documents (krav.md, notat.md) contain neither a table nor a sheet, so the difference could not have appeared even against argparse. A test and the code agreeing over a set where the difference cannot appear.

Repair, red first. Two tests, both red before the change:

  • test_the_build_signature_defaults_are_the_build_command_defaults — compares the signature's default against argparse's for every parameter argparse passes through at the same type. Red output named exactly the two: {'keep_table_heading': (False, True), 'sheet_section_rows': (False, True)}. Parameters argparse converts (segments 'on'→bool, pdf_headings 'none'→bool, okf_version caller-owned) are excluded by the type test, not by a list.
  • test_a_sheet_reaches_the_project_bundle_as_it_reaches_the_build_command — the same folder through project.create and through okf_main(["build", …]), on a document whose concept count actually moves with the two flags. On the first draft (a two-row table) it passed; that draft is recorded here because it is the same failure as the invariant it replaces — a green over an empty set. With five numeric rows it went red (2 segments vs 7) and is the shipped form.

After. okf project writes 26 concepts / 52 files, and diff -rq against the okf build bundle of the same folder reports 0 differences.

3 A — the first screen

README.md now opens with, in this order: what it is in one sentence; one uv tool install line; okf project <folder> and claude; the three shapes of request; one line saying where the rest is. The phase-status paragraph that used to be lines 319 moved down under ## What this library is — moved, not deleted.

llms.txt was rewritten in the same shape: the new tag, the okf project form, the three modes, and a pointer into the Claude Code section.

Acceptance, measured:

$ grep -o "@v0\.[0-9.]*" README.md llms.txt | sed 's/.*://' | sort -u
@v0.7.0
$ printf 'x %s/y\n' "$HOME" > kp.txt; grep -c "$HOME" kp.txt   # control: can find
1
$ grep -c "$HOME" README.md llms.txt
README.md:0
llms.txt:0

Earlier tags are kept under ### Earlier tags, as history, labelled as history and not as commands. The guard pairing is measured, not guessed: v0.7.0's own [tool.uv.sources] names llm-ingestion-guard v1.3.0, and § 5 shows the install resolving exactly that.

4 B — three modes, relative paths

## Modes in the template, the instantiated skills/okf-consume/SKILL.md, and therefore in every generated skill. Question (the default), Hypothesis, and Task that produces a document. The five markings are untouched — a test asserts every literal okf check fixes is still present, so the modes added no sixth.

Relative paths: skill.project_root_of(out) reads <root>/.claude/skills/<name> off the path and returns <root>; skill.as_written(path, base=root) writes a path relative to it when it is under it, and leaves it absolute otherwise. A path outside the project root stays absolute on purpose: ../../.. is not more portable, only harder to read.

Measured on a bundle built by okf project in a cold directory:

$ grep -nE '(^|[[:space:]])/[A-Za-z]' .claude/skills/dokumenter-consume/SKILL.md
29:  --out /tmp/payload.json
42:  --payload /tmp/payload.json
$ grep -c "$HOME" .claude/skills/dokumenter-consume/SKILL.md
0
$ grep -n "\.okf/dokumenter\|\.claude/skills/dokumenter-consume/SKILL.md" <skill>
26:  .okf/dokumenter \
41:  --skill .claude/skills/dokumenter-consume/SKILL.md \

Two paths into the machine → 0. Both commands run from the project root:

$ okf consume .okf/dokumenter --question … --ref … --out /tmp/payload.json ; echo $?
0
$ okf check --skill .claude/skills/dokumenter-consume/SKILL.md --payload /tmp/payload.json
conformant: 15 rules over 8 excerpts and 18 withheld entries, 0 findings

5 The install measured from a cold, empty directory

uv tool install cannot point at an unpushed tag over git, so the tag was exported and installed from a local path. This is stated rather than glossed: the bytes are the tag's (git archive v0.7.0), the channel is not.

$ git archive v0.7.0 | tar -x -C /tmp/o6-install/export
$ uv tool install --force "llm-ingestion-okf[extract] @ file:///tmp/o6-install/export"
 + llm-ingestion-guard==1.3.0 (from git+…@44e2b31…)
 + llm-ingestion-okf==0.7.0 (from file:///tmp/o6-install/export)
Installed 1 executable: okf

The guard pairing the README states is what resolved. Then, in an empty directory, with no PYTHONPATH, no VIRTUAL_ENV, and this repository nowhere on PATH (which okf~/.local/bin/okf):

documents in 5
concepts out 26
files out 52 md
left out of the bundle 0 of 5
landed whole as one concept 1 of 5 (named in the output)
okf check conformant, 15 rules, 0 findings
paths into any checkout in the skill 0

Order of operations, as the order required: the tag was created (§ 6) before this was run, because the measurement is of the tag.

6 The three end-to-end runs (C1's instrument)

Skills installed in a project directory outside this repository; cd <project> && claude -p --allowedTools=Bash,Read,Grep,Glob "<the question>" (row 3, which produces a document, additionally Write,Edit); one run per row, no re-run for a better answer. Sources grep-verified against the bundle tree with the denominator stated.

CORRECTED 2026-09-09 (round 9). This section first stated the method as cd <project> && claude -p "<the question>", without the tool list. Re-run that way -- including with --permission-mode acceptEdits -- the okf consume call is refused, the model answers without ever opening the bundle, and a hypothesis run marks both premises undecidable-from-bundle. All four runs reported below in fact used the explicit --allowedTools list above; the method line omitted it, so as written the section did not reproduce. The NUMBERS are unaffected -- what was wrong was the recipe for getting them.

# Mode Asked Sources verified Denominator
1 question "hva er kravene til pris?" 8 of 8 sha256 prefixes found in the payload 8 excerpts in the payload
2 hypothesis "stemmer det at totalentreprenøren bærer risikoen for massehåndtering, og at massene prises som fastpris i basiskostnaden?" 2 of 2 sha256 prefixes; the verbatim quote found in 1 file 8 excerpts; 52 md files in the bundle
3 document task "lag krav-pris.md med alle krav til pris, ett avsnitt per krav, med dokument og kravnummer" 28 of 28 64-hex digests; 24 of 24 concept paths exist on disk 80 known digests (excerpt fields + 52 concept files + 5 source documents + ref)

Invented numbers: 0 across the three runs, against the denominators above. Every digest and every concept path a run cited was found.

Run 2 answered per premise, which is what the mode asks for: premise 2 refuted with the contradicting sentence quoted, premise 1 split into mengderisiko (refuted, derived), enhetsprisrisiko (confirmed, extracted), the tiltaksklasse question undecidable-from-bundle marked [sourced-not-sufficient] on that premise, and the hypothesis-level verdict stated as derived. No fourth verdict value appeared.

Run 3 wrote the document. 221 lines, 37 requirement paragraphs, and — the part that matters — the discipline is in the file: 24 extracted, 18 derived, 3 [unverifiable-from-bundle], 1 [unread], 2 [sourced-not-sufficient] standing where the requirement belongs rather than dropped, the bundle ref, and considered / withheld / delivered declared in the document's own header table. It also reported that req_number is 0 of 26 in this bundle and marked its own numbering derived rather than inventing requirement numbers — which is the failure the mode exists to prevent.

7 C — two bundles in one project

Two okf project runs with different --id on two topically different document sets, split from ~/okf-test/dokumenter into scratch (~/okf-test read-only):

.okf/pris      13 concepts   ref sha256-tree:477ed72de1d3be18…
.okf/teknikk   10 concepts   ref sha256-tree:e53bed17f7b8a5e2…
.claude/skills/pris-consume   .claude/skills/teknikk-consume

Both exist side by side with different refs. The order's (vii) is confirmed on the descriptions: the two description: lines differ only in the bundle id, the concept count and the ref — there is no topic signal in either.

Routing measured with no skill named in the prompt, one run per question:

Question Belongs to Chosen Ref in the answer
"hva kreves ved prøvedrift?" teknikk teknikk e53bed17f7b8a5e2…
"hva er påslagsprosenten for materialer?" pris pris 477ed72de1d3be18…

2 of 2. The first run's output names both bundles before settling on teknikk, so the choice was made rather than defaulted to. No topic signal was added to the generator, because the measurement did not call for one: the bundle id carried enough. That is a property of ids chosen meaningfully, not of the description line, and § 8 says so.

8 Honesty limits

  • N = 1 per row in § 6 and § 7. Four claude -p runs total, no re-runs. A single run is not a rate.
  • Routing is 2 questions over 2 bundles, on one corpus split in two. It was not measured for more than two bundles, for topically overlapping bundles, or for ids that carry no topic (alfa/beta). The mechanism that worked was the id, and a caller who passes --id a and --id b has removed it. The description line still carries no topic signal.
  • Run 2 overstated one negative. It wrote "NS 8407-bestemmelsene er ikke i bundelen". grep -ril '8407' .okf/dokumenter --include='*.md' finds 3 of 52 files (control: pris matches 21 of 52, so the query can find). All three are in withheld concepts, and the answer did name dokument-for-avtaleinngåelse/tabell-linje-30 as the one withheld concept that might carry contract terms — so the coverage limitation was reported and the sentence above it was still stronger than what was measured.
  • Run 3 changed the instrument mid-task and said so. It re-ran the pre-pass with --k 26 and --cost-vocabulary after the default k=8 delivered 8 of 26, and declared both cuts in the file. The document's numbers are therefore for a non-default payload; the § 6 default figures are the ones for the shipped configuration.
  • The install in § 5 came from a local export, not a git URL. The tag is unpushed by instruction, so the channel differs from the README's line even though the bytes do not.
  • No pptx was in any measured folder. The pptx fasit stands where round 7 left it.
  • The K2 reference bundle was not rebuilt. This session changed no segmentation rule and no proposer default; the cli.build signature change affects only callers who used the Python API without passing the two flags, of which project.create was the one in this repository. tests/test_default_bundle_pin.py is untouched and green.