open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

chore(tools): remove okf_adjudicate -- nothing here starts Claude Code

Browse source 
Operator rule, 2026-09-20, for every repository: our code does not start Claude
Code. `tools/okf_adjudicate.py` did, with `subprocess.run` on a hardcoded
`/Users/.../claude` -- a path to one binary on one machine, in a public
repository.

Removed with its test file. References: **17 -> 3**, all three in the
CHANGELOG, and that is a deliberate deviation stated in the removal entry
itself: two of them describe what the tool did under the versions that shipped
it, and a changelog that edits its own past is not a record. The three K3/K4/K5
reports that used it now say so in the past tense, with the removal noted where
the tool was named.

It is the only such place, measured rather than assumed: `subprocess.*claude|
CLAUDE_CLI|claude", *"-` over `src` and `tools` gives **0** here and **3** on
`df5a118`. Every other subprocess in `tools/` starts `git`, `pytest` through
`sys.executable`, a PDF utility, or the coordination script `okf_watch.py`
owns -- none of them a model.

`okf build --gate` and the guard are untouched; nothing that judges CONTENT
moved. What this removes is a tool that asked a model to judge a proposal, and
the only measurement that ever rested on it is a 2026-09-02 baseline, which is
reported and dated.

2318 passed, 2 skipped (2334 before, minus this tool's 18 tests, plus 2 new).
`ruff check` and `ruff format --check` clean over 159 files, `mypy --strict
src/` clean over 25. Wheel built and installed into a clean venv: `okf --help`
exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Correction to `23ef6bb`: its `9 -> 6` is true of the 74 files the span touched
when it was measured, and the span has grown since -- re-measured over the
finished span's 79 files, the same pattern is **11 -> 7**, the one extra pair
being `tools/okf_adjudicate.py` (gone with this commit) and
`tests/test_okf_skill.py`'s own known-positive for the absolute-path check.
This commit is contained in:
Kjell Tore Guttormsen 2026-09-20 15:40:52 +02:00
commit 16ec574687
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
2 changed files with 9 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

9
docs/2026-09-02-k3-k4-k5-baseline.md
View file

@ -53,7 +53,8 @@ independence.
## K4a — determinism, binary
**PASS.** One document was adjudicated with `tools/okf_adjudicate.py`, then run
**PASS.** One document was adjudicated with the adjudication tool this
repository carried at the time (the tool was removed 2026-09-20: nothing in this repository starts another program to judge anything), then run
through the path twice into two bundles, and the two compared:
```
@ -82,9 +83,9 @@ seconds per document, and nothing here measures that.
### A finding the method did not anticipate
**4 of 12 verdicts could not be recorded at all.** `tools/okf_adjudicate.py`
parses the proposal before writing, and the parser refuses a plan with zero
entries — correctly, because an empty plan replayed on the run path would
**4 of 12 verdicts could not be recorded at all.** The adjudication tool
(the tool was removed 2026-09-20: nothing in this repository starts another program to judge anything) parsed the proposal before
writing, and its parser refused a plan with zero entries — correctly, because an empty plan replayed on the run path would
persist nothing for a document that was dropped. The consequence is that the
one judgement most worth recording about those four documents — *the proposal
is empty, and that is wrong* — has no artifact to live in. The K3 counts above

7
docs/2026-09-02-k3-k4-k5-metode.md
View file

@ -76,9 +76,10 @@ human one**, and every figure resting on it inherits that: this is a baseline
produced by a machine reading proposals, not a human annotation study. Nothing
here should be read as a human adjudication rate.
The judgement is recorded against the proposal with `tools/okf_adjudicate.py`,
which leaves the proposal byte-untouched and writes the verdict as a sibling.
Its advisory model leg stays **off**: pre-annotation has been measured lowering
The judgement was recorded against the proposal with the adjudication tool
this repository carried at the time (the tool was removed 2026-09-20: nothing in this repository starts another program to judge anything),
which left the proposal byte-untouched and wrote the verdict as a sibling.
Its advisory model leg stayed **off**: pre-annotation has been measured lowering
a good annotator's accuracy, and a leg that cannot be switched off is a leg
whose value can never be measured.