llm-ingestion-okf/tools
Kjell Tore Guttormsen 6776c37d23 feat(consume): measure the budget lock, add one flag-gated top-rank reservation
The prior measurement (docs/2026-09-08-blindsone-below-k-k2.md SS 3) found that
the budget, not the ranking, is the second lock on a mandate-shaped cost
question -- and that the same mechanism was a REGRESSION on the question that
works: raising `--k` to 16 evicted the gold concept, because the exact knapsack
maximises a SUM of fused scores and has no opinion about rank, so twenty small
excerpts out-value one that costs 56.5 % of the budget.

Measured here on the same 629-concept bundle, with the three known-positive
figures from `4c699fd` reproduced first:

- Corpus distribution, denominator 629: median excerpt 857 B, max 223 391 B,
  3 concepts over the limit alone.
- Candidate rule (b), a corpus-derived budget, is FALSIFIED by two numbers: two
  defensible derivations are 49x apart on the same corpus, the small one turns
  the gold concept into `over_budget_alone` (13 refusals against 2), the large
  one changes nothing at the default k. A budget is the consumer's constraint,
  not a property of the corpus; `--limit` already belongs to the caller.
- Built instead, behind `--reserve-top-rank` (default OFF): the top-ranked
  candidate gets its bytes before the pack runs, AFTER the `over_budget_alone`
  pre-exclusion and never before, and the payload declares `budget.reserved`.
- It fixes the eviction: k=16 and k=24 deliver the gold concept at rank 1,
  costing one and two excerpts, and 20.4 % / 27.3 % FEWER o200k tokens.
- It changes the delivered list in 2 of 24 measured combinations -- both of them
  that eviction. In the other 22 the list, its order and `spent` are identical.
- It does NOT close the mandate-shaped blind spot: that concept ranks 10, not 1.
  The one delivering command is `--cost-vocabulary --k 12 --limit 160000`
  (62 149 tokens against 58 401), and that is a consumer's decision.

11 new tests (RED first), 7 mutations 7 red with an unmutated negative control
green before and after; two of the seven survived the first test set and the
tests were strengthened. Default payload byte-identical, both goldens unchanged.

Report: docs/2026-09-08-blindsone-laas2-budsjett-k2.md
Suite 1279 green, mypy --strict clean over 28 files, ruff clean.

Co-Authored-By: Claude <claude-opus-5>
2026-09-08 05:18:30 +02:00
..
okf_adjudicate.py fix(tools): do not blame the plan for a verdict okf_adjudicate built 2026-09-05 07:35:14 +02:00
okf_cid_measure.py feat(tools): a re-measurable CID-glyph-share instrument for K2 2026-09-04 20:23:34 +02:00
okf_consume.py feat(consume): measure the budget lock, add one flag-gated top-rank reservation 2026-09-08 05:18:30 +02:00
okf_consume_measure.py test(consume): hit@8 over six questions against a random-ranker baseline 2026-09-07 09:37:14 +02:00
okf_contract_check.py feat(tools): okf_contract_check reads the contract's checkable half 2026-09-02 16:09:27 +02:00
okf_corpus_run.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
okf_fidelity.py test(fidelity): K2 on pptx, odt and rtf -- 59/59 and 20/20, on a set we wrote 2026-09-07 05:22:12 +02:00
okf_outline_measure.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
okf_propose_segments.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
okf_table_measure.py feat(tools): a re-measurable grid-table reach instrument for K3 [skip-docs] 2026-09-07 10:56:12 +02:00
okf_watch.py feat(watch): weekly OKF upstream watch that can prove it found nothing 2026-08-23 20:38:37 +02:00