llm-ingestion-okf/docs/2026-09-08-blindsone-laas2-budsjett-k2.md

The second lock: budget reserved for the top-ranked candidate, measured on a 629-concept corpus

Date: 2026-09-08 · Order: 20260908T025839Z-9903024761-from-.claude · Instrument: tools/okf_consume.py at 4c699fd plus the one flag this document reports · Prior finding: docs/2026-09-08-blindsone-below-k-k2.md § 3, § 6, § 7 and § 9.

The document this one continues measured a mandate-shaped cost question against a 629-concept corpus and found two independent locks on the one concept in that corpus carrying a price. Lock 1 is vocabulary, and --cost-vocabulary moves the concept from candidate rank 249 of 269 to 10 of 278. Lock 2 is the budget: one excerpt costs 67 838 B, 56.5 % of the 120 000 B budget, and the knapsack maximises a SUM, so it prefers many small excerpts. The same sweep found a regression on the question that WORKS: raising k to 16 evicts the gold concept, which makes k a dial that can remove the one document a question was asked about.

This document measures lock 2, reports the one flag-gated rule built after the measurement, and prices the two rules the measurement did not justify.

The corpus is external and private to the measurement; no document name, path or body from it appears here. Documents are named by shape ("the priced table") and the numbers are counts.

---

0. What IS measured, and what is NOT

Measured. The excerpt-weight distribution of the whole corpus with its denominator; today's cut strategy read out of the code rather than summarised; the one rule that was built, on three questions (mandate-shaped, specific, control), both settings of --cost-vocabulary, and k in {8, 12, 16, 24} -- 24 combinations, each with delivered count, delivered list, budget spent, payload bytes and o200k tokens; the exact excerpts the reservation displaced; and the two candidate rules the order named, priced against the same corpus.

Known-positive, run first. Three figures from 4c699fd, re-measured on this tree before any change:

#	case	expected	measured
1	specific question, k=8, both flags off	164 987 B / 40 425 o200k tok	164 987 B / 40 425
2	specific question, k=16, flags off	priced table over_budget_after_knapsack	over_budget_after_knapsack
3	mandate question, --cost-vocabulary, k=12	priced table over_budget_after_knapsack	over_budget_after_knapsack

The first pair is the figure published 2026-09-07 by the consumer (portfolio-optimiser docs/2026-09-07-okf-prepass-i-debatten.md § 1), byte for byte and token for token; the tokenizer is tiktoken o200k_base. An instrument that has not reproduced a known figure has not been shown to count (consumption contract § 7.4).

NOT measured. That the rule helps any corpus other than this one: one corpus, three questions and 24 combinations is not a sample. Not measured either: whether a model ANSWERS better with the top-ranked excerpt present -- that needs a live model and is the consumer's measurement, in their S7 form, not this one. And not measured: that DEFAULT_LIMIT = 120 000 is the right budget. This document recommends; the default is the operator's decision and is unchanged here.

Nothing is decided about the default. The flag ships OFF. With it off every payload this repository emits is byte-identical to 4c699fd, the suite's byte-identity test covers exactly that, and both golden fixtures are unchanged.

---

1. Setup

The bundle is the 629-concept build of the corpus produced by okf build on 5a0c879, identity sha256-tree:f14872a01104e47474093611b1960c6c541e4701dc40147a00c8e1b337c8a92a, the same tree 4c699fd measured. It is still a HEAD build: 4c699fd touched tools/, tests/, docs/, README.md and CLAUDE.md and no file under src/, so no code on the build path moved between the two commits. Every command in this document is offline: no model call, no socket, no clock.

---

2. What decides today, verbatim

From tools/okf_consume.py, quoted rather than summarised:

• DEFAULT_LIMIT = 120_000 — "Chosen, not derived, and the reason is a measurement rather than a taste: at 60 000 the largest realistic gold concept (101 313 B encoded) falls to the over_budget_alone pre-exclusion, so a CORRECT implementation would fail its own acceptance criteria ... A starting point to be moved by measurement."
• knapsack — "The exact 0/1 knapsack: indices of the highest-value subset that fits. Exact rather than greedy-by-density, which has an unbounded approximation factor."
• cut — "Exclusions run before the pack, each naming its rule, because 'it did not fit' and 'it could never be delivered' are different facts about the cut."
• WEIGHT_BUCKET = 500 — "bucketing UP the item and DOWN the capacity keeps the error one-sided, so the pack may under-deliver by a bucket and can never over-spend."
• The two budget rules in the closed set: over_budget_alone (a pre-exclusion, before the DP) and over_budget_after_knapsack (a drop by the pack itself).

And the contract, docs/consumption-contract.md § 7.6: a skill "SHOULD state what a typical answer costs against the whole-bundle denominator, and the corpus size at which its strategy stops fitting its budget. A strategy with no stated breaking point cannot be observed to have passed it."

In one line: the pack maximises a sum of fused scores under a byte budget, and a sum has no opinion about rank. A candidate ranked first is worth one summand. Twenty candidates ranked 2 to 21 are worth twenty. Nothing in the DP prefers the first, and nothing in it should -- the defect is that the cut asks the DP a question whose answer it then reads as an answer about relevance.

The corpus's own distribution, since the candidates argue about it

Over all 629 concepts (every one of them yields an excerpt; the denominator is 629, not a subset):

statistic	bytes
min	369
median	857
mean	3 131
p90	3 440
p99	53 970
max	223 391
exceed DEFAULT_LIMIT alone	3 of 629

The distribution is not merely skewed, it is bimodal in practice: a median of 857 B and a maximum of 223 391 B is a factor of 261.

---

3. The three candidate rules, priced

(b) a profile-sized budget, derived from the corpus — FALSIFIED, and by two numbers rather than one. Two derivations are equally defensible from the table above, and on the SAME corpus they are 49 times apart:

derivation	limit (B)	specific question, k=8
8 x median excerpt	6 856	delivered 6, gold concept over_budget_alone, 13 concepts refused outright (2 today)
largest excerpt x 1.5	335 086	delivered 8, spent 72 910 — identical delivered set to today

The small derivation breaks the known-positive: the question that works stops working, because the concept that answers it can no longer fit alone. The large one changes nothing at the default k while tripling a ceiling nobody was touching. A rule whose answer moves by a factor of 49 with the choice of statistic is not a rule, it is a parameter wearing a derivation.

There is a second, larger reason, and it is about ownership rather than arithmetic: a budget is the CONSUMER's constraint, not a property of the corpus. It exists because a context window is finite. A library that sizes the budget to the data hands a caller a payload their model may not hold, and does it silently, in the name of fitting a document the caller never asked to be sized around. --limit is already the caller's, and it is the right place for that decision.

(c) no rule — FALSIFIED for the specific question, and it is the reason this order exists. Doing nothing leaves the § 3 regression in place: at k=16 and k=24 the gold concept of the question that WORKS is evicted. "Keep k=8" is a mitigation that depends on nobody raising k, and --k is a public flag.

(a) the rule that was built: the top-ranked candidate is reserved its bytes before the pack runs, behind --reserve-top-rank, default off. The measurement in § 2 says the failure is that the DP has no opinion about rank. So: give rank one a floor rather than a bid, and let the pack fill what is left.

Three properties, each with a test that goes red without it:

• The reservation runs AFTER the over_budget_alone pre-exclusion, never before. A candidate that alone exceeds the budget is still refused by name, and the reservation then applies to the highest-ranked candidate that could be delivered at all. Reserving for an excerpt the gate can never admit would deliver bytes the gate refuses.
• It reserves the fused TOP, not the heaviest and not the first by id. The test uses a fixture whose heaviest excerpt is deliberately a lower-ranked one, because that is the only place the two come apart.
• The reservation is paid once. A reserved excerpt left in the pack's pool competes for the budget it has already been given and wins it back from the next candidate; the pool excludes it and the capacity is reduced by exactly its weight, so the one-sided bucket arithmetic still cannot over-spend.

The payload declares it. When a reservation is made, budget carries reserved: {concept_id, bytes} -- a cut whose strategy changed without saying so is the silent cut § 5.3 exists to forbid. The member is absent when the flag is off, which is what keeps the default payload byte-identical, and § 8 permits additional members, so a payload carrying it still passes the contract checker with zero findings.

Development order: eleven tests, the failing ones written first. Seven mutations of the shipped rule were run against them; all seven are red, with an unmutated negative control green before and after the run (reserve the last-ranked; reserve the heaviest; capacity not reduced; the reserved item left in the pool; default flipped on; the pre-exclusion bypassed; the declaration dropped). Two of the seven survived the first version of the tests -- the pool mutation and the pre-exclusion bypass -- and the tests were strengthened until they did not.

---

4. The rule, measured on three questions

Same bundle, same budget, --cost-vocabulary off and on, k swept. Only the rows where the delivered SET moves are given in full; the other 20 are summarised below them, which is the honest way round because the summary is the larger claim.

question	vocab	reserve	k	delivered	payload B	o200k tok	priced table
specific	off	off	16	15	194 946	65 237	over_budget_after_knapsack
specific	off	on	16	14	206 102	51 951	delivered, rank 1
specific	off	off	24	20	197 287	66 799	over_budget_after_knapsack
specific	off	on	24	18	191 409	48 545	delivered, rank 1

Of the 24 measured combinations (3 questions x 2 vocabulary settings x 4 k values), the reservation changes the delivered list in exactly 2 -- and both of them are the § 3 eviction. In the other 22 the delivered list is identical, in the same order, with spent equal to the byte; the payload differs only by the declaration the flag adds (+129 B / +46 o200k tok on the mandate question, +151 B / +51 on the control). The reason is not a coincidence: where the pack already delivered the top-ranked candidate, reserving it changes nothing to compute.

The control behaves exactly as the summary predicts. A question naming no cost term is unaffected by --cost-vocabulary at every k (byte-identical, the finding 4c699fd reported) and its delivered set is unaffected by the reservation at every k -- its top-ranked candidate, 23 900 B, was already being delivered. Unlike the vocabulary flag, this one is not gated on the question and does not need to be: it is gated on whether the pack was about to drop rank one.

---

5. What the rule costs, stated as a working question

The reservation is not free, and the cost is visible in the two rows that moved:

	k=16	k=24
delivered	15 -> 14	20 -> 18
reserved for rank 1	67 838 B	67 838 B
bought from	2 excerpts, 56 913 B	3 excerpts, 74 066 B
payload bytes	194 946 -> 206 102 (+5.7 %)	197 287 -> 191 409 (-3.0 %)
o200k tokens	65 237 -> 51 951 (-20.4 %)	66 799 -> 48 545 (-27.3 %)

Two things in that table deserve to be read rather than skimmed.

The excerpts it displaced left under a rule that already existed. They are over_budget_after_knapsack, the same rule that was removing the gold concept a moment earlier. The closed rule set did not grow, and the payload still partitions: every considered concept is delivered or withheld under a named rule.

Bytes and tokens disagree about the direction, and the gate counts bytes. The reserved excerpt is a spreadsheet render: 206 102 B costs 51 951 o200k tokens (4.0 B/token) where the 194 946 B it replaced cost 65 237 (3.0 B/token). So on this corpus the reservation delivered the gold concept and made the payload cheaper for the model while making it larger for the gate. That is a property of this document's shape, not a general one, and it is exactly why the budget unit is NAMED in the payload (§ 7.1) rather than assumed.

---

6. Does this close the blind spot? No -- and the honest line is one command

For the mandate-shaped question the reservation changes nothing. The priced table is candidate rank 10 with --cost-vocabulary on, not rank 1, so it is never the reserved candidate; the delivered set and spent are identical at every k measured, with both flags on.

Both locks open at once only with a raised budget, and that is a caller's decision, not a default. Measured:

python3 tools/okf_consume.py <bundle> \
  --question "<the mandate-shaped question>" \
  --cost-vocabulary --k 12 --limit 160000

delivers the priced table at rank 10, 12 excerpts, 150 249 B spent, 62 149 o200k tokens -- against 58 401 for the same run at the default limit that withholds it. Adding --reserve-top-rank changes neither the set nor the byte count, for the reason above. So the mandate question's answer costs +3 748 tokens (+6.4 %) and one flag plus one explicit limit; nothing in this repository decides that it should be paid.

And "delivered" is still not "answered". Whether a model does better with those bytes present is unmeasured here and is the consumer's measurement, in their S7 form, with a live model. This document moves a document from unaffordable to delivered under a stated command. That is not the same as an answer, and the distinction is the whole reason both are written down.

---

7. Honesty limits

• One corpus, three questions, 24 combinations. Generality is NOT demonstrated. The reservation's effect depends on there being a large-and-top-ranked excerpt at all; a corpus of uniform small concepts would never notice the flag.
• The rule was written with the failing case visible. The § 3 eviction is the case that motivated it, and the fixture that tests it is built to that shape. The fixture is synthetic and carries no corpus constant, but it is not a blind test.
• Two of the seven mutations survived the first test set. They were caught by re-running, not by design, and the two tests that now kill them were written afterwards. A reader should weight "seven of seven red" accordingly.
• The token direction is a property of one document. The −20 % and −27 % above come from replacing prose with a spreadsheet render. A corpus whose large concepts are prose would show the opposite sign, and the byte figure -- the one the gate enforces -- is up in one of the two rows.
• --reserve-top-rank has no consumer. Nobody asked for it. It exists so the § 3 regression could be measured against real code rather than argued about, and so a decision about the default can be made on numbers.

---

8. Recommendation

1. Ship --reserve-top-rank OFF by default, and keep DEFAULT_K = 8. At k=8 neither measured question needs the flag; the regression it fixes only appears when a caller raises k. What the flag buys is that raising k is no longer able to remove the top-ranked concept -- which is what a caller raising k was trying to avoid in the first place.
2. Do not derive the budget from the corpus. § 3 prices both derivations: one breaks the known-positive, the other changes nothing at the default k, and they are 49x apart. --limit already belongs to the caller.
3. Answer the contract's § 7.6 question with the number that is actually true for this corpus. The strategy does not stop fitting its budget at a corpus SIZE. It stops fitting when a single relevant concept costs a large share of the budget -- here 56.5 % -- and the answer is either a raised --limit or this flag, depending on whether the caller wants the concept or the count.
4. The blind spot for a mandate-shaped question stays open at the default, and it is now fully priced. Both locks are measured, both have a command that removes them, and the remaining decision -- whether a mandate-shaped question should cost 62 149 tokens instead of 58 401 -- is a consumer's, not this library's.

---

9. Verification log

#	Claim	Command → result
1	The instrument reproduces a published figure	specific question, k=8, flags off → 164 987 B / 40 425 o200k, equal to the published pair
2	The § 3 eviction is still there before the change	specific question, k=16, flags off → over_budget_after_knapsack
3	The § 6 mandate row is still there	mandate, --cost-vocabulary, k=12 → over_budget_after_knapsack
4	The corpus distribution	629 excerpts: median 857 B, max 223 391 B, 3 over the limit alone
5	Candidate (b), small derivation	limit 6 856 → gold concept over_budget_alone, 13 refusals against 2
6	Candidate (b), large derivation	limit 335 086 → delivered set identical to today at k=8
7	The rule fixes the regression	specific question, k=16 and k=24, --reserve-top-rank → delivered, rank 1
8	The rule's cost	k=16: 15 → 14 excerpts, 2 displaced (56 913 B), tokens −20.4 %
9	It changes nothing elsewhere	22 of 24 combinations: identical delivered list, identical spent
10	It does not deliver the mandate's priced table	every k in {8, 12, 16, 24}, both flags → withheld
11	Both locks, one command	--cost-vocabulary --k 12 --limit 160000 → rank 10, 62 149 o200k tok
12	The default does not move	flag off ⇒ byte-identical payload; both goldens unchanged
13	Seven mutations, seven red	negative control green before and after the mutation run
14	The declaration stays conformant	payload with budget.reserved → contract checker 0 findings
15	Suite, types, lint	pytest -q 1279 passed; mypy --strict src/ tools/ 28 files clean; ruff check + ruff format --check clean