feat(consume): measure the below-k blind spot, add one flag-gated vocabulary bridge

The consumer report (portfolio-optimiser, S7 SS 2) found that a mandate-shaped
cost question withheld the corpus's one priced table under `below_k`. Measured
here, on a bundle proven byte-identical to a fresh HEAD rebuild:

- The mechanism is a VOCABULARY gap, not a `k` defect: two of three ranking
  signals are exactly 0.0 and the concept is candidate 249 of 269.
- The k-sweep buys nothing: k in {8,12,16,24,32,64,128} all withhold it, at
  +9.5 % tokens. It also found a regression -- for the question that WORKS,
  k >= 16 EVICTS the gold concept, because one 67 838 B excerpt is 56.5 % of
  the budget and the knapsack maximises a sum.
- Two proposed rules were falsified BEFORE any code: number/table density ranks
  the priced table 178/165/46 of 269 (the form is unfilled, so it is
  number-poor), and per-document spread puts its document 30th of 35.

Built instead, behind `--cost-vocabulary` (default OFF, DEFAULT byte-identical):
one declared cost/price/quantity vocabulary family that bridges a question and a
document naming money with different words. It moves the concept from candidate
rank 249 to 10 -- and does NOT close the blind spot: the budget still refuses
it, which is now a separately measured second lock.

Seven RED tests first; six mutations of the rule, six red (two survived the
first version of the tests and the tests were strengthened). Control: a question
with no cost term produces a byte-identical payload with the flag on, at every
k, on the real corpus. Known-positive: 164 987 B / 40 425 o200k tokens, equal to
the published pair.

Report: docs/2026-09-08-blindsone-below-k-k2.md
Suite 1268 green, mypy --strict clean over 28 files, both goldens unchanged.

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 04:55:45 +02:00
commit 4c699fdbb1
5 changed files with 616 additions and 12 deletions

View file

@ -266,7 +266,17 @@ and fixtures, never code.
its bytes do not have. Three exit codes: 0 written, 1 refused, 2 did not run.
The first instantiated consumption skill is `skills/okf-consume/`; the
measurement behind it, including the control that FAILED, is
`docs/2026-09-07-okf-konsumskill-maaling.md`.
`docs/2026-09-07-okf-konsumskill-maaling.md`. **The ranking is this
repository's own choice** — the contract binds a payload, not a retrieval
algorithm (§ 10) — and it has ONE optional widening, `--cost-vocabulary`,
**off by default**: a declared cost/price/quantity vocabulary family that
bridges a question and a document naming money with different words, gated on
the QUESTION carrying such a term, so a question without one is byte-identical
either way. It moves a measured case from candidate rank 249 to 10 and does
**not** deliver it: the budget is a second, independent lock, and closing that
one is a decision nobody has made. Measured, with the two rules falsified
before building and the `k`-sweep that showed a higher `k` can EVICT a gold
concept, in `docs/2026-09-08-blindsone-below-k-k2.md`.
## Workflow