open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity
llm-ingestion-okf/tests
History
Kjell Tore Guttormsen 4c699fdbb1 feat(consume): measure the below-k blind spot, add one flag-gated vocabulary bridge 
The consumer report (portfolio-optimiser, S7 SS 2) found that a mandate-shaped
cost question withheld the corpus's one priced table under `below_k`. Measured
here, on a bundle proven byte-identical to a fresh HEAD rebuild:

- The mechanism is a VOCABULARY gap, not a `k` defect: two of three ranking
  signals are exactly 0.0 and the concept is candidate 249 of 269.
- The k-sweep buys nothing: k in {8,12,16,24,32,64,128} all withhold it, at
  +9.5 % tokens. It also found a regression -- for the question that WORKS,
  k >= 16 EVICTS the gold concept, because one 67 838 B excerpt is 56.5 % of
  the budget and the knapsack maximises a sum.
- Two proposed rules were falsified BEFORE any code: number/table density ranks
  the priced table 178/165/46 of 269 (the form is unfilled, so it is
  number-poor), and per-document spread puts its document 30th of 35.

Built instead, behind `--cost-vocabulary` (default OFF, DEFAULT byte-identical):
one declared cost/price/quantity vocabulary family that bridges a question and a
document naming money with different words. It moves the concept from candidate
rank 249 to 10 -- and does NOT close the blind spot: the budget still refuses
it, which is now a separately measured second lock.

Seven RED tests first; six mutations of the rule, six red (two survived the
first version of the tests and the tests were strengthened). Control: a question
with no cost term produces a byte-identical payload with the flag on, at every
k, on the real corpus. Known-positive: 164 987 B / 40 425 o200k tokens, equal to
the published pair.

Report: docs/2026-09-08-blindsone-below-k-k2.md
Suite 1268 green, mypy --strict clean over 28 files, both goldens unchanged.

Co-Authored-By: Claude <claude-opus-5>
2026-09-08 04:55:45 +02:00
..
fixtures test(propose): pin today's grid-table split with a golden that can fire 2026-09-07 10:45:05 +02:00
test_adjudicate.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
test_attested_computation.py fix(frontmatter): a nested key must not substitute for a top-level one 2026-08-31 23:31:53 +02:00
test_cid_measure.py feat(tools): a re-measurable CID-glyph-share instrument for K2 2026-09-04 20:23:34 +02:00
test_cli_build.py fix(build,consume): stamp every segmented concept, exclude a linked log.md from concept navigation 2026-09-08 03:21:36 +02:00
test_contract_check.py feat(tools): okf_contract_check reads the contract's checkable half 2026-09-02 16:09:27 +02:00
test_corpus_run.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
test_docs_promises.py feat(inbox): walk the drop directory recursively 2026-09-07 04:11:00 +02:00
test_error_codes.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_exception_chaining.py test(errors): pin __cause__ preservation on every fail-fast wrap 2026-07-23 07:02:47 +02:00
test_extract.py test(extract): hand-built office fixtures with frozen extracted text 2026-09-02 14:14:27 +02:00
test_faceted_index.py feat(profiles): a faceted index policy and the additive STRUCTURED_V1 profile 2026-08-27 00:30:42 +02:00
test_fidelity.py test(fidelity): K2 on pptx, odt and rtf -- 59/59 and 20/20, on a set we wrote 2026-09-07 05:22:12 +02:00
test_file_connector.py feat(connectors): add the file connector with fail-closed path boundary 2026-07-16 19:55:08 +02:00
test_frontmatter_nesting.py fix(frontmatter): a nested key must not substitute for a top-level one 2026-08-31 23:31:53 +02:00
test_golden.py feat(okf-v0.2): D5 — the v0.2 golden fixture, with okf_version in root frontmatter 2026-07-31 17:27:48 +02:00
test_guard_adapter.py build(deps): pin llm-ingestion-guard v1.3.0 so the gate reads our own goldens 2026-09-03 20:41:03 +02:00
test_http_connector.py feat(connectors): add the http connector and wire the network gate 2026-07-16 20:06:21 +02:00
test_import_facets.py docs(plan): record the Door C facet decisions, the mutation result, and the harness that lied 2026-08-27 11:03:37 +02:00
test_import_flow.py feat(import): Door C flow against an injected import gate (Phase 2 step 5) 2026-07-25 06:57:25 +02:00
test_inbox.py feat(inbox): fail-fast on filenames over the 255-byte limit 2026-07-25 06:27:43 +02:00
test_inbox_flow.py feat(inbox): walk the drop directory recursively 2026-09-07 04:11:00 +02:00
test_inbox_recursion.py feat(inbox): walk the drop directory recursively 2026-09-07 04:11:00 +02:00
test_inbox_structure.py feat(inbox): Door B derives structure and reprojects the index additively 2026-08-27 00:37:12 +02:00
test_index.py feat(index): maintain existing indexes on re-materialization (spec §6) 2026-07-16 20:04:21 +02:00
test_index_policy.py fix(profiles): P1-F1 — a permitted root key is not a required one 2026-07-31 18:16:13 +02:00
test_index_sort.py test(index): pin Door C's cross-run ordering bound 2026-09-01 19:58:19 +02:00
test_k2_office_fixtures.py test(fixtures): a synthetic K2 denominator for pptx, odt and rtf 2026-09-07 05:17:18 +02:00
test_load_bearing.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_manifest.py feat(manifest): reject [/] in extraction title (ingest-spec §4) 2026-07-23 21:33:47 +02:00
test_materialize.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_multi_source_provenance.py feat(materialize): sources takes a list and renders N flow mappings 2026-09-02 16:12:19 +02:00
test_okf_consume.py feat(consume): measure the below-k blind spot, add one flag-gated vocabulary bridge 2026-09-08 04:55:45 +02:00
test_okf_v0_2_characterization.py fix(frontmatter): a nested key must not substitute for a top-level one 2026-08-31 23:31:53 +02:00
test_okf_v0_2_profile.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_okf_watch.py feat(watch): weekly OKF upstream watch that can prove it found nothing 2026-08-23 20:38:37 +02:00
test_outline_measure.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
test_packaging.py build(extract): vendor the converter binary as a pinned wheel 2026-09-02 14:03:23 +02:00
test_pandoc_binary.py feat(extract): resolve the vendored converter and refuse an unpinned version 2026-09-02 14:07:20 +02:00
test_profile.py feat(profiles): a profile may name a per-suffix renderer 2026-09-02 14:19:18 +02:00
test_profile_threading.py test(profiles): pin the re-run promise V1 was published under 2026-08-09 12:39:23 +02:00
test_propose_segments.py feat(propose): --table-grid selects Arm E, absent is off [skip-docs] 2026-09-07 10:53:59 +02:00
test_render.py feat(render): add §5 body renderers as pure functions 2026-07-16 19:52:21 +02:00
test_render_hook.py feat(profiles): a profile may name a per-suffix renderer 2026-09-02 14:19:18 +02:00
test_root_frontmatter_emission.py feat(okf-v0.2): D5 — the v0.2 golden fixture, with okf_version in root frontmatter 2026-07-31 17:27:48 +02:00
test_segmentation.py feat(segmentation): parse the adjudication state a plan already carries 2026-09-02 14:44:06 +02:00
test_segmented_collisions.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_segmented_golden.py feat(examples): SEGMENTED_V1 golden fixture and decision record 2026-09-01 00:31:15 +02:00
test_segmented_identity.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_segmented_inbox.py feat(inbox): accept one segmentation plan per document 2026-09-02 14:51:53 +02:00
test_segmented_index.py feat(inbox): surface adjudication state and its dwell time 2026-09-02 14:49:45 +02:00
test_segmented_okf_v0_2_golden.py test(examples): byte-pinned golden for SEGMENTED_OKF_V0_2 2026-09-02 14:24:05 +02:00
test_segmented_profile.py feat(profiles): SEGMENTED_OKF_V0_2 so a segmented bundle can declare its spec 2026-09-02 14:21:15 +02:00
test_segmented_rebuild.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_segmented_rounds.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_sql_connector.py feat(connectors): add the sql connector (read-only sqlite, env-resolved) 2026-07-16 19:56:41 +02:00
test_strict_v1.py feat(phase-3): the index policy becomes configurable, with the reader that judges it 2026-07-25 20:30:54 +02:00
test_structure.py fix(structure): a derived reference needs a cue, not just a number shape 2026-08-29 09:26:57 +02:00
test_table_measure.py feat(tools): a re-measurable grid-table reach instrument for K3 [skip-docs] 2026-09-07 10:56:12 +02:00