llm-ingestion-okf/tests
Kjell Tore Guttormsen 56ae274246 fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index
Two producer-side findings from the consumer's S7c acid test (ordre 20260908T063454Z-3648220855-from-.claude), both measured on K2 before and after, both with the corpus rebuilt from scratch.

FUNN 3 -- THE FORM. The converter's default markdown writer emits simple tables, which pad every cell out to the width of the widest cell in its column. Measured on the tender's price sheet: one 594-character prose cell produced a 67 244-character whitespace carpet with runs of up to 887 characters between a label and its amount, 19 integral amounts carrying a converter `.0`, and a header row naming one column. The bytes reached a live model in 2 of 11 prompts and 0 of 11 answers. The spreadsheet row now writes pipe tables with `--columns=1` (load-bearing: the pipe writer pads to a width computed from it, so at the default 72 a narrow table gains runs of up to 45). Same sheet after: 11 048 characters, longest run 2, one row per line, 0 artificial `.0`. Spreadsheet-only, and the scoping is pinned by three digests -- the same change moves the odt fixture 1366 -> 1105, so it can fail.

The `.0` rewrite is bounded twice: to a cell whose whole content is such a number, anchored between unescaped pipes, and skipped when the literal is in the workbook's shared string table -- the converter renders the number 92 and the TEXT "92.0" identically, so the output alone cannot tell them apart. Read with zipfile and xml.etree; no new dependency.

FUNN 2 -- THE LOG LINK. `link_log_in_root_index` (95eb271) is removed. Consumption contract SS 9.2 forbids a consumer from enumerating the bundle directory unless the profile says the index is derived, so the index tree is the entire map a consumer may use and everything it links is a document: their navigator returned 630 where our pre-pass counts 629, and a corpus run's own log was citable as content. The log is still written to the bundle root (SPEC section 9); `tools/okf_consume.py` keeps its exclusion for the bundles already built with the link.

K2 rebuilt twice. BEFORE reproduces the consumer's ref exactly (`sha256-tree:f14872a0...c8a92a`, 629 concepts) and their three consume figures to the token (57 289 / 62 149 / 58 401). AFTER: 629 concepts, `merged + coded rejections = 43 = N`, new ref `sha256-tree:c26eed6a...e3261f`, 627 of 629 concepts byte-identical, 1104 of 1108 files identical to the delivered bundle.

ONE REGRESSION, MEASURED AND NOT FIXED: on the mandate-shaped question with the vocabulary bridge the priced concept moves from candidate rank 10 to 19, so `--k 12` withholds it `below_k`; `--cost-vocabulary --k 20` delivers it at 65 912 o200k. The cause is measured rather than argued -- restoring only the concept's title on the new short body ranks it 10 again. The chain ends at the orphan check (`propose.py:461`), which drops the sheet heading once a table block opens two lines below it. That is the already-reported orphan gate, and changing it is a default-ON segmentation rule affecting every document type. The specific question is unaffected: rank 1 before and after. The priced excerpt's budget share falls from 56.5 % to 9.7 %.

11 new tests (RED first), 8 mutations, 8 red, with an unmutated control green each time. One mutation survived twice before the fixture could make it fire, and both survivals are written down. 1279 -> 1287 tests. mypy --strict clean on 28 files. ruff clean. Both proposer goldens byte-unchanged. One frozen literal moved with the fix and is reported rather than hidden.

Report: docs/2026-09-08-prisform-og-loggen-k2.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-08 10:06:58 +02:00
..
fixtures fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index 2026-09-08 10:06:58 +02:00
test_adjudicate.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
test_attested_computation.py fix(frontmatter): a nested key must not substitute for a top-level one 2026-08-31 23:31:53 +02:00
test_cid_measure.py feat(tools): a re-measurable CID-glyph-share instrument for K2 2026-09-04 20:23:34 +02:00
test_cli_build.py fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index 2026-09-08 10:06:58 +02:00
test_contract_check.py feat(tools): okf_contract_check reads the contract's checkable half 2026-09-02 16:09:27 +02:00
test_corpus_run.py fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index 2026-09-08 10:06:58 +02:00
test_docs_promises.py feat(inbox): walk the drop directory recursively 2026-09-07 04:11:00 +02:00
test_error_codes.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_exception_chaining.py test(errors): pin __cause__ preservation on every fail-fast wrap 2026-07-23 07:02:47 +02:00
test_extract.py fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index 2026-09-08 10:06:58 +02:00
test_faceted_index.py feat(profiles): a faceted index policy and the additive STRUCTURED_V1 profile 2026-08-27 00:30:42 +02:00
test_fidelity.py test(fidelity): K2 on pptx, odt and rtf -- 59/59 and 20/20, on a set we wrote 2026-09-07 05:22:12 +02:00
test_file_connector.py feat(connectors): add the file connector with fail-closed path boundary 2026-07-16 19:55:08 +02:00
test_frontmatter_nesting.py fix(frontmatter): a nested key must not substitute for a top-level one 2026-08-31 23:31:53 +02:00
test_golden.py feat(okf-v0.2): D5 — the v0.2 golden fixture, with okf_version in root frontmatter 2026-07-31 17:27:48 +02:00
test_guard_adapter.py build(deps): pin llm-ingestion-guard v1.3.0 so the gate reads our own goldens 2026-09-03 20:41:03 +02:00
test_http_connector.py feat(connectors): add the http connector and wire the network gate 2026-07-16 20:06:21 +02:00
test_import_facets.py docs(plan): record the Door C facet decisions, the mutation result, and the harness that lied 2026-08-27 11:03:37 +02:00
test_import_flow.py feat(import): Door C flow against an injected import gate (Phase 2 step 5) 2026-07-25 06:57:25 +02:00
test_inbox.py feat(inbox): fail-fast on filenames over the 255-byte limit 2026-07-25 06:27:43 +02:00
test_inbox_flow.py feat(inbox): walk the drop directory recursively 2026-09-07 04:11:00 +02:00
test_inbox_recursion.py feat(inbox): walk the drop directory recursively 2026-09-07 04:11:00 +02:00
test_inbox_structure.py feat(inbox): Door B derives structure and reprojects the index additively 2026-08-27 00:37:12 +02:00
test_index.py feat(index): maintain existing indexes on re-materialization (spec §6) 2026-07-16 20:04:21 +02:00
test_index_policy.py fix(profiles): P1-F1 — a permitted root key is not a required one 2026-07-31 18:16:13 +02:00
test_index_sort.py test(index): pin Door C's cross-run ordering bound 2026-09-01 19:58:19 +02:00
test_k2_office_fixtures.py test(fixtures): a synthetic K2 denominator for pptx, odt and rtf 2026-09-07 05:17:18 +02:00
test_load_bearing.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_manifest.py feat(manifest): reject [/] in extraction title (ingest-spec §4) 2026-07-23 21:33:47 +02:00
test_materialize.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_multi_source_provenance.py feat(materialize): sources takes a list and renders N flow mappings 2026-09-02 16:12:19 +02:00
test_okf_consume.py fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index 2026-09-08 10:06:58 +02:00
test_okf_v0_2_characterization.py fix(frontmatter): a nested key must not substitute for a top-level one 2026-08-31 23:31:53 +02:00
test_okf_v0_2_profile.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_okf_watch.py feat(watch): weekly OKF upstream watch that can prove it found nothing 2026-08-23 20:38:37 +02:00
test_outline_measure.py feat(cli): okf build, one installed command for folder in, bundle out 2026-09-07 05:06:33 +02:00
test_packaging.py build(extract): vendor the converter binary as a pinned wheel 2026-09-02 14:03:23 +02:00
test_pandoc_binary.py feat(extract): resolve the vendored converter and refuse an unpinned version 2026-09-02 14:07:20 +02:00
test_profile.py feat(profiles): a profile may name a per-suffix renderer 2026-09-02 14:19:18 +02:00
test_profile_threading.py test(profiles): pin the re-run promise V1 was published under 2026-08-09 12:39:23 +02:00
test_propose_segments.py feat(propose): --table-grid selects Arm E, absent is off [skip-docs] 2026-09-07 10:53:59 +02:00
test_render.py feat(render): add §5 body renderers as pure functions 2026-07-16 19:52:21 +02:00
test_render_hook.py feat(profiles): a profile may name a per-suffix renderer 2026-09-02 14:19:18 +02:00
test_root_frontmatter_emission.py feat(okf-v0.2): D5 — the v0.2 golden fixture, with okf_version in root frontmatter 2026-07-31 17:27:48 +02:00
test_segmentation.py feat(segmentation): parse the adjudication state a plan already carries 2026-09-02 14:44:06 +02:00
test_segmented_collisions.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_segmented_golden.py feat(examples): SEGMENTED_V1 golden fixture and decision record 2026-09-01 00:31:15 +02:00
test_segmented_identity.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_segmented_inbox.py feat(inbox): accept one segmentation plan per document 2026-09-02 14:51:53 +02:00
test_segmented_index.py feat(inbox): surface adjudication state and its dwell time 2026-09-02 14:49:45 +02:00
test_segmented_okf_v0_2_golden.py test(examples): byte-pinned golden for SEGMENTED_OKF_V0_2 2026-09-02 14:24:05 +02:00
test_segmented_profile.py feat(profiles): SEGMENTED_OKF_V0_2 so a segmented bundle can declare its spec 2026-09-02 14:21:15 +02:00
test_segmented_rebuild.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_segmented_rounds.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_sql_connector.py feat(connectors): add the sql connector (read-only sqlite, env-resolved) 2026-07-16 19:56:41 +02:00
test_strict_v1.py feat(phase-3): the index policy becomes configurable, with the reader that judges it 2026-07-25 20:30:54 +02:00
test_structure.py fix(structure): a derived reference needs a cue, not just a number shape 2026-08-29 09:26:57 +02:00
test_table_measure.py feat(tools): a re-measurable grid-table reach instrument for K3 [skip-docs] 2026-09-07 10:56:12 +02:00