open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity
llm-ingestion-okf/tests
History
Kjell Tore Guttormsen cb0c10b421 fix(retrieval-gate): the judge reads a passage as its exact reconstruction — M15 felled, row 7 15 of 15 
Row 8's judge required every delivered text to be the whole concept's bytes
(the row-6 identity), which is older than v1.1 C1: an excerpt `as_passage` cut
to its answering passage was class e even when its span carried the citation.
PM's re-measurement of e503f6a found that the dominant miss class of row 8 on
the real set.

`passage_span` accepts a passage only when the delivered text IS what the
bundle's bytes rebuild: the span is body[start:end] byte for byte in the
DELIVERED body (the offsets land a frontmatter's length off in the file), `of`
is that body's length, `[...]` stands exactly where text is left out, and the
one other line allowed is a heading line of the body above the span, or a
prefix of one. The citation is read in the span alone. A passage that is not
its reconstruction is class e with its own detail. A quote only in the heading
or the markers is class d.

Chose to check that the heading is one of the body's own heading lines above
the span, and not to re-derive WHICH heading `as_passage` picks, because the
guarantee is "the bundle's bytes"; the nearest-heading rule is the product's
presentation and a judge that copies it agrees by construction.

M15 (a passage carrying one sentence the file does not) is row 7's fifteenth
mutant. No synthetic concept was long enough to be cut, so a new fixture
delivers one (DELIVERY, ~6 600 characters, set set-passage.json); the corpus
pin moved with it. MUTANT_BAR unchanged, src/ untouched.

Gate, synthetic: rows 1 and 6 13/13 -> 14/14, row 7 14/14 -> 15 of 15 GREEN
(M15 -> row 1, 6), rows 2/3/4 unchanged, GATE RED: rows 5, 8. Row 8 rerun on
the real set: no class e miss remains (its numbers are kept out of this public
history, per the wiki directive). Suite after git add,
FORCE_COLOR unset: 2450 passed, 1 skipped (+6). ruff, format, mypy --strict
clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-22 23:53:09 +02:00
..
fixtures fix(retrieval-gate): the judge reads a passage as its exact reconstruction — M15 felled, row 7 15 of 15 2026-09-22 23:53:09 +02:00
test_accounting_gate.py test(gates): retire the K2 track; re-measure the retrieval gate's premises for BM25 2026-09-21 10:05:43 +02:00
test_asset_bundle.py feat(assets): a bundle carries the images its sources declare (0.10.0) 2026-09-17 10:01:31 +02:00
test_asset_gate.py feat(assets): a bundle carries the images its sources declare (0.10.0) 2026-09-17 10:01:31 +02:00
test_asset_import.py feat(assets): a bundle carries the images its sources declare (0.10.0) 2026-09-17 10:01:31 +02:00
test_asset_limits.py fix(assets): budget every link by what its decoder COSTS (0.10.1) 2026-09-18 19:00:38 +02:00
test_asset_structure.py feat(assets): a bundle carries the images its sources declare (0.10.0) 2026-09-17 10:01:31 +02:00
test_asset_viewable.py test(assets): the RLE8 rule is short of the independent decoder, in 703 of 22 620 streams 2026-09-19 22:46:13 +02:00
test_assets.py test(mutants,assets): a mutant is judged by the suite that owns it, 45 of 45 2026-09-19 19:33:23 +02:00
test_attested_computation.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_block_sources_flat_readers.py test(gates): retire the K2 track; re-measure the retrieval gate's premises for BM25 2026-09-21 10:05:43 +02:00
test_bm25_ranking.py feat(consume): the payload says when the bundle looks like it does not cover the question 2026-09-21 07:12:43 +02:00
test_bold_title.py feat(propose,consume,tools): the type that declares nothing, and the prefix that is not a word 2026-09-09 23:05:45 +02:00
test_bundle_identity.py feat(check): the contract and the checker state the payload's v1.1 forms 2026-09-21 08:47:46 +02:00
test_bundle_map.py feat(mcp): the bundle's map, and a working method that reads it first 2026-09-21 08:37:12 +02:00
test_cid_measure.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_cli_build.py feat(cli,consume): the first span, and the fusion that punished fine-graining 2026-09-09 15:40:01 +02:00
test_cli_gate.py feat(accounting): okf build accounts for every source element 2026-09-17 18:35:20 +02:00
test_code_fence.py fix(gate,propose): okf build runs a real guard; a code fence declares no structure 2026-09-16 00:19:50 +02:00
test_content_accounting.py fix(accounting): a document refused whole is never clean, and the run says how many (H1) 2026-09-19 05:48:57 +02:00
test_contents_name_part.py feat(propose,extract,cli): a title that ends in a number, and a converter's own anchor in a concept id 2026-09-09 21:45:08 +02:00
test_contract_check.py feat(consume): the withheld set is counts plus names, not one entry per concept 2026-09-20 23:32:22 +02:00
test_contract_subquestions_passage.py feat(check): the contract and the checker state the payload's v1.1 forms 2026-09-21 08:47:46 +02:00
test_converter_attribute_titles.py feat(propose,extract,cli): a title that ends in a number, and a converter's own anchor in a concept id 2026-09-09 21:45:08 +02:00
test_corpus_run.py fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index 2026-09-08 10:06:58 +02:00
test_coverage_signal.py feat(consume): the payload says when the bundle looks like it does not cover the question 2026-09-21 07:12:43 +02:00
test_depth_seven.py test(extract): a section below markdown's sixth level keeps its own level in the plan 2026-09-11 14:02:49 +02:00
test_docs_promises.py feat(assets): every carried image is one a model can be shown 2026-09-19 08:07:04 +02:00
test_document_identity.py feat(identity): an STS document's doc-number names its directory and its title the address 2026-09-11 03:08:30 +02:00
test_document_prior.py feat(propose,consume,profiles,importer): recovery yields to declaration, and 9 % of the corpus that was in no segment 2026-09-09 14:17:00 +02:00
test_error_codes.py fix(assets): bound every link of the filter chain, and cover the backstop 2026-09-18 15:34:07 +02:00
test_exception_chaining.py test(errors): pin __cause__ preservation on every fail-fast wrap 2026-07-23 07:02:47 +02:00
test_extract.py test(extract): the file that IS the product had no reader, so state what one owes 2026-09-10 03:30:48 +02:00
test_faceted_index.py feat(profiles): a faceted index policy and the additive STRUCTURED_V1 profile 2026-08-27 00:30:42 +02:00
test_fidelity.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_file_connector.py feat(connectors): add the file connector with fail-closed path boundary 2026-07-16 19:55:08 +02:00
test_first_span.py feat(cli,consume): the first span, and the fusion that punished fine-graining 2026-09-09 15:40:01 +02:00
test_folder_of_bundles.py feat(check): the checker and the contract read a folder's reply 2026-09-21 10:50:14 +02:00
test_frontmatter_nesting.py test(gates): retire the K2 track; re-measure the retrieval gate's premises for BM25 2026-09-21 10:05:43 +02:00
test_gate_refusal_is_loud.py feat(build): name the documents the gate refused whole, and close as 1.0.0 2026-09-20 21:44:49 +02:00
test_golden.py feat(okf-v0.2): D5 — the v0.2 golden fixture, with okf_version in root frontmatter 2026-07-31 17:27:48 +02:00
test_guard_adapter.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_http_connector.py feat(connectors): add the http connector and wire the network gate 2026-07-16 20:06:21 +02:00
test_import_consumable.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_import_facets.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_import_flow.py feat(import): Door C flow against an injected import gate (Phase 2 step 5) 2026-07-25 06:57:25 +02:00
test_inbox.py feat(inbox): fail-fast on filenames over the 255-byte limit 2026-07-25 06:27:43 +02:00
test_inbox_flow.py feat(inbox): walk the drop directory recursively 2026-09-07 04:11:00 +02:00
test_inbox_recursion.py feat(inbox): walk the drop directory recursively 2026-09-07 04:11:00 +02:00
test_inbox_structure.py feat(inbox): Door B derives structure and reprojects the index additively 2026-08-27 00:37:12 +02:00
test_index.py feat(index): maintain existing indexes on re-materialization (spec §6) 2026-07-16 20:04:21 +02:00
test_index_policy.py fix(profiles): P1-F1 — a permitted root key is not a required one 2026-07-31 18:16:13 +02:00
test_index_sort.py test(index): pin Door C's cross-run ordering bound 2026-09-01 19:58:19 +02:00
test_index_star_row.py feat(propose,consume,profiles,importer): recovery yields to declaration, and 9 % of the corpus that was in no segment 2026-09-09 14:17:00 +02:00
test_k2_office_fixtures.py feat(propose,extract,cli): a title that ends in a number, and a converter's own anchor in a concept id 2026-09-09 21:45:08 +02:00
test_link_line_signal.py feat(consume): the door's link line is out of the default reading 2026-09-12 18:56:53 +02:00
test_load_bearing.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_manifest.py feat(manifest): reject [/] in extraction title (ingest-spec §4) 2026-07-23 21:33:47 +02:00
test_materialize.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_mcp_entry.py docs(project): the server is the standard way in, the skill the supplement 2026-09-21 10:36:52 +02:00
test_mcp_gate.py feat(mcp): serve OKF bundles over MCP in two shapes, plus the generic skill 2026-09-20 10:25:55 +02:00
test_mcp_server.py feat(mcp): serve OKF bundles over MCP in two shapes, plus the generic skill 2026-09-20 10:25:55 +02:00
test_md_declared_headings.py feat(propose,cli): the coverage gap had one cause, and round 7's own decomposition did not reproduce 2026-09-09 17:43:56 +02:00
test_multi_source_provenance.py fix(profiles,materialize,structure,consume): a block sources sequence is decoded, not skipped 2026-09-12 16:42:22 +02:00
test_okf_consume.py test(gates): retire the K2 track; re-measure the retrieval gate's premises for BM25 2026-09-21 10:05:43 +02:00
test_okf_consume_shim.py feat(propose,consume,tools): the type that declares nothing, and the prefix that is not a word 2026-09-09 23:05:45 +02:00
test_okf_project.py docs(project): the server is the standard way in, the skill the supplement 2026-09-21 10:36:52 +02:00
test_okf_skill.py feat(skill): one generic skill by default, carrying a working method and an answer form 2026-09-20 23:53:34 +02:00
test_okf_v0_2_characterization.py fix(profiles,materialize,structure,consume): a block sources sequence is decoded, not skipped 2026-09-12 16:42:22 +02:00
test_okf_v0_2_profile.py feat(profiles): DEFAULT stamps commons' O2 generated, V1 executed 2026-08-09 12:29:05 +02:00
test_okf_watch.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_outline_gate.py feat(propose,consume,profiles,importer): recovery yields to declaration, and 9 % of the corpus that was in no segment 2026-09-09 14:17:00 +02:00
test_outline_measure.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_packaging.py fix(assets): bound what the run pays, not what the document claims (0.10.1) 2026-09-18 13:41:18 +02:00
test_pandoc_binary.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_parent_index.py test(structure): the index resolves a parent naming a segment of its own document 2026-09-11 13:48:11 +02:00
test_parent_reaches_reader.py feat(check): the contract and the checker state the payload's v1.1 forms 2026-09-21 08:47:46 +02:00
test_parent_text.py docs(consume,build): both parent defaults stay off, and the reason is now a measurement 2026-09-11 13:39:50 +02:00
test_passage_delivery.py feat(consume): BM25 ranking by passage and title, a large concept delivered as its passage 2026-09-21 06:22:56 +02:00
test_pdf_font_and_ocr.py feat(extract,cli): typography as a PDF heading source and OCR behind an optional group, both off 2026-09-08 23:10:47 +02:00
test_pdf_heading_reserve.py feat(propose,cli): typography as a reserve, and the two of our own numbers it took to measure it 2026-09-09 00:25:51 +02:00
test_pdf_outline.py test(pdf): two bookmarks on one line, and nothing counts the one that is lost 2026-09-10 03:16:14 +02:00
test_profile.py feat(inbox): point every concept at the document it came from, with a locator per format 2026-09-08 14:39:24 +02:00
test_profile_threading.py test(profiles): pin the re-run promise V1 was published under 2026-08-09 12:39:23 +02:00
test_propose_segments.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_propose_sheet_sections.py feat(propose,extract,cli): a title that ends in a number, and a converter's own anchor in a concept id 2026-09-09 21:45:08 +02:00
test_provenance.py fix(extract): HTML collapsed to one line, so 828 of 828 sections had no boundary 2026-09-09 23:57:47 +02:00
test_quality.py test(gates): retire the K2 track; re-measure the retrieval gate's premises for BM25 2026-09-21 10:05:43 +02:00
test_read_path_containment.py fix(consume): every read path into a bundle is contained, not just okf_fetch 2026-09-20 15:20:38 +02:00
test_render.py feat(render): add §5 body renderers as pure functions 2026-07-16 19:52:21 +02:00
test_render_hook.py feat(profiles): a profile may name a per-suffix renderer 2026-09-02 14:19:18 +02:00
test_retrieval_gate.py fix(retrieval-gate): the judge reads a passage as its exact reconstruction — M15 felled, row 7 15 of 15 2026-09-22 23:53:09 +02:00
test_root_frontmatter_emission.py feat(okf-v0.2): D5 — the v0.2 golden fixture, with okf_version in root frontmatter 2026-07-31 17:27:48 +02:00
test_round7_defaults.py feat(cli,consume): the first span, and the fusion that punished fine-graining 2026-09-09 15:40:01 +02:00
test_run_frontmatter.py fix(frontmatter): write a value a YAML reader reads back, and read both forms 2026-09-11 11:09:49 +02:00
test_segmentation.py feat(segmentation): parse the adjudication state a plan already carries 2026-09-02 14:44:06 +02:00
test_segmented_collisions.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_segmented_golden.py test(golden): a clean extract is not a checkout, and says so 2026-09-19 19:19:34 +02:00
test_segmented_identity.py fix(segmentation): hash the extracted text and let the plan key fire 2026-09-02 14:38:20 +02:00
test_segmented_inbox.py feat(inbox): accept one segmentation plan per document 2026-09-02 14:51:53 +02:00
test_segmented_index.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_segmented_okf_v0_2_golden.py test(examples): byte-pinned golden for SEGMENTED_OKF_V0_2 2026-09-02 14:24:05 +02:00
test_segmented_profile.py feat(profiles): SEGMENTED_OKF_V0_2 so a segmented bundle can declare its spec 2026-09-02 14:21:15 +02:00
test_segmented_rebuild.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_segmented_rounds.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_shared_id_prefix.py feat(consume): the withheld set is counts plus names, not one entry per concept 2026-09-20 23:32:22 +02:00
test_shell_parent.py fix(structure): the index resolves a parent naming a segment of its own document 2026-09-11 13:55:03 +02:00
test_soek_gate.py test(soek): a red search gate measuring what the asker RECEIVES, one exit code 2026-09-21 04:03:11 +02:00
test_soft_hyphen_door.py test(gates): retire the K2 track; re-measure the retrieval gate's premises for BM25 2026-09-21 10:05:43 +02:00
test_span_gaps.py feat(propose,cli): the coverage gap had one cause, and round 7's own decomposition did not reproduce 2026-09-09 17:43:56 +02:00
test_sql_connector.py feat(connectors): add the sql connector (read-only sqlite, env-resolved) 2026-07-16 19:56:41 +02:00
test_stem_prefix.py feat(propose,consume,tools): the type that declares nothing, and the prefix that is not a word 2026-09-09 23:05:45 +02:00
test_strict_v1.py feat(phase-3): the index policy becomes configurable, with the reader that judges it 2026-07-25 20:30:54 +02:00
test_structure.py fix(structure): a derived reference needs a cue, not just a number shape 2026-08-29 09:26:57 +02:00
test_sts_description.py test(description): a spec point a YAML reader cannot read verbatim is not written 2026-09-11 03:46:11 +02:00
test_subquestions.py feat(mcp): the bundle's map, and a working method that reads it first 2026-09-21 08:37:12 +02:00
test_table_measure.py chore(ruff): the acceptance was whatever the default happened to be [skip-docs] 2026-09-09 23:15:17 +02:00
test_table_title_inheritance.py feat(consume): a table fragment is read under the heading it stands under 2026-09-21 08:26:20 +02:00
test_tie_shared_rank.py feat(consume): BM25 ranking by passage and title, a large concept delivered as its passage 2026-09-21 06:22:56 +02:00
test_title_covered.py test(consume): the lookup partition still wins, and the CLI defaults agree with the signature 2026-09-10 22:20:42 +02:00
test_title_covered_rise.py test(consume): a short title the question happens to cover passes the section it names 2026-09-11 01:23:14 +02:00
test_working_method.py docs: remove what this repository published about a consumer's corpus 2026-09-21 04:28:15 +02:00
test_xml_declared_route.py test(propose): a declared XML structure does not take the route declared structure takes 2026-09-10 06:55:00 +02:00
test_xml_sts.py test(extract): an exact invariant does not get to be 0.999998 2026-09-10 04:07:35 +02:00
test_yaml_frontmatter.py fix(profiles,materialize,structure,consume): a block sources sequence is decoded, not skipped 2026-09-12 16:42:22 +02:00