fix(extract,build): write a spreadsheet as pipe tables, stop linking the run log from the index
Two producer-side findings from the consumer's S7c acid test (ordre 20260908T063454Z-3648220855-from-.claude), both measured on K2 before and after, both with the corpus rebuilt from scratch.
FUNN 3 -- THE FORM. The converter's default markdown writer emits simple tables, which pad every cell out to the width of the widest cell in its column. Measured on the tender's price sheet: one 594-character prose cell produced a 67 244-character whitespace carpet with runs of up to 887 characters between a label and its amount, 19 integral amounts carrying a converter `.0`, and a header row naming one column. The bytes reached a live model in 2 of 11 prompts and 0 of 11 answers. The spreadsheet row now writes pipe tables with `--columns=1` (load-bearing: the pipe writer pads to a width computed from it, so at the default 72 a narrow table gains runs of up to 45). Same sheet after: 11 048 characters, longest run 2, one row per line, 0 artificial `.0`. Spreadsheet-only, and the scoping is pinned by three digests -- the same change moves the odt fixture 1366 -> 1105, so it can fail.
The `.0` rewrite is bounded twice: to a cell whose whole content is such a number, anchored between unescaped pipes, and skipped when the literal is in the workbook's shared string table -- the converter renders the number 92 and the TEXT "92.0" identically, so the output alone cannot tell them apart. Read with zipfile and xml.etree; no new dependency.
FUNN 2 -- THE LOG LINK. `link_log_in_root_index` (95eb271) is removed. Consumption contract SS 9.2 forbids a consumer from enumerating the bundle directory unless the profile says the index is derived, so the index tree is the entire map a consumer may use and everything it links is a document: their navigator returned 630 where our pre-pass counts 629, and a corpus run's own log was citable as content. The log is still written to the bundle root (SPEC section 9); `tools/okf_consume.py` keeps its exclusion for the bundles already built with the link.
K2 rebuilt twice. BEFORE reproduces the consumer's ref exactly (`sha256-tree:f14872a0...c8a92a`, 629 concepts) and their three consume figures to the token (57 289 / 62 149 / 58 401). AFTER: 629 concepts, `merged + coded rejections = 43 = N`, new ref `sha256-tree:c26eed6a...e3261f`, 627 of 629 concepts byte-identical, 1104 of 1108 files identical to the delivered bundle.
ONE REGRESSION, MEASURED AND NOT FIXED: on the mandate-shaped question with the vocabulary bridge the priced concept moves from candidate rank 10 to 19, so `--k 12` withholds it `below_k`; `--cost-vocabulary --k 20` delivers it at 65 912 o200k. The cause is measured rather than argued -- restoring only the concept's title on the new short body ranks it 10 again. The chain ends at the orphan check (`propose.py:461`), which drops the sheet heading once a table block opens two lines below it. That is the already-reported orphan gate, and changing it is a default-ON segmentation rule affecting every document type. The specific question is unaffected: rank 1 before and after. The priced excerpt's budget share falls from 56.5 % to 9.7 %.
11 new tests (RED first), 8 mutations, 8 red, with an unmutated control green each time. One mutation survived twice before the fixture could make it fire, and both survivals are written down. 1279 -> 1287 tests. mypy --strict clean on 28 files. ruff clean. Both proposer goldens byte-unchanged. One frozen literal moved with the fix and is reported rather than hidden.
Report: docs/2026-09-08-prisform-og-loggen-k2.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
6776c37d23
commit
56ae274246
13 changed files with 1018 additions and 128 deletions
|
|
@ -426,3 +426,46 @@ def test_the_packaged_modules_do_not_reach_back_into_the_clone() -> None:
|
|||
for name in ("propose.py", "corpus.py", "cli.py"):
|
||||
source = (PROJECT_ROOT / "src" / "llm_ingestion_okf" / name).read_text(encoding="utf-8")
|
||||
assert "sys.path" not in source
|
||||
|
||||
|
||||
def test_the_root_index_does_not_link_the_run_log(tmp_path: Path) -> None:
|
||||
"""Producer and consumer must count the same documents. Measured: they did not.
|
||||
|
||||
`link_log_in_root_index` (`95eb271`) appended a markdown link to `log.md`
|
||||
from the root index so a reader entering there could reach the one file
|
||||
carrying `N`. The consumption contract SS 9.2 forbids a consumer from
|
||||
enumerating the bundle directory unless the named profile says the index is
|
||||
derived -- which for this profile it does not -- so the index tree IS the
|
||||
whole map a consumer is allowed to use. Anything the index links is a
|
||||
document, by that contract.
|
||||
|
||||
The cost was measured on K2 by the first consumer to walk the bundle with a
|
||||
live model: their navigator followed the link and returned 630 documents
|
||||
where this repository's own pre-pass counts 629, and the corpus run's own
|
||||
log was reachable and citable as content. Our pre-pass excluding `log.md`
|
||||
(`5a0c879`, F2) fixed the count on OUR side only; the disagreement is
|
||||
produced HERE.
|
||||
|
||||
The log still exists at the bundle root, which is where SS 9 puts it and is
|
||||
all F2 ever required. Upstream's own bundles show the link was never
|
||||
required either: measured at `9a15b13`, 0 of the 24 shipped `index.md`
|
||||
files name the single `log.md` in the set.
|
||||
"""
|
||||
inbox = inbox_with_subdirectories(tmp_path)
|
||||
bundle = tmp_path / "bundle"
|
||||
assert build(inbox, bundle) == 0
|
||||
|
||||
assert (bundle / "log.md").is_file(), "the log itself stays in the bundle"
|
||||
index = (bundle / "index.md").read_text(encoding="utf-8")
|
||||
assert "log.md" not in index
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "tools"))
|
||||
import okf_consume
|
||||
|
||||
concepts = {
|
||||
name
|
||||
for name in tree(bundle)
|
||||
if name.endswith(".md") and not name.endswith("index.md") and name != "log.md"
|
||||
}
|
||||
assert concepts, "the fixture must produce concepts for this control to mean anything"
|
||||
assert len(okf_consume.enumerate_concepts(bundle)) == len(concepts)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue