open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

test(corpus): a rebuild with log.md present reproduces the bundle exactly

Browse source 
`log.md` is written INTO a directory Door B enumerates on the next round: it
matches the concept glob and is excluded only by `index.md`'s name, so a
rebuild could have seen it as pre-existing curated content or pruned it.
Rebuild-equals-incremental is the property the segmented bundle rests on.

Measured on the real artifact, not only the synthetic: the K2 corpus was run
a second time into the same bundle and compared against a snapshot with
`diff -r`, exit 0 over all 1108 files. The test pins the same property in
seconds instead of 13 minutes.

Also corrects the report's reproduction command -- it documented plan
filenames the run did not use, and re-running it into the existing plans
directory would leave two files claiming one `source_sha256`, which
`_resolve_plans` refuses.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-03 05:16:25 +02:00
commit 2a2eb9c8bd
2 changed files with 72 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

33
docs/2026-09-03-k2-bundle-rebuild.md
View file

@ -61,22 +61,33 @@ Three commits, each test-first.
## The rebuild
Bundle: `~/corpora/okf-telling-20260829/K2-bundle-20260903/` (5.5 MB, 29
directories). Plans: `~/corpora/okf-telling-20260829/K2-plans-20260903/`.
Both outside the repository, both durable.
Bundle: `~/corpora/okf-telling-20260829/K2-bundle-20260903/` — 5.5 MB, 1108
files: 629 concepts, 478 `index.md` (one per directory, the profile sets
`per_directory`), and one `log.md`. Plans:
`~/corpora/okf-telling-20260829/K2-plans-20260903/` — 28 of them, one per
document that had something to propose. Both outside the repository, both
durable.
**Reproduce into a FRESH plans directory.** A plan is selected by
`source_sha256`, so re-running the proposer into a directory that already holds
these plans leaves two files claiming the same hash, and `_resolve_plans`
refuses that rather than picking one:
```
PLANS=~/corpora/okf-telling-20260829/K2-plans-$(date +%Y%m%d-%H%M%S); mkdir -p "$PLANS"
i=0
for f in ~/corpora/okf-telling-20260829/K2/trinn1/*; do
b=$(basename "$f"); .venv/bin/python tools/okf_propose_segments.py "$f" \
--out ~/corpora/okf-telling-20260829/K2-plans-20260903/"${b%.*}".json \
i=$((i+1)); b=$(basename "$f")
.venv/bin/python tools/okf_propose_segments.py "$f" \
--out "$PLANS/$(printf '%02d' $i).json" \
--path-prefix "${b%.*}" --proposed-at 2026-09-03T00:00:00Z
done
done # exit 1 for the 11 with nothing to propose, exit 2 for the 4 unreadable
.venv/bin/python tools/okf_corpus_run.py \
--corpus ~/corpora/okf-telling-20260829/K2/trinn1 \
--report ~/corpora/okf-telling-20260829/K2-bundle-20260903-report.md \
--bundle ~/corpora/okf-telling-20260829/K2-bundle-20260903 \
--ingested-at 2026-09-03T00:00:00Z \
--plans-dir ~/corpora/okf-telling-20260829/K2-plans-20260903 \
--plans-dir "$PLANS" \
--bundle-id k2-trinn1-20260903 --okf-version 0.2
```
@ -97,10 +108,18 @@ Converter as the harness resolved it: the vendored `pypandoc` binary, version
| concepts over 100 000 characters | 4/629 |
| wall time | 784.29 s total, 18.239 s per file |
| K1b | `39 + 4 = 43 = N`, run exited `0` |
| rebuild == incremental | `diff -r` exit `0` over all 1108 files |
The previous bundle, re-measured here rather than quoted: 39 concepts, **0/39**
carrying `adjudication`, max body 267 548 characters, no `log.md`.
`log.md` is a file the run path writes into a directory Door B enumerates on
the next round, so the rebuild property was re-measured rather than assumed:
the whole corpus was run a second time into the same bundle and compared
against a snapshot with `diff -r`, which exited `0`. A test in
`tests/test_corpus_run.py` pins the same property on a synthetic corpus, so it
fails in seconds rather than in 13 minutes.
**K1b is now recoverable from the bundle alone**, which was the point of §9:
```

46
tests/test_corpus_run.py
View file

@ -321,3 +321,49 @@ def test_without_a_plan_directory_the_run_is_unchanged(tmp_path: Path) -> None:
concepts = [path for path in bundle.rglob("*.md") if path.name not in ("index.md", "log.md")]
assert len(concepts) == 1
assert "adjudication:" not in concepts[0].read_text(encoding="utf-8")
def test_a_second_run_into_the_same_bundle_reproduces_it_byte_for_byte(tmp_path: Path) -> None:
"""`log.md` is a file the harness writes INTO a directory Door B enumerates
on the next round: it matches the concept glob and is excluded only by
`index.md`'s name, so a rebuild could see it as pre-existing curated content
or prune it. Rebuild-equals-incremental is the property the segmented bundle
is built on, and a log that broke it would be worse than no log.
"""
root = corpus(tmp_path, {"doc.md": SEGMENTABLE, "flat.md": SUBSTANTIVE})
plans = tmp_path / "plans"
plans.mkdir()
_propose(root / "doc.md", plans / "doc.json")
bundle = tmp_path / "bundle"
argv = [
"--corpus",
str(root),
"--report",
str(tmp_path / "r.md"),
"--bundle",
str(bundle),
"--ingested-at",
INGESTED_AT,
"--plans-dir",
str(plans),
"--bundle-id",
"k2",
"--okf-version",
"0.2",
]
assert okf_corpus_run.main(argv) == 0
first = {
path.relative_to(bundle).as_posix(): path.read_bytes()
for path in sorted(bundle.rglob("*"))
if path.is_file()
}
assert "log.md" in first
assert okf_corpus_run.main(argv) == 0
second = {
path.relative_to(bundle).as_posix(): path.read_bytes()
for path in sorted(bundle.rglob("*"))
if path.is_file()
}
assert second == first