feat(inbox): walk the drop directory recursively

Door B listed `inbox.iterdir()` and kept only top-level files. A file in a
subdirectory was neither ingested nor refused: it appeared in none of the
result's buckets, so a nested drop produced a bundle that was silently short
of what was dropped and no count said so. That broke the K1b identity for any
inbox with folders in it. Operator decision 2026-09-06.

- `walk_inbox` is the ONE walk rule, shared with `tools/okf_corpus_run.py`:
  the denominator N is now counted over exactly the set of files the door
  ingests, rather than over a second listing that happened to agree.
- Sorted on the whole relative path, not the basename, so the order is a
  function of the tree; that is what keeps rebuild-from-scratch byte-equal to
  an incremental update.
- A concept's `source_file` is the path relative to the inbox root,
  `/`-separated. The concept NAME still comes from the basename, so two
  folders holding one basename hit the existing §3 collision refusal instead
  of one silently claiming the other's concept.
- Dot-directories and a bundle directory inside the inbox are skipped with a
  CODE, in a new `InboxResult.skipped`. Recursion makes the door's own output
  reachable as its own input; a silent skip would be the same
  absence-without-a-denominator defect one level down.
- `--path-prefix` reduces per component and rejoins with `/`, so the caller
  driving a nested corpus can carry the relative directory. Reducing the whole
  string folded the separator into a `-` and flattened `sub/sub2`.

`tests/test_inbox_flow.py::test_subdirectories_are_not_walked` asserted the
opposite and is superseded in place, with the reason written down.

Measured on the K2 corpus (flat, N=43): 39/43 merged, 4 coded, K1b holds. The
bundle digest is
`1472e98aec8643c5beee540f4c42b5e437bd26e7c61d69a91bcff799f06a6d13` over 1108
files -- byte-identical to a run of the same corpus at 190086f WITHOUT this
change (`diff -r` exit 0), so recursion costs a flat inbox nothing. It differs
from the stored 2026-09-03 artifact by one line in `index.md`
(`- [Corpus run history](log.md)`), which 95eb271 added 15 hours after that
bundle was built.

Suite 1113 passed, `ruff` clean, `mypy --strict src/ tools/` clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-07 04:11:00 +02:00
commit aa87eb8818
11 changed files with 489 additions and 39 deletions

View file

@ -52,6 +52,43 @@ def test_the_denominator_is_the_directory_not_a_literal(tmp_path: Path) -> None:
assert report.n == len(list(root.iterdir()))
def test_the_denominator_counts_files_in_subdirectories_too(tmp_path: Path) -> None:
"""N is the corpus, and the corpus is the whole tree.
The measurement and the door must walk by the SAME rule -- `walk_inbox` is
the one implementation -- or the report would state a denominator over a
different set of files than the one that was ingested, and the
conservation identity would hold over the wrong N.
"""
root = corpus(tmp_path, {"a.md": SUBSTANTIVE})
(root / "sub").mkdir()
(root / "sub" / "b.md").write_text(SUBSTANTIVE, encoding="utf-8", newline="")
report = okf_corpus_run.measure(root, tmp_path / "bundle", ingested_at=INGESTED_AT)
assert report.n == 2
assert report.persisted == 2
# `unaccounted` is `dropped - merged - coded`: it can only be empty if the
# measurement's names and the door's names are the SAME strings, which is
# what pins the two walks to one rule rather than to two that agree today.
assert report.unaccounted == ()
# Re-extracted through `corpus / source_file`, so the relative name has to
# resolve back to the file it came from.
assert report.substantive == 2
def test_the_denominator_skips_a_bundle_written_inside_the_corpus(tmp_path: Path) -> None:
root = corpus(tmp_path, {"a.md": SUBSTANTIVE})
bundle = root / "bundle"
okf_corpus_run.measure(root, bundle, ingested_at=INGESTED_AT)
second = okf_corpus_run.measure(root, bundle, ingested_at=INGESTED_AT)
assert second.n == 1
assert second.persisted == 1
assert second.unaccounted == ()
def test_the_conservation_identity_holds_and_the_run_exits_zero(tmp_path: Path) -> None:
root = corpus(
tmp_path,