open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

fix(corpus): the root index links the bundle's own log

Browse source 
Measured on the K2 artifact by a consumer: `log.md` was on disk and no
index named it, so a reader entering the bundle at `index.md` -- the walk
section 8 exists to support -- never reached the one file carrying `N`.

Stated as a LOCAL choice rather than conformance, because it is one.
Upstream's own bundles do not link their log: measured at `9a15b13`, 0 of
the 24 shipped `index.md` files name the single `log.md` in the set, with
the same grep form finding `tables/index.md` in 4 of them as the
known-positive control. That shows the link is not REQUIRED -- not that
it is disallowed. `docs/plan/okf-v0.2-alignment.md` P1-F6 already
recorded the upstream shape; a line there now separates the two claims,
since reserved names still stay out of an `entries_match_directory`
listing and this profile has that off.

It lives in the harness because the library cannot make it. The log's
content IS the run's outcome, so it cannot exist when the indexes are
projected, and an index that enumerated it off the directory would gain
the link only from the second run onward -- breaking
rebuild-equals-incremental, the property the segmented bundle is built
on.

The membership test is load-bearing and was measured, not assumed. The
two reprojections disagree about this line: the per-directory one drops
every managed entry before re-emitting its block, while the flat one
keeps a managed line whose target is not an owned concept, deliberately,
so that a regex cannot delete curated content. Appending unconditionally
therefore doubled the entry on the second unsegmented run, which is why
both run modes are pinned separately.

1052 -> 1054 tests. `mypy --strict` clean, `ruff` clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-03 20:23:56 +02:00
commit 95eb271f57
4 changed files with 180 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

14
CHANGELOG.md
View file

@ -17,6 +17,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
dated from `ingested_at`, so `merged + sum(coded rejections) == N` is
checkable from the bundle alone rather than only from a report that does not
travel with it. Without `--plans-dir` a run is unchanged.
- **The corpus harness links that `log.md` from the bundle's root `index.md`.**
Measured on the K2 artifact by a consumer: the log was on disk and no index
named it, so a reader entering the bundle at `index.md` -- the walk section 8
exists to support -- never reached the one file carrying `N`. A LOCAL choice
rather than conformance, and stated as one: upstream's own bundles do not
link their log (measured at `9a15b13`, 0 of 24 shipped `index.md` files name
the single `log.md` in the set), which shows the link is not required, not
that it is disallowed. It lives in the harness because the library cannot
make it -- the log's content is the run's outcome, so an index that
enumerated it off the directory would gain the link only from the second run
and break rebuild-equals-incremental. Written only when absent, because the
two reprojections differ: the per-directory one drops the line as a managed
entry, the flat one keeps it (its target is not an owned concept), and
appending unconditionally doubled it on the second unsegmented run.
Measured cause: the harness passed `STRUCTURED_V1` and no plans, so a
43-document corpus arrived as 39 flat concepts with no `adjudication` key