open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

fix(build,consume): stamp every segmented concept, exclude a linked log.md from concept navigation

Browse source 
Two producer-side defects from the S7 acid test (ordre 20260907T234741Z-9578626297-from-.claude), both reproduced on K2 before and after.

F1: `okf build --ingested-at` alone stamped only 11/629 concepts -- the unsegmented ones, which read the call's value directly. The 618 segmented concepts read `segment.ingested_at`, the plan's `proposed_at`, independently defaulted to `DEFAULT_STAMP`. `proposed_at` now falls back to `ingested_at` when omitted; neither flag passed still yields `DEFAULT_STAMP` for both.

F2: the consumption pre-pass's index walk counted a root-linked `log.md` (`corpus.link_log_in_root_index`, `95eb271`) as a concept, inflating a 629-concept K2 rebuild to 630 and letting the log rank and cut like real content. The link stays -- the contract is silent on `log.md` and `95eb271` already named it a LOCAL choice -- but the walk now treats `LOG_NAME` like the index itself: reachable, never a concept.

K2 rebuilt twice from the same corpus and diffed against the delivered `K2-bundle-20260903`: FOR (stashed fix, matching fbaac6d) reproduces po's numbers exactly -- 619/1108 files differ, 618 ingested_at-only, ref `sha256-tree:4ffd750c...`. ETTER (fix applied) leaves exactly 1 line differing (the deliberate log link, predating this fix) -- 0 files stamped 1970, 629/629 stamped 2026-09-03, ref `sha256-tree:f14872a0...`. The delivered bundle's ref is unchanged before and after (`sha256-tree:9a4e5561...a968b5`), since it carries no log link and the new branch never fires. Conservation identity holds both times: merged + coded rejections = 43 = N, 39/0/4.

1258 -> 1260 tests. mypy --strict clean on 28 files. ruff clean. Both goldens byte-unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 03:21:36 +02:00
commit 5a0c8794af
5 changed files with 111 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

30
src/llm_ingestion_okf/cli.py
View file

@ -25,14 +25,16 @@ started walking recursively: `a/krav.pdf` and `b/krav.pdf` would both reduce to
impossible rather than merely detectable. For a flat inbox the relative path IS
the basename, which is why the published bundles' bytes do not move.
**Omitted timestamps do not come from the clock.** `--ingested-at` and
`--proposed-at` default to `DEFAULT_STAMP`, one constant used for both. A
wall-clock default would put a changing byte into the artifact and break
**Omitted timestamps do not come from the clock.** `--ingested-at` defaults to
`DEFAULT_STAMP`, and `--proposed-at` defaults to `--ingested-at` -- so a caller
who names one clock stamps the whole bundle with it, and a caller who names
neither gets `DEFAULT_STAMP` for both, one constant either way. A wall-clock
default would put a changing byte into the artifact and break
rebuild-equals-incremental (K6) for every caller who did not pass the flags --
the property the segmented bundle is built on, and the one a convenience
default is most likely to take away silently. The epoch is deliberate and
readable as what it is: a stamp nobody set. A caller who wants a real ingest
time passes one.
readable as what it is: a stamp nobody set. A caller who wants the proposal and
the replay dated differently passes both explicitly.
## What it does not decide
@ -111,7 +113,7 @@ def build(
bundle: Path,
*,
ingested_at: str = DEFAULT_STAMP,
proposed_at: str = DEFAULT_STAMP,
proposed_at: str | None = None,
bundle_id: str | None = None,
okf_version: str | None = None,
segments: bool = True,
@ -122,7 +124,18 @@ def build(
Keyword-only with defaults, so a caller who takes this as an API keeps a
source-compatible call when a flag is added.
`proposed_at` defaults to `ingested_at` rather than independently to
`DEFAULT_STAMP`: a caller naming one clock is naming "when this ran", not
asking for two different clocks. Left independently defaulted, a caller who
set only `--ingested-at` stamped the unsegmented concepts (which read the
call's `ingested_at` directly) and left every segmented one -- which reads
`segment.ingested_at`, the plan's `proposed_at` -- on `DEFAULT_STAMP`.
Measured on K2: 11 of 629 concepts. Two independently-set clocks are still
reachable by passing `--proposed-at` explicitly.
"""
if proposed_at is None:
proposed_at = ingested_at
if not segments:
report = measure(inbox, bundle, ingested_at=ingested_at, profile=STRUCTURED_V1)
_write_log(bundle, report, profile=STRUCTURED_V1)
@ -218,8 +231,9 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
)
build_parser.add_argument(
"--proposed-at",
default=DEFAULT_STAMP,
help=f"written into every proposal. Default {DEFAULT_STAMP}, for the same reason",
default=None,
help=f"written into every proposal. Defaults to --ingested-at (itself {DEFAULT_STAMP} "
"if that is also omitted), so naming one clock never leaves the other unset",
)
build_parser.add_argument(
"--segments",