fix(build,consume): stamp every segmented concept, exclude a linked log.md from concept navigation

Two producer-side defects from the S7 acid test (ordre 20260907T234741Z-9578626297-from-.claude), both reproduced on K2 before and after.

F1: `okf build --ingested-at` alone stamped only 11/629 concepts -- the unsegmented ones, which read the call's value directly. The 618 segmented concepts read `segment.ingested_at`, the plan's `proposed_at`, independently defaulted to `DEFAULT_STAMP`. `proposed_at` now falls back to `ingested_at` when omitted; neither flag passed still yields `DEFAULT_STAMP` for both.

F2: the consumption pre-pass's index walk counted a root-linked `log.md` (`corpus.link_log_in_root_index`, `95eb271`) as a concept, inflating a 629-concept K2 rebuild to 630 and letting the log rank and cut like real content. The link stays -- the contract is silent on `log.md` and `95eb271` already named it a LOCAL choice -- but the walk now treats `LOG_NAME` like the index itself: reachable, never a concept.

K2 rebuilt twice from the same corpus and diffed against the delivered `K2-bundle-20260903`: FOR (stashed fix, matching fbaac6d) reproduces po's numbers exactly -- 619/1108 files differ, 618 ingested_at-only, ref `sha256-tree:4ffd750c...`. ETTER (fix applied) leaves exactly 1 line differing (the deliberate log link, predating this fix) -- 0 files stamped 1970, 629/629 stamped 2026-09-03, ref `sha256-tree:f14872a0...`. The delivered bundle's ref is unchanged before and after (`sha256-tree:9a4e5561...a968b5`), since it carries no log link and the new branch never fires. Conservation identity holds both times: merged + coded rejections = 43 = N, 39/0/4.

1258 -> 1260 tests. mypy --strict clean on 28 files. ruff clean. Both goldens byte-unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-08 03:21:36 +02:00
commit 5a0c8794af
5 changed files with 111 additions and 9 deletions

View file

@ -286,6 +286,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
The verdict parse now raises `AdjudicationError`, so the message names what The verdict parse now raises `AdjudicationError`, so the message names what
actually failed. Exit 2 either way; nothing is written either way. actually failed. Exit 2 either way; nothing is written either way.
- **`okf build --ingested-at` alone now stamps every concept, not just the
unsegmented ones.** `--proposed-at` defaulted independently to
`DEFAULT_STAMP`, so a caller who set only `--ingested-at` stamped the 11 of
629 concepts that read the call's value directly, while the 618 segmented
concepts -- which read `segment.ingested_at`, the plan's `proposed_at` --
stayed on `1970-01-01T00:00:00Z`. Measured on K2 rebuilt at `fbaac6d`.
`proposed_at` now defaults to `ingested_at` when omitted; a caller who wants
the proposal and the replay dated differently still passes both explicitly.
Neither flag passed still yields `DEFAULT_STAMP` for both, byte-identical to
before.
- **The consumption pre-pass (`tools/okf_consume.py`) no longer counts a
linked `log.md` as a concept.** `link_log_in_root_index` (`corpus.py`,
`95eb271`) links a run's own log from the root index for bundle navigation;
the index walk that enumerates concepts followed that link like any other
and counted the log as one, inflating a 629-concept K2 rebuild to 630 and
letting the log rank and get cut like real content. The link stays --
`docs/consumption-contract.md` is silent on `log.md`, and `95eb271` already
named the link a LOCAL choice rather than conformance -- but the walk now
recognises `LOG_NAME` the same way it recognises the index itself: reachable
for navigation, never a concept. A bundle whose index does not link its log
(every bundle built before `95eb271`, including the delivered
`K2-bundle-20260903`) computes the same `sha256-tree` ref before and after.
## [0.5.0a2] — 2026-07-31 ## [0.5.0a2] — 2026-07-31
**This is the pre-release the pilots pin. `v0.5.0a1` was tagged and abandoned **This is the pre-release the pilots pin. `v0.5.0a1` was tagged and abandoned

View file

@ -25,14 +25,16 @@ started walking recursively: `a/krav.pdf` and `b/krav.pdf` would both reduce to
impossible rather than merely detectable. For a flat inbox the relative path IS impossible rather than merely detectable. For a flat inbox the relative path IS
the basename, which is why the published bundles' bytes do not move. the basename, which is why the published bundles' bytes do not move.
**Omitted timestamps do not come from the clock.** `--ingested-at` and **Omitted timestamps do not come from the clock.** `--ingested-at` defaults to
`--proposed-at` default to `DEFAULT_STAMP`, one constant used for both. A `DEFAULT_STAMP`, and `--proposed-at` defaults to `--ingested-at` -- so a caller
wall-clock default would put a changing byte into the artifact and break who names one clock stamps the whole bundle with it, and a caller who names
neither gets `DEFAULT_STAMP` for both, one constant either way. A wall-clock
default would put a changing byte into the artifact and break
rebuild-equals-incremental (K6) for every caller who did not pass the flags -- rebuild-equals-incremental (K6) for every caller who did not pass the flags --
the property the segmented bundle is built on, and the one a convenience the property the segmented bundle is built on, and the one a convenience
default is most likely to take away silently. The epoch is deliberate and default is most likely to take away silently. The epoch is deliberate and
readable as what it is: a stamp nobody set. A caller who wants a real ingest readable as what it is: a stamp nobody set. A caller who wants the proposal and
time passes one. the replay dated differently passes both explicitly.
## What it does not decide ## What it does not decide
@ -111,7 +113,7 @@ def build(
bundle: Path, bundle: Path,
*, *,
ingested_at: str = DEFAULT_STAMP, ingested_at: str = DEFAULT_STAMP,
proposed_at: str = DEFAULT_STAMP, proposed_at: str | None = None,
bundle_id: str | None = None, bundle_id: str | None = None,
okf_version: str | None = None, okf_version: str | None = None,
segments: bool = True, segments: bool = True,
@ -122,7 +124,18 @@ def build(
Keyword-only with defaults, so a caller who takes this as an API keeps a Keyword-only with defaults, so a caller who takes this as an API keeps a
source-compatible call when a flag is added. source-compatible call when a flag is added.
`proposed_at` defaults to `ingested_at` rather than independently to
`DEFAULT_STAMP`: a caller naming one clock is naming "when this ran", not
asking for two different clocks. Left independently defaulted, a caller who
set only `--ingested-at` stamped the unsegmented concepts (which read the
call's `ingested_at` directly) and left every segmented one -- which reads
`segment.ingested_at`, the plan's `proposed_at` -- on `DEFAULT_STAMP`.
Measured on K2: 11 of 629 concepts. Two independently-set clocks are still
reachable by passing `--proposed-at` explicitly.
""" """
if proposed_at is None:
proposed_at = ingested_at
if not segments: if not segments:
report = measure(inbox, bundle, ingested_at=ingested_at, profile=STRUCTURED_V1) report = measure(inbox, bundle, ingested_at=ingested_at, profile=STRUCTURED_V1)
_write_log(bundle, report, profile=STRUCTURED_V1) _write_log(bundle, report, profile=STRUCTURED_V1)
@ -218,8 +231,9 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
) )
build_parser.add_argument( build_parser.add_argument(
"--proposed-at", "--proposed-at",
default=DEFAULT_STAMP, default=None,
help=f"written into every proposal. Default {DEFAULT_STAMP}, for the same reason", help=f"written into every proposal. Defaults to --ingested-at (itself {DEFAULT_STAMP} "
"if that is also omitted), so naming one clock never leaves the other unset",
) )
build_parser.add_argument( build_parser.add_argument(
"--segments", "--segments",

View file

@ -372,6 +372,30 @@ def test_the_default_stamp_is_named_once(tmp_path: Path) -> None:
assert cli.DEFAULT_STAMP[:10] in (bundle / "log.md").read_text(encoding="utf-8") assert cli.DEFAULT_STAMP[:10] in (bundle / "log.md").read_text(encoding="utf-8")
def test_ingested_at_alone_stamps_every_concept_the_same(tmp_path: Path) -> None:
"""`--ingested-at` without `--proposed-at` must stamp every concept alike.
Measured on K2 (S7 acid test, F1): passing only `--ingested-at` stamped 11
of 629 concepts -- the unsegmented whole-document concepts, which read the
call's `ingested_at` directly. The 618 segmented concepts read
`segment.ingested_at`, which is the PLAN's `proposed_at`, independently
defaulted to `DEFAULT_STAMP` when the caller never set it. A caller naming
one clock is naming "when this ran", not asking for two different clocks.
"""
inbox = inbox_with_subdirectories(tmp_path)
bundle = tmp_path / "bundle"
assert build(inbox, bundle, "--ingested-at", INGESTED_AT) == 0
concepts = {
name: body
for name, body in tree(bundle).items()
if name.endswith(".md") and not name.endswith("index.md") and name != "log.md"
}
assert concepts, "the fixture must produce concepts for this control to mean anything"
for name, body in concepts.items():
assert f"ingested_at: {INGESTED_AT}".encode() in body, name
assert cli.DEFAULT_STAMP.encode() not in body, name
# --- the one implementation ------------------------------------------------ # --- the one implementation ------------------------------------------------

View file

@ -112,6 +112,37 @@ def _copy_bundle(source: Path, target: Path) -> None:
destination.write_bytes(path.read_bytes()) destination.write_bytes(path.read_bytes())
def test_the_index_walk_excludes_a_linked_log_from_concept_navigation(tmp_path: Path) -> None:
"""A linked `log.md` is bundle metadata, not a concept -- measured on K2 (S7 F2).
`link_log_in_root_index` (`corpus.py`) links a run's own log from the root
index so a reader entering at `index.md` can reach it (`95eb271`). That link
makes the log reachable by the same walk this instrument uses to enumerate
concepts, and a walk that does not distinguish "linked" from "concept"
counts it as a 630th concept on a 629-concept bundle -- exactly what the S7
acid test measured, with the log then ranked and cut like real content.
"""
from llm_ingestion_okf.corpus import LOG_NAME, link_log_in_root_index
from llm_ingestion_okf.profiles import SEGMENTED_OKF_V0_2
copy = tmp_path / "bundle"
_copy_bundle(GOLDEN, copy)
(copy / LOG_NAME).write_text("# Corpus run history\n\nN = 3\n", encoding="utf-8")
link_log_in_root_index(copy, SEGMENTED_OKF_V0_2)
index = (copy / "index.md").read_text(encoding="utf-8")
assert "](log.md)" in index, (
"the fixture must actually link the log for this control to mean anything"
)
found = okf_consume.enumerate_concepts(copy)
assert found == (
"krav/1-1/foerste-krav",
"krav/1-2/andre-krav",
"veiledning",
)
assert not any(concept.endswith("log") for concept in found)
def test_the_ref_covers_the_indexes_too_since_the_walk_reads_them(tmp_path: Path) -> None: def test_the_ref_covers_the_indexes_too_since_the_walk_reads_them(tmp_path: Path) -> None:
# The docstring claims every byte that can reach a payload is inside the # The docstring claims every byte that can reach a payload is inside the
# ref. An index byte can: it decides which concepts are reachable at all. # ref. An index byte can: it decides which concepts are reachable at all.

View file

@ -44,6 +44,7 @@ from typing import Literal
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src")) sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
from llm_ingestion_okf.corpus import LOG_NAME # noqa: E402
from llm_ingestion_okf.inbox import ( # noqa: E402 from llm_ingestion_okf.inbox import ( # noqa: E402
ADJUDICATION_ADJUDICATED, ADJUDICATION_ADJUDICATED,
ADJUDICATION_PROPOSED, ADJUDICATION_PROPOSED,
@ -105,8 +106,16 @@ def _walk_index_tree(
target = _join(parent, entry.target) target = _join(parent, entry.target)
if target is None: if target is None:
continue continue
if target.rsplit("/", 1)[-1] == index_name: name = target.rsplit("/", 1)[-1]
if name == index_name:
pending.append(target) pending.append(target)
elif name == LOG_NAME:
# A run's own log, linked for bundle navigation
# (`corpus.link_log_in_root_index`) -- reachable, but metadata,
# never a concept. Counted as one it inflated the K2 walk to
# 630 against a 629-concept bundle and let the log rank and
# cut like real content (measured, S7 F2).
continue
elif target.endswith(suffix): elif target.endswith(suffix):
concepts.add(target[: -len(suffix)]) concepts.add(target[: -len(suffix)])
return _byte_sorted(indexes), _byte_sorted(concepts) return _byte_sorted(indexes), _byte_sorted(concepts)