fix(build,consume): stamp every segmented concept, exclude a linked log.md from concept navigation
Two producer-side defects from the S7 acid test (ordre 20260907T234741Z-9578626297-from-.claude), both reproduced on K2 before and after. F1: `okf build --ingested-at` alone stamped only 11/629 concepts -- the unsegmented ones, which read the call's value directly. The 618 segmented concepts read `segment.ingested_at`, the plan's `proposed_at`, independently defaulted to `DEFAULT_STAMP`. `proposed_at` now falls back to `ingested_at` when omitted; neither flag passed still yields `DEFAULT_STAMP` for both. F2: the consumption pre-pass's index walk counted a root-linked `log.md` (`corpus.link_log_in_root_index`, `95eb271`) as a concept, inflating a 629-concept K2 rebuild to 630 and letting the log rank and cut like real content. The link stays -- the contract is silent on `log.md` and `95eb271` already named it a LOCAL choice -- but the walk now treats `LOG_NAME` like the index itself: reachable, never a concept. K2 rebuilt twice from the same corpus and diffed against the delivered `K2-bundle-20260903`: FOR (stashed fix, matchingfbaac6d) reproduces po's numbers exactly -- 619/1108 files differ, 618 ingested_at-only, ref `sha256-tree:4ffd750c...`. ETTER (fix applied) leaves exactly 1 line differing (the deliberate log link, predating this fix) -- 0 files stamped 1970, 629/629 stamped 2026-09-03, ref `sha256-tree:f14872a0...`. The delivered bundle's ref is unchanged before and after (`sha256-tree:9a4e5561...a968b5`), since it carries no log link and the new branch never fires. Conservation identity holds both times: merged + coded rejections = 43 = N, 39/0/4. 1258 -> 1260 tests. mypy --strict clean on 28 files. ruff clean. Both goldens byte-unchanged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
35b8c9e805
commit
5a0c8794af
5 changed files with 111 additions and 9 deletions
24
CHANGELOG.md
24
CHANGELOG.md
|
|
@ -286,6 +286,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
The verdict parse now raises `AdjudicationError`, so the message names what
|
||||
actually failed. Exit 2 either way; nothing is written either way.
|
||||
|
||||
- **`okf build --ingested-at` alone now stamps every concept, not just the
|
||||
unsegmented ones.** `--proposed-at` defaulted independently to
|
||||
`DEFAULT_STAMP`, so a caller who set only `--ingested-at` stamped the 11 of
|
||||
629 concepts that read the call's value directly, while the 618 segmented
|
||||
concepts -- which read `segment.ingested_at`, the plan's `proposed_at` --
|
||||
stayed on `1970-01-01T00:00:00Z`. Measured on K2 rebuilt at `fbaac6d`.
|
||||
`proposed_at` now defaults to `ingested_at` when omitted; a caller who wants
|
||||
the proposal and the replay dated differently still passes both explicitly.
|
||||
Neither flag passed still yields `DEFAULT_STAMP` for both, byte-identical to
|
||||
before.
|
||||
|
||||
- **The consumption pre-pass (`tools/okf_consume.py`) no longer counts a
|
||||
linked `log.md` as a concept.** `link_log_in_root_index` (`corpus.py`,
|
||||
`95eb271`) links a run's own log from the root index for bundle navigation;
|
||||
the index walk that enumerates concepts followed that link like any other
|
||||
and counted the log as one, inflating a 629-concept K2 rebuild to 630 and
|
||||
letting the log rank and get cut like real content. The link stays --
|
||||
`docs/consumption-contract.md` is silent on `log.md`, and `95eb271` already
|
||||
named the link a LOCAL choice rather than conformance -- but the walk now
|
||||
recognises `LOG_NAME` the same way it recognises the index itself: reachable
|
||||
for navigation, never a concept. A bundle whose index does not link its log
|
||||
(every bundle built before `95eb271`, including the delivered
|
||||
`K2-bundle-20260903`) computes the same `sha256-tree` ref before and after.
|
||||
|
||||
## [0.5.0a2] — 2026-07-31
|
||||
|
||||
**This is the pre-release the pilots pin. `v0.5.0a1` was tagged and abandoned
|
||||
|
|
|
|||
|
|
@ -25,14 +25,16 @@ started walking recursively: `a/krav.pdf` and `b/krav.pdf` would both reduce to
|
|||
impossible rather than merely detectable. For a flat inbox the relative path IS
|
||||
the basename, which is why the published bundles' bytes do not move.
|
||||
|
||||
**Omitted timestamps do not come from the clock.** `--ingested-at` and
|
||||
`--proposed-at` default to `DEFAULT_STAMP`, one constant used for both. A
|
||||
wall-clock default would put a changing byte into the artifact and break
|
||||
**Omitted timestamps do not come from the clock.** `--ingested-at` defaults to
|
||||
`DEFAULT_STAMP`, and `--proposed-at` defaults to `--ingested-at` -- so a caller
|
||||
who names one clock stamps the whole bundle with it, and a caller who names
|
||||
neither gets `DEFAULT_STAMP` for both, one constant either way. A wall-clock
|
||||
default would put a changing byte into the artifact and break
|
||||
rebuild-equals-incremental (K6) for every caller who did not pass the flags --
|
||||
the property the segmented bundle is built on, and the one a convenience
|
||||
default is most likely to take away silently. The epoch is deliberate and
|
||||
readable as what it is: a stamp nobody set. A caller who wants a real ingest
|
||||
time passes one.
|
||||
readable as what it is: a stamp nobody set. A caller who wants the proposal and
|
||||
the replay dated differently passes both explicitly.
|
||||
|
||||
## What it does not decide
|
||||
|
||||
|
|
@ -111,7 +113,7 @@ def build(
|
|||
bundle: Path,
|
||||
*,
|
||||
ingested_at: str = DEFAULT_STAMP,
|
||||
proposed_at: str = DEFAULT_STAMP,
|
||||
proposed_at: str | None = None,
|
||||
bundle_id: str | None = None,
|
||||
okf_version: str | None = None,
|
||||
segments: bool = True,
|
||||
|
|
@ -122,7 +124,18 @@ def build(
|
|||
|
||||
Keyword-only with defaults, so a caller who takes this as an API keeps a
|
||||
source-compatible call when a flag is added.
|
||||
|
||||
`proposed_at` defaults to `ingested_at` rather than independently to
|
||||
`DEFAULT_STAMP`: a caller naming one clock is naming "when this ran", not
|
||||
asking for two different clocks. Left independently defaulted, a caller who
|
||||
set only `--ingested-at` stamped the unsegmented concepts (which read the
|
||||
call's `ingested_at` directly) and left every segmented one -- which reads
|
||||
`segment.ingested_at`, the plan's `proposed_at` -- on `DEFAULT_STAMP`.
|
||||
Measured on K2: 11 of 629 concepts. Two independently-set clocks are still
|
||||
reachable by passing `--proposed-at` explicitly.
|
||||
"""
|
||||
if proposed_at is None:
|
||||
proposed_at = ingested_at
|
||||
if not segments:
|
||||
report = measure(inbox, bundle, ingested_at=ingested_at, profile=STRUCTURED_V1)
|
||||
_write_log(bundle, report, profile=STRUCTURED_V1)
|
||||
|
|
@ -218,8 +231,9 @@ def parse_args(argv: list[str] | None) -> argparse.Namespace:
|
|||
)
|
||||
build_parser.add_argument(
|
||||
"--proposed-at",
|
||||
default=DEFAULT_STAMP,
|
||||
help=f"written into every proposal. Default {DEFAULT_STAMP}, for the same reason",
|
||||
default=None,
|
||||
help=f"written into every proposal. Defaults to --ingested-at (itself {DEFAULT_STAMP} "
|
||||
"if that is also omitted), so naming one clock never leaves the other unset",
|
||||
)
|
||||
build_parser.add_argument(
|
||||
"--segments",
|
||||
|
|
|
|||
|
|
@ -372,6 +372,30 @@ def test_the_default_stamp_is_named_once(tmp_path: Path) -> None:
|
|||
assert cli.DEFAULT_STAMP[:10] in (bundle / "log.md").read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def test_ingested_at_alone_stamps_every_concept_the_same(tmp_path: Path) -> None:
|
||||
"""`--ingested-at` without `--proposed-at` must stamp every concept alike.
|
||||
|
||||
Measured on K2 (S7 acid test, F1): passing only `--ingested-at` stamped 11
|
||||
of 629 concepts -- the unsegmented whole-document concepts, which read the
|
||||
call's `ingested_at` directly. The 618 segmented concepts read
|
||||
`segment.ingested_at`, which is the PLAN's `proposed_at`, independently
|
||||
defaulted to `DEFAULT_STAMP` when the caller never set it. A caller naming
|
||||
one clock is naming "when this ran", not asking for two different clocks.
|
||||
"""
|
||||
inbox = inbox_with_subdirectories(tmp_path)
|
||||
bundle = tmp_path / "bundle"
|
||||
assert build(inbox, bundle, "--ingested-at", INGESTED_AT) == 0
|
||||
concepts = {
|
||||
name: body
|
||||
for name, body in tree(bundle).items()
|
||||
if name.endswith(".md") and not name.endswith("index.md") and name != "log.md"
|
||||
}
|
||||
assert concepts, "the fixture must produce concepts for this control to mean anything"
|
||||
for name, body in concepts.items():
|
||||
assert f"ingested_at: {INGESTED_AT}".encode() in body, name
|
||||
assert cli.DEFAULT_STAMP.encode() not in body, name
|
||||
|
||||
|
||||
# --- the one implementation ------------------------------------------------
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -112,6 +112,37 @@ def _copy_bundle(source: Path, target: Path) -> None:
|
|||
destination.write_bytes(path.read_bytes())
|
||||
|
||||
|
||||
def test_the_index_walk_excludes_a_linked_log_from_concept_navigation(tmp_path: Path) -> None:
|
||||
"""A linked `log.md` is bundle metadata, not a concept -- measured on K2 (S7 F2).
|
||||
|
||||
`link_log_in_root_index` (`corpus.py`) links a run's own log from the root
|
||||
index so a reader entering at `index.md` can reach it (`95eb271`). That link
|
||||
makes the log reachable by the same walk this instrument uses to enumerate
|
||||
concepts, and a walk that does not distinguish "linked" from "concept"
|
||||
counts it as a 630th concept on a 629-concept bundle -- exactly what the S7
|
||||
acid test measured, with the log then ranked and cut like real content.
|
||||
"""
|
||||
from llm_ingestion_okf.corpus import LOG_NAME, link_log_in_root_index
|
||||
from llm_ingestion_okf.profiles import SEGMENTED_OKF_V0_2
|
||||
|
||||
copy = tmp_path / "bundle"
|
||||
_copy_bundle(GOLDEN, copy)
|
||||
(copy / LOG_NAME).write_text("# Corpus run history\n\nN = 3\n", encoding="utf-8")
|
||||
link_log_in_root_index(copy, SEGMENTED_OKF_V0_2)
|
||||
index = (copy / "index.md").read_text(encoding="utf-8")
|
||||
assert "](log.md)" in index, (
|
||||
"the fixture must actually link the log for this control to mean anything"
|
||||
)
|
||||
|
||||
found = okf_consume.enumerate_concepts(copy)
|
||||
assert found == (
|
||||
"krav/1-1/foerste-krav",
|
||||
"krav/1-2/andre-krav",
|
||||
"veiledning",
|
||||
)
|
||||
assert not any(concept.endswith("log") for concept in found)
|
||||
|
||||
|
||||
def test_the_ref_covers_the_indexes_too_since_the_walk_reads_them(tmp_path: Path) -> None:
|
||||
# The docstring claims every byte that can reach a payload is inside the
|
||||
# ref. An index byte can: it decides which concepts are reachable at all.
|
||||
|
|
|
|||
|
|
@ -44,6 +44,7 @@ from typing import Literal
|
|||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
|
||||
|
||||
from llm_ingestion_okf.corpus import LOG_NAME # noqa: E402
|
||||
from llm_ingestion_okf.inbox import ( # noqa: E402
|
||||
ADJUDICATION_ADJUDICATED,
|
||||
ADJUDICATION_PROPOSED,
|
||||
|
|
@ -105,8 +106,16 @@ def _walk_index_tree(
|
|||
target = _join(parent, entry.target)
|
||||
if target is None:
|
||||
continue
|
||||
if target.rsplit("/", 1)[-1] == index_name:
|
||||
name = target.rsplit("/", 1)[-1]
|
||||
if name == index_name:
|
||||
pending.append(target)
|
||||
elif name == LOG_NAME:
|
||||
# A run's own log, linked for bundle navigation
|
||||
# (`corpus.link_log_in_root_index`) -- reachable, but metadata,
|
||||
# never a concept. Counted as one it inflated the K2 walk to
|
||||
# 630 against a 629-concept bundle and let the log rank and
|
||||
# cut like real content (measured, S7 F2).
|
||||
continue
|
||||
elif target.endswith(suffix):
|
||||
concepts.add(target[: -len(suffix)])
|
||||
return _byte_sorted(indexes), _byte_sorted(concepts)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue