open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

docs(plan): record the end-to-end gate run and its one failure

Browse source 
The plan's Verification section held twelve gates that cross step
boundaries; /trekexecute checks per-step manifests and never ran them.
Run against the tree at 63565bd with the baseline at 770d8d4, before
any new commit, so the gates measure the delivered work and not
themselves.

Eleven passed. Gate 7 failed: one hunk in tests/test_profile.py, whose
closed field set on BundleProfile gained "segmentation" after
assumption A4 was falsified during execution and the edit ratified.
The gate's mechanism is byte-equality; the property it stands for was
measured directly instead — removing the member makes the test red, so
the assertion is a closed set with one more legitimate term, not a
loosened one. Recorded as failed rather than rewritten to pass.

Two mandatory known-positive controls fired: the zsh :e modifier ate a
path in git cat-file, and an unquoted for-loop did not word-split under
zsh. Both had produced a false absence that read as an answer.

Co-Authored-By: Claude <claude-opus-5>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-01 19:24:02 +02:00
commit 36af65c0c9
1 changed files with 36 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

36
docs/plan/concept-granularity.md
View file

@ -136,6 +136,42 @@ hidden: a consumer meets both mechanisms.
`tests/test_error_codes.py`.** The baseline pin forbade editing the
registry file. Migrate once the pin is released.
## The end-to-end gate run, and its one failure
`/trekexecute` verifies per-step manifests; the plan's `Verification` section
holds twelve gates that cross step boundaries and none of them had been run.
They were run 2026-09-01 against the tree at `63565bd` with the baseline at
`770d8d4`, before any new commit — measuring the tree afterwards would have let
the gates score the very work they exist to bound.
**Denominator: twelve gates, eleven passed, one failed.** The failure is gate 7,
"no existing test weakened": `git diff` against baseline returns one hunk, in
`tests/test_profile.py::test_a_profile_is_assembled_from_its_policies`, whose
closed field set on `BundleProfile` gained `"segmentation"`. The plan's
assumption A4 — that no existing test file would be edited — was falsified during
execution and the edit was ratified by the operator. The gate's mechanism is
byte-equality against baseline; the property it stands for is that no assertion
was loosened, and that property was measured directly rather than inferred:
removing `"segmentation"` from the set makes the test fail. The assertion is a
closed set with one more legitimate member. The gate stays recorded as failed
rather than rewritten to pass, because a gate edited until it is green has
stopped answering the question it was written for.
Two of the plan's mandatory known-positive controls fired, and both would
otherwise have produced a false absence. `git cat-file -e "$SHA:examples/..."`
lost its leading `e` to the zsh `:e` history modifier and reported the wrong
error for every path; an unquoted `for f in $NEW` did not word-split under zsh
and collapsed ten separate checks into one that could only ever fail. Neither
was visible in the result — each looked like an answer. The remaining controls
confirmed that `git diff` can find a change, `comm` can report a missing test id,
the `sed` count pattern can match, `pytest -rs` prints `SKIPPED` against an
injected mark, and `grep` returns 2 on a path error and 0 on a planted match.
Measured across the suite: 853 tests collected against a baseline of 728, 125 of
them in the ten new modules, none skipped, no baseline test id missing, `comm`'s
stderr empty, four golden directories byte-identical to baseline, and the run
path free of any model or subprocess reference across thirteen source files.
## Relation to the two orders
- `…2527032751` (identity) — executed. Form (c) built, form (a) measured as