docs(retrieval-gate): the round the gate had to resist the work it judges
README's row table and the `--real` paragraph, CLAUDE.md's gate entry, CHANGELOG under 0.10.1 (no new version, no tag), and the full report. The report carries the eight attacks before and after, the row table before and after, row 8's three-set measurement with its two granularities apart, the mutation table as properties rather than as code lines, and § 6 -- the three things this round does NOT close: row 3 can still be made green by an honest change that costs elsewhere (the gate is the unit, not the row), the judge's parser is still `consume`'s and is bounded by measurement rather than removed, and `SPECS_SHA256` digests the corpus's fields rather than the bundle's bytes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
149836c722
commit
23588e5ada
4 changed files with 335 additions and 16 deletions
65
CHANGELOG.md
65
CHANGELOG.md
|
|
@ -282,6 +282,71 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||
|
||||
### Fixed
|
||||
|
||||
- **The retrieval gate had to resist the work it judges: four of eight
|
||||
cheating attacks went through it, and they are closed (2026-09-19).** PM's
|
||||
checkpoint on `2c8296b` took rows 3, 5, 7 and 8 GREEN without one label
|
||||
becoming true or one concept ranking better. An eval written before the
|
||||
capability has one job beyond being red today, so the gate was repaired
|
||||
before anything is built against it. `src/` is untouched.
|
||||
- **Row 8 requires all three named sets** (`wiki-20`, `r761-sk2`,
|
||||
`vegnormal-32`) and is NOT RUN otherwise. It counted whatever `--real`
|
||||
gave it, so one set of three read `6 of 6 GREEN` — and this repository's
|
||||
own test asserted `(1, 1, GREEN)` for a single set. The numbers the run
|
||||
DID measure are still printed: a missing set must not cost the reader the
|
||||
set that was measured.
|
||||
- **Its headline is at QUESTION granularity**, and was `quoted + concept`
|
||||
over `quoted_units + concept_units` on the line above the detail saying
|
||||
the two are not summed. The three sets share no unit — a citation, a
|
||||
section title and a requirement number — so their sum is a number that is
|
||||
none of them.
|
||||
- **Rows 2 and 3 take their denominator from the pinned set, not the run.**
|
||||
At `k = 32` the fixtures declaring class b are delivered, and they used to
|
||||
leave the denominator: rows 1, 2, 3 and 6 all read green at once. A forced
|
||||
fixture that stops producing its declared class is a BROKEN PREMISE now,
|
||||
printed as one and counted against its row.
|
||||
- **Row 3 carries a known-positive.** With `--source-quota` off every
|
||||
printed reason is true — not a lie, an empty measurement — so a set may
|
||||
declare `source_quota_in_force` and the row is NOT RUN for it when the
|
||||
default and quota-off cuts deliver the same concepts. **The control's own
|
||||
premise was measured first and was false where it was first put:** over
|
||||
the five existing sets the two cuts deliver the SAME concepts (the quota
|
||||
is topped back up), 52 labels moving `source_quota_exceeded` → `below_k`
|
||||
with 0 deliveries changing. `set-quota.json` is the one set where the
|
||||
quota genuinely decides.
|
||||
- **Row 5 reads git for the half a registration cannot assert about
|
||||
itself.** Two files PM wrote in the moment came back `7 of 7 GREEN`. Three
|
||||
of its ten checks now read history: committed and unmodified, its commit
|
||||
is not itself a ranking change, and a ranking change landed AFTER it —
|
||||
the last being the one that cannot be self-attested. What git cannot prove
|
||||
(that nobody read the number first) is stated in the row.
|
||||
- **Row 7's roster is pinned apart from the list it names.** The bar is a
|
||||
share, so seven duplicate `k = 1` mutants read `18 of 20 GREEN` with the
|
||||
same two survivors. `MUTANT_ROSTER` and `MUTANT_COUNT` are separate
|
||||
constants, duplicates are refused, and the bar is the roster's length.
|
||||
- **The synthetic corpus is pinned like the sets** (`SPECS_SHA256`). A tuned
|
||||
corpus was caught by row 2's forced classes and not by a pin.
|
||||
- **`M14` closes PM's G9**: `hit = bool(hit_ids) and bool(confirmed)` is
|
||||
reached only by a delivery that still carries the citation and is no
|
||||
longer the concept file's bytes. It is felled and no production line
|
||||
changed — the term was observable and unobserved. The judge's
|
||||
independence is measured with it: index warmed BEFORE the patch, every
|
||||
unit a miss; index built UNDER it, every unit a hit. The gate never builds
|
||||
one under a mutation, and that is in `LIMITS`.
|
||||
- **Row 9 takes `--k2 SET SHA256 BUNDLE`** in this gate's own set shape, and
|
||||
a set of another size is refused as another set wearing K2's name. Without
|
||||
one it stays RED rather than NOT RUN: its denominator is known.
|
||||
- **Row 8 ran, against all three real sets**: **44 of 64 questions**, 7 of
|
||||
29 at citation granularity and 38 of 50 at concept granularity, 33 of 34
|
||||
misses class b. wiki (6 of 20) and r761 (7 of 7) reproduce PM's recorded
|
||||
figures exactly; vegnormal measures 31 of 43 citations where PM recorded
|
||||
32, a one-citation disagreement between two instruments over the same
|
||||
pinned bytes, stated and not resolved here.
|
||||
- Rows 1 and 6 go 9 of 9 to 10 of 10 (one added fixture, one added hit).
|
||||
Every other row is unchanged and the verdict is unchanged:
|
||||
`GATE RED: rows 3, 4, 5, 7, 8, 9`, exit 1, byte-identical over two runs.
|
||||
`mypy --strict` on the gate goes 8 errors to 0. Report:
|
||||
[`docs/2026-09-19-gjenfinningsgaten-motstand.md`](docs/2026-09-19-gjenfinningsgaten-motstand.md).
|
||||
|
||||
- **The conversion claim the content-accounting gate believes now comes from
|
||||
the RUN, not from the bundle's prose (0.10.1).** The previous round bound
|
||||
the claim to a pointer block, which closed the two forgeries PM had
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue