Kjell Tore Guttormsen
8d990e06d3
Records the reference artifacts Phase 5 swaps will be measured against, and
the gate that reads them. No extraction yet: this is the "before" picture,
and it had to land first or every later comparison would be confounded.
Typed test/ rather than feat/ deliberately - the only production change is
one widened export; the rest is gate, artifacts and generator.
Written failing-first (7 red on missing artifacts), then generated.
Three layers, because the plan's "assert .source/.flags of every regex" is
necessary but not sufficient:
1. regex records - .source/.flags off the COMPILED object. After a swap a
pattern is new RegExp(jsonString, flags), so the plan's named hazard
(JSON backslash-doubling on 83+18 regexes) is visible here and nowhere
else. Source-text comparison cannot see it.
2. table records - key/value digests. Most of what Phase 4 moves is not a
regex at all: HOMOGLYPH_MAP (x3, AS-IS), the typosquat tokens and the
four OWASP maps are char->char and string->string data. A regex-only
dump is blind to a broken homoglyph swap, i.e. to the bulk of the
payload. Operator decision: widen the dump.
3. file records - sha256 of the five moving-set sources. This dissolves
STATE's open question (how to enumerate every regex): it is complete by
construction, covering inline regexes in function bodies that no export
walk reaches, with no JS parser in a zero-dep repo. A lexical count
would have pinned a lie - severity.mjs scores 4 "regexes" that way and
exports none.
Both layers were proven to fire, not assumed to: mutating one HOMOGLYPH_MAP
entry reddens the table layer, and widening an inline regex inside
decodeHexEscapes (unreachable by any export walk) reddens the file layer.
HOMOGLYPH_MAP is now exported from string-utils.mjs. That export is a source
change the plan already flags as a surface hazard ("private tables become
loaded"), so it is pre-paid here rather than confounding the before/after.
Reference run: the 61 showcase payloads through the real hook entry points,
sequentially - array order is semantic and the plan forbids key-sorting, so
concurrency is removed rather than sorted away. 61/61 match expectation,
which also settles the plan's open assumption that payloads.json expectations
match current behaviour. Coverage is recorded, not assumed: 47/83 patterns,
with the other 36 listed by key so the gate never implies coverage it lacks.
Suite counts are per-file, each file run alone - a total is unattributable,
and the three known timing-sensitive files flake only under concurrency.
2045 pass / 0 fail across 91 files, matching the pre-existing count exactly.
The gate's own file is excluded (it reads the artifact the run produces) and
that exclusion is named in the artifact.
Suite: 2053/2053, 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
46 lines
2.9 KiB
Markdown
46 lines
2.9 KiB
Markdown
# Golden baseline — v8 Phase 5
|
||
|
||
Reference artifacts recorded **before** any commons extraction, so that each
|
||
table-by-table swap in Phase 5 can be proven behaviour-preserving (or rolled
|
||
back). Generated and checked by one code path:
|
||
|
||
```bash
|
||
node scripts/golden-baseline.mjs # check only, exits 1 on drift
|
||
node scripts/golden-baseline.mjs --write # (re-)bless
|
||
node scripts/golden-baseline.mjs --write --suite # also refresh suite counts (slow)
|
||
```
|
||
|
||
The gate is `tests/lib/golden-baseline.test.mjs`.
|
||
|
||
## The artifacts
|
||
|
||
| File | What it pins | Why that layer exists |
|
||
|------|--------------|-----------------------|
|
||
| `patterns.json` | `.source` + `.flags` of every RegExp reachable from the walked modules' exports | After a swap a pattern is `new RegExp(jsonString, flags)`. The plan's named hazard — JSON backslash-doubling — is visible **only** on the compiled object. |
|
||
| ↳ table records | key/value digest of `HOMOGLYPH_MAP`, `TYPOSQUAT_SUSPICIOUS_TOKENS`, `SEVERITY`, the four OWASP maps | Most of what Phase 4 moves is not a regex. A regex-only dump is blind to a broken homoglyph or OWASP-map swap, i.e. blind to the bulk of the payload. |
|
||
| ↳ file records | sha256 of the five source files in the moving set | The completeness layer, complete **by construction**: it covers regexes inlined in function bodies (`NAMED` at `string-utils.mjs:291`, the BIDI/tag/PUA ranges at 357–404) that no export walk can reach. |
|
||
| `reference-run.json` | the 61 showcase payloads through the real hook entry points, plus coverage | Byte identity over a corpus that trips a handful of patterns would prove almost nothing, so coverage is recorded and uncovered patterns are listed by name. |
|
||
| `suite-counts.json` | per-file pass/fail, each file run alone | A total is unattributable, and `npm test` runs files concurrently where three timing-sensitive files flake. Running each alone is the only reproducible recording. |
|
||
|
||
## Why there is no regex enumerator
|
||
|
||
The obvious design — parse the sources and enumerate every regex literal — was
|
||
rejected. It needs a JS parser this zero-dependency repo does not have, and a
|
||
lexical approximation is contaminated by comments and division (`severity.mjs`
|
||
scores 4 "regexes" that way and exports none). The two-layer split — export
|
||
walk for what the scanners actually use, file digest for everything else —
|
||
answers the same question without a parser.
|
||
|
||
## During a swap
|
||
|
||
A diff here means the swap changed observable behaviour. **Roll the swap back.**
|
||
`--write` is for deliberately re-blessing a change you have already decided is
|
||
correct, not for making the gate quiet.
|
||
|
||
## Known coverage gap
|
||
|
||
The reference run exercises 47 of 83 injection patterns. The other 36 are
|
||
listed by key under `coverage.uncoveredPatterns` — a swap that breaks one of
|
||
those is not caught by the reference run, only by the pattern dump. That is an
|
||
honest limit of the corpus, not a bug in the gate; closing it means growing
|
||
the conformance corpus (Phase 5 step 5).
|