open/llm-security
1
0
Fork 0
Code Issues Releases 1 Activity

chore(llm-security): pull commons subtree to v0.3.0, corpus 89 -> 90 cases

Browse source 
The subtree pull carries exactly one detection-data change: commons
converged `hybrid-xss:script-tag` on our open-tag-only form
(`<script\b[^>]*>`), dropping the `[\s\S]*?<\/script>` tail that was a
recall hole. Measured before the pull, not taken on their word: a
throwaway differential compared all 83 positions of the v0.3.0 lexicon
against the live source tables on source, flags, label and alias --
0 divergences, in order. The four vendored files were then re-hashed
against `git show v0.3.0:<file>` upstream; all four byte-identical.

Everything else in v0.2.0..v0.3.0 is additive: the CHANGELOG, the
divergence doc, spec text, the new §1.1 declaration schema, and one new
conformance case.

That new case is why the corpus tripwire moves. `manifest.count` is now
90 and `count_by_scope['lexicon/injection-lexicon.json']` is 84, because
`hybrid-xss__script-tag--src-no-close` gives the script pattern a SECOND
case. The tripwire fired on its own (actual 90, expected 89) rather than
being adjusted pre-emptively, so it is proven live this session.

`aliasMap.size` deliberately stays 83: the case-to-pattern relation is
now many-to-one, and only the alias map is a bijection. The header
comment says so explicitly, so the next reader does not "fix" the 83
into an 84.

Corpus: 84/84 passed, 6 not-applicable. Golden gate untouched (8/8) --
no table is built from the lexicon yet, which is the next commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017XDdiKC9ZXmcSUQ2m84s6y
This commit is contained in:
Kjell Tore Guttormsen 2026-08-11 14:05:07 +02:00
commit 21a52ac49d
1 changed files with 9 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

14
tests/lib/conformance-corpus.test.mjs
View file

@ -1,10 +1,14 @@
// conformance-corpus.test.mjs — runs the vendored llm-security-commons
// conformance corpus through the real scanForInjection() entry point.
//
// v8 Phase 5 step 3. The corpus is a cross-runtime contract: the same 83 cases
// v8 Phase 5 step 3. The corpus is a cross-runtime contract: the same 84 cases
// are run by llm-security and by llm-ingestion-pipeline-security, so a table
// swap in step 4 that quietly changes what we detect shows up here as a named
// failing case rather than as nothing at all. Normative semantics live in
// failing case rather than as nothing at all. Note that 84 cases exercise 83
// patterns: commons v0.3.0 gave `hybrid-xss:script-tag` a second case
// (`--src-no-close`) when it converged on our open-tag-only form, so the
// case-to-pattern relation is many-to-one and only the alias map is a
// bijection. Normative semantics live in
// scanners/commons/spec/conformance-corpus.md; where this file and that
// document disagree, the document wins.
//
@ -21,7 +25,7 @@
// error: §1 means *we tried and could not*, not-applicable means *this
// question was never addressed to us*. Such a case MUST still be
// enumerated, MUST NOT count as a pass, and MUST NOT leave the
// denominator. `83/83 passed, 6 not-applicable` is the shape.
// denominator. `84/84 passed, 6 not-applicable` is the shape.
//
// §4 A runtime MUST reject a `match` it does not implement rather than
// degrade to a weaker comparison. Hence the explicit throw.
@ -107,7 +111,7 @@ const notApplicableCases = manifest.cases.filter((c) => !isApplicable(c));
/**
* §1.1's anti-narrowing constraint, as a gate rather than an intention.
*
* Measured, not assumed: narrowing DECLARED_TABLES to `[]` turns all 89 cases
* Measured, not assumed: narrowing DECLARED_TABLES to `[]` turns all 90 cases
* not-applicable and leaves this suite GREEN with zero cases run — precisely
* the exit §1.1 forbids. "Narrowing is visible as a code change" describes a
* reviewer, not a gate, so the floor below is derived and asserted instead.
@ -218,7 +222,7 @@ describe('commons conformance corpus — exact-within-scope through scanForInjec
assert.deepEqual(onDisk, [...manifest.cases].sort(),
'case directories on disk disagree with manifest.cases');
assert.equal(onDisk.length, manifest.count, 'manifest.count disagrees with its own case list');
assert.equal(manifest.count, 89, 'corpus size changed — re-verify before adjusting this number');
assert.equal(manifest.count, 90, 'corpus size changed — re-verify before adjusting this number');
});
it('maps every one of our injection labels onto a lexicon pattern_id', () => {