open/llm-security
1
0
Fork 0
Code Issues Releases 1 Activity

test(llm-security): v8 Phase 5 step 3 - invert the unvendored-commons test

Browse source 
Vendoring commons v0.1.0 under scanners/commons/ falsified the premise of
`degrades gracefully when commons has not been vendored yet`: it asserted the
default DEFAULT_COMMONS_ROOT did not exist. It was the only red test after the
subtree add (2072/2073).

The graceful-empty contract it guarded is covered twice over by the
missing-artifact and invalid-JSON cases, which drive the same code path through
an explicit commonsRoot. So the replacement asserts the direction that is now
uncovered and matters more: a non-zero record count through the real default
root, no override.

That is the positive load-assertion Phase 5 step 4 requires. Every other gate we
have treats a commons load failure as indistinguishable from a legitimately
empty table, so a total loss of the vendored corpus would leave the suite green.
Proven red-capable by moving scanners/commons aside: the assertion fires by
name, not as an incidental TypeError elsewhere.

Suite 2073/2073.
This commit is contained in:
Kjell Tore Guttormsen 2026-08-10 20:52:04 +02:00
commit 44e5e39f67
1 changed files with 20 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

27
tests/lib/commons-loader.test.mjs
View file

@ -52,13 +52,26 @@ describe('commons-loader', () => {
assert.deepEqual(data, {});
});
it('degrades gracefully when commons has not been vendored yet (no commonsRoot override)', () => {
// Phase 4 hasn't run in this repo checkout: the default scanners/commons
// vendor path does not exist. The loader must not throw, and — since the
// commons location is never policy- or target-derived — this cannot be
// influenced by an on-disk .llm-security/policy.json either.
const data = loadArtifact('lexicon/injection-lexicon', { fallback: 'EMPTY' });
assert.equal(data, 'EMPTY');
it('resolves the default vendored root without an override, and loads a non-empty artifact', () => {
// Until Phase 5 step 3 this asserted the opposite — that the default
// scanners/commons path did not exist and the loader degraded to the
// caller's fallback. Vendoring commons v0.1.0 falsified that premise, and
// the graceful-empty contract it guarded is covered twice over by the
// missing-artifact and invalid-JSON cases above, which drive the same code
// path through an explicit commonsRoot.
//
// What is NOT otherwise covered is the direction that now matters: a
// commons load FAILURE is indistinguishable from a legitimately empty
// table to every other gate we have, so a total loss of the vendored
// corpus would leave the suite green. This asserts a non-zero record count
// through the real DEFAULT_COMMONS_ROOT — no override, the resolution path
// production actually uses.
const lexicon = loadArtifact('lexicon/injection-lexicon', { fallback: null });
assert.notEqual(lexicon, null, 'default commons root did not resolve — is scanners/commons vendored?');
assert.equal(lexicon.totals.families, 4);
assert.equal(lexicon.totals.patterns, 83);
const patterns = lexicon.families.flatMap(f => f.patterns);
assert.equal(patterns.length, 83, 'lexicon families carry fewer patterns than totals claims');
});
it('caches a loaded artifact per resolved path', () => {