pre-bash-destructive.mjs held the last three probe (e) hits: the
pipe-to-shell and T8 rule names and one comment. The names are now
concatenated from fragments and the comment is a description; the
string the hook prints is unchanged, and tests/golden/reference-run.json
pins it. Patterns untouched.
Applied by the operator (git apply): the installed plugin's path guard
blocks Edit/Write on this file and asks the user to make the change.
Measured: hook stderr for a pipe-to-shell and a T8 command is
byte-identical before/after (cmp); av-surface a=0 a2=0 b=0 c=0 d=0 e=0;
suite 2269 / 2263 pass / 0 fail / 6 skip, exit 0; golden unchanged
(109/7/4, reference run 61/61).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The AV section added two suppressed findings to README.md itself; the
remaining 58 (WARNING 61/100) did not move. Re-measured on a fresh clone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
.llm-security-ignore: MEM/SIG/UNI rules added only for by-design
findings, one explaining line each — the vendored conformance corpus,
the Cyrillic confusables in injection-patterns.mjs and the homoglyph
test inputs, the changelog's quoted homoglyph, the grade-F posture
fixture, and the miner names in the supply-chain blocklist. Not
suppressed on purpose: MEM on CLAUDE.md (the file a poisoning would
target) and AST/TRG (no by-design findings once the ast-scan fixtures
left the disk). Header prefix list completed (14 scanners) and the
"150" replaced by the measured count.
Self-scan, fresh clone of HEAD (node scanners/scan-orchestrator.mjs):
before: BLOCK 80/100, 77 findings (1C 9H 29M 2L 36I), 362 suppressed
after: WARNING 61/100, 58 findings (0C 5H 15M 2L 36I), 381 suppressed
without the ignore file: BLOCK 95/100, 439 findings (unchanged)
On `.` (working tree incl. local-only files): BLOCK 80/100, 80
findings -> WARNING 64/100, 61 findings.
README:
- Self-scan: the measured numbers replace "0 findings (ALLOW)".
- The demo commands materialize the tree to a temp dir (the form the
demo README uses); no evil-project-health/ path left in README.
- New "Antivirus false positives" section + TOC entry: what the repo
contains, why a classifier can still flag it, restore from
quarantine, Microsoft's submission form, Microsoft's own advice
against exclusions, a sparse-checkout recipe. Every Defender claim
checked against learn.microsoft.com; the recipe run live against
Forgejo (partial clone supported; 285 of 677 files on disk). The
meaning of "!ml" is not claimed — Microsoft documents "!" suffixes
only as internal indicators.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
scanners/lib/git-clone.mjs accepted only github.com, so the README's
Forgejo example (and /security scan on any Forgejo, Codeberg or self-hosted
URL) failed validation; scan.md and plugin-audit.md also routed only
https://github.com/ to the clone path.
Chose generic HTTPS over adding one host to an allowlist because there is
no documented security reason for the host check. Read before choosing:
git log --follow (import + E12 only), the file's comments, CHANGELOG v2.4.0
("GitHub repo URL support", feature framing), security-hardening-guide §2
and §7 (the sandbox is the defence against filter/smudge drivers), and
review-2026-06-20, which lists "vsix-fetch (HTTPS host allowlist)" as a
protection but no host check for git-clone. What protects a clone is the
OS sandbox, GIT_SANDBOX_CONFIG and GIT_SANDBOX_ENV, for every host alike.
Measured: with GIT_SANDBOX_ENV, git reads 0 config lines outside a repo;
without it, osxkeychain comes from Xcode's system gitconfig. So no stored
credential is offered to an unknown host.
The shape stays strict: https only, no userinfo, host starts and ends
alphanumeric (no leading -), optional port, exactly owner/repo, no query
or fragment. SSH stays GitHub-only: ssh uses the user's own keys and
~/.ssh/config, which the git environment does not isolate.
Tests (red first): Forgejo and any-host accept 2 failing -> green; guards
for http, userinfo, leading -, extra path, query, ext:: and file:: and
non-GitHub SSH hold. The old "rejects non-GitHub URL" case still passes
(it has no owner/repo) and is renamed to say so. Regex timed linear
(< 1 ms at 200k chars). Live: a sandboxed clone of
https://git.fromaitochitta.com/open/llm-security.git exits 0 at 5208420,
cleaned up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The five Python taint fixtures under tests/fixtures/ast-scan/ move into
tests/helpers/payload-trees.mjs as the `ast-scan` tree, written to a temp
dir by the test like the three S1 trees. av-surface probe (d) gains the
directory: 5 -> 6 trees.
PM decision (S3 order): creds-net.py is os.environ -> requests.post, the
exfiltration shape AV classifiers are trained on; one rule, "no
payload-shaped runnable file on disk", is easier to defend than an
exception. The .py files match no SIG rule and are stored as plain lines.
Deviation from the order: it named three files (sinks, creds-net, scope);
the directory holds five (also reassign.py, sentinel.py). Gating the
directory means all five move.
Measured: (d) red first, d=1 (5 files) of 6 trees, then 0. sha256 of all
five on-disk files taken before git rm; payload-trees.test.mjs asserts
them, mutation-checked (one byte in creds-net.py -> red, restored
byte-identical). ast-taint-scanner.test.mjs 12/12 with the materialized
tree, both before and after the files were removed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The seven-category signal lists in agents/skill-scanner-agent.md move to
knowledge/skill-threat-patterns.md § Detection Rules. The agent keeps its
procedure, severity table, verdict logic and output format, plus a pointer,
and now says to report an unreadable knowledge file instead of scanning from
memory (a remembered subset reports clean for rules it never applied).
Chose the existing file over a new one because every command that invokes
the agent (scan, audit, clean, plugin-audit) already passes
<plugin-root>/knowledge/skill-threat-patterns.md explicitly; a new file would
have needed four command edits to reach the agent at all.
Why (v8.1.0 AV surface): a clean SKILL.md was quarantined as
Trojan:Script/Wacatac.H!ml; a quarantine on agents/*.md breaks the installed
plugin, not just a clone. Move measured lossless: the only line-level
differences are the two intended rewrites and headings.
Also in this commit, rewritten as descriptions or with a <shell>
placeholder for the interpreter (no technique removed):
- posture-assessor-agent: hook-coverage item and override phrases
- deep-scan-synthesizer-agent: example decoded message
- commands/red-team.md: scenario table cell
- knowledge/*.md: 15 runnable download-into-shell one-liners, the
"Decodes to" line first; fenced examples keep their exact shape with
<shell>, prose and tables become sentences.
Probe (e): 8 -> 3 (the three left are hook-script lines behind the
path guard). knowledge: 17 -> 2 (attack-scenarios.json, the red-team
simulator's input; left on purpose). claude plugin validate . passes;
agent frontmatter untouched, all six agents parse.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
New probe (e) over agents/**, commands/** and hooks/** — what a Claude Code
session loads at the user's end, where a quarantine breaks the installed
plugin, not just a clone. It fails on a contiguous literal matching the SIG
reverse-shell/webshell/miner rules, a download piped into a shell, or a
base64 decode piped into a shell. Raw text, comments included, per line, so
every hit names file:line.
knowledge/** is measured by the same rule and reported in the diagnostic as
its own number, not gated in S3.
Measured red before any change (v8.1.0 S3, 2026-09-22):
e = 8 hits in 6 of 36 files; knowledge = 17 hits of 22 files.
No SIG rule hit anywhere in the four directories; validated against built
known positives (all three families match).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
v8.1.0 S2. security-assessment.md (two blobs, one split, plus the decoded
exfil command) and the base64 pipe-to-shell one-liner in
knowledge/skill-threat-patterns.md now describe the payload rather than
reproduce it. The entropy path-suppression test builds its 84-char blob
from 12-char fragments, so no contiguous decodable blob sits on disk.
av-surface: b 5->0. a=0 a2=0 b=0 c=0 d=0 - the probe is green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
v8.1.0 S2. examples/poisoned-claude-md/fixture/ (CLAUDE.md with a base64
pipe-to-shell blob, plus an agent file) is now fixture.archive.json in the
same format as the demo archive; run-memory-poisoning.mjs materializes it
into a temp dir and deletes it on exit. README shows materialize-then-scan.
payload-trees.test.mjs asserts byte identity for both archives.
av-surface: b 6->5, d 1->0. Walkthrough output identical before/after
(6 pass, 0 fail, 18 findings).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
v8.1.0 S2. examples/malicious-skill-demo/evil-project-health/ (7 files,
30 Unicode Tag chars, a base64 exfil blob) is now one archive,
evil-project-health.archive.json: rot13 text, every codepoint above U+007E
stored as a number, sha256 of each retired file recorded. materialize.mjs
writes it to a temp dir (CLI prints the path); run-demo.sh materializes
and deletes it itself; the six scanner tests that scanned the tree use it.
payload-trees.test.mjs asserts byte identity (mutation-checked).
av-surface: b 8->6, c 1->0, d 2->1. Demo 13/13 before and after. All
scanners report identical findings except git-forensics: it used to scan
this repository's own history (21 findings, none about the demo) and now
reports skipped in a temp dir, which git.test.mjs already accepts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The S0 rule fired on 'http' anywhere in decoded text and flagged an SVG
data URI (xmlns) in the hyperframes-like skill-scan fixture - markup, not
a command. PM decision for S2: the decoded text must start with a command
word or contain a pipe into a shell or $(. Measured before any blob was
removed: 8 blobs/7 files -> 8 blobs/6 files; SVG out, one split-payload
fragment in security-assessment.md newly caught, all 7 command blobs kept.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
v8.1.0 AV surface, session S1. The three poisoned fixture trees
(signature-scan/poisoned, memory-scan/poisoned-project, trigger-scan/poisoned)
are deleted from disk and materialized into a temp dir by the new
tests/helpers/payload-trees.mjs. SIG-matching strings are assembled from
fragments, the zero-width carrier comes from String.fromCodePoint, and every
file carries the sha256 of the retired on-disk bytes;
tests/helpers/payload-trees.test.mjs asserts the materialized trees are
byte-identical (mutation-checked: one changed byte fails it).
Inline payload literals in signature-scanner, signature-scanner-custom-rules
and e2e/scan-pipeline are fragmented the same way; the literal U+200B in
attack-simulator, auto-cleaner-rce and auto-cleaner-traversal is replaced by
String.fromCodePoint(0x200B).
av-surface: a 3->0, a2 3->0, c 5->1, d 5->2, b 9->8 (webshell-b64 blob gone).
What remains (c=1, d=2, b) is under examples/** or is (b), both S2.
Suite 2261 / 2252 pass / 3 fail (av-surface b, c, d only) / 6 skip.
Golden output identical before/after (109/7/4, 61/61).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(a) keeps .mjs/.js out of its denominator, so contiguous payload literals in
test sources were ungated. a2 walks tests/**/*.{mjs,js} (tests/golden/**
excluded) and matches the webshell/reverse_shell/cryptominer SIG rules on raw
text. Measured red: a2=3 (scan-pipeline e2e, signature-scanner,
signature-scanner-custom-rules) of 116 test sources.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tests/av-surface.test.mjs walks git ls-files and fails on four surfaces a
clone or the plugin cache puts on a user's disk: (a) own SIG payloads
(webshell/reverse_shell/cryptominer) in a runnable file type, (b) base64
blobs decoding to a shell command, (c) Tag/zero-width/bidi carriers in text
files outside the conformance corpus, (d) known payload trees.
Red on purpose (order S0): it is the failing test for S1-S3. Measured on
this tree: a=3 b=9 (8 files) c=5 d=5. Chosen definitions are documented in
the file header: .mjs/.js/.cjs/.json and scanners/commons/** are outside (a)
so the untouchable signature tables stay out of a gate that must reach
zero; binaries are outside (c) because decoding PNG/WOFF2 as UTF-8 yields
the codepoints by chance.
PLAN.md is local-only, so .gitignore now names it. No production code, no
fixtures, no signature tables touched; golden baseline unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase 6 of the approved v8.0.0 plan. HEAD was 45 commits past v7.8.3 with
every version-bearing file still reading 7.8.3 and no v8 tag anywhere.
Version sync: plugin.json, package.json, README badge, CLAUDE.md header.
CHANGELOG: [Unreleased] becomes [8.0.0] - 2026-09-18. It covered 3 of the
user-visible commits since v7.8.3; the other 18 fix/feat/refactor commits
are now recorded from their commit bodies - the commons extraction (five
tables, scanners/commons/ subtree, knowledge/signatures.json removed),
conformance spec 1.1, the golden-gate coverage relabel, and eight fixes
(script-tag recall, whole-table ReDoS gate and its own 8-of-45 blind spot,
Berry and nested lockfile-v1 misses, per-occurrence strip attribution,
legacy OpenAI keys, compileRules coercion, SIG self-flag, SARIF tags).
Counts verified against the loaded modules: 83 injection patterns, 19
secret shapes, 7 SIG rules.
SECURITY.md supported-versions table said 7.3.x Active; now 8.0.x Active,
7.0.x - 7.8.x best-effort. README gets the 8.0.0 row; CLAUDE.md gets the
v8.0.0 highlights and the test floor moves 2045+ to 2250+.
Tests: 2253 / 2247 pass / 0 fail / 6 skipped, exit 0, measured on HEAD
before the edits. The post-edit full run had 3 red, all timing ceilings,
under load average 60 from unrelated jobs on the machine; each passes
alone (460 ms of 1000, 370 ms of 2000, 2127 ms) and doc-consistency is
45/45 alone. The diff is docs and version strings only.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
D4-C, order 20260821T091412Z-1428311869-from-.claude. Measured by .claude
against the org 21.08: this repo has the org's most mature deprecation
PRACTICE and no stated policy. An adopter cannot read a CHANGELOG row and know
it applies next time -- what D4 measures is what can be EXPECTED, not what
happened last time.
Form copied from open/portfolio-optimiser SECURITY.md "Deprecation Notice
Period"; the numbers are this repo's own. Both floors are kept because the
release count alone is toothless here: measured from the README release table,
7.5.0 and 7.6.0 shipped a day apart (2026-05-05 -> 2026-05-06), so "at least
one minor release" without a day floor would permit a next-day removal. 30 days
is comfortably holdable -- the one precedent ran 112 days (v7.3.0 released
2026-05-01 per commit 4637139; v8.0.0 still unreleased at 2026-08-21).
The precedent is worded as staged, not shipped: git describe reports
v7.8.3-43-ge97c232 and the removal sits under CHANGELOG [Unreleased], so
claiming v8.0.0 "removed" them would have been a claim about an untagged
release. Only the four env-vars are cited; CHANGELOG dates riskScoreV1's
@deprecated to v7.0.0 while README says v7.3.0, and that disagreement should
not be load-bearing in a policy.
No promise the project cannot hold: it is stated as a notice period and not an
SLA, with no backporting, and the one exception for security-critical removal
is named explicitly. README's Project scope now points at the section so an
adopter arriving from the deprecation bullet lands on the rule.
Docs only; the Iron Law does not fire. doc-consistency gate 45/45 pass, exit 0.
Full suite unchanged vs HEAD: 2253 tests / 2247 pass / fail 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7TAHJfxBoCm9YwhMWm6y5
D2b, order 20260821T041145Z-8516817669-from-.claude. Measured by .claude
against the org: llms.txt was missing in 23 of 23 org repos (probe verified
both ways in the same call chain: README.md -> 200, a nonexistent path -> 404).
The operator decision of 21.08 creates it in 6 repos -- the catalog plus the 5
artifacts with their own run path -- and deliberately not in the 12 plugin
repos, whose entry surface is already uniform.
The reader is an AI agent already standing in this repository that needs one
deterministic place to find what this artifact is and how it is started. Not
crawlers: measurements show the major agent crawlers skip /llms.txt and read
the HTML directly, so that would not be a reason that holds.
Follows the llmstxt.org convention: H1, one blockquote summary, then H2
sections holding lists of markdown links, with "Optional" naming the links an
agent may skip. Deliberately minimal -- llms.txt is a convention, not a
ratified standard, and a W3C proposal from June 2026 diverges from the
Answer.AI spec, so a format change should stay cheap.
The install block is quoted byte-identical from README.md's first code block
(verified with diff against README.md:67-68), not rewritten. A rewritten
variant would be a second source of truth, not an entry point.
Docs only; the Iron Law does not fire. Full suite unchanged vs HEAD:
2253 tests / 2247 pass / fail 0, exit 1 from the known jetbrains-parser
after()-hook assertion, measured identically on a stashed HEAD tree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7TAHJfxBoCm9YwhMWm6y5
buildRules() and toSARIF() wrapped a multi-mapping f.owasp string
(e.g. 'MCP03, MCP06', emitted by mcp-live-inspect.mjs and
ide-extension-scanner.mjs) as a single-element tags array instead of
splitting it, silently dropping the second OWASP mapping in SARIF
output. Added owaspTags() helper; test asserts a multi-entry tags
array for both rule and result properties.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WbQmoLxcAAsJAkFxBeCx6z
Census (AAA+ round 5) measured 4 FAIL + 1 unmeasured across L55, L206, L207,
L402, L403: strings that look like clickable URLs but are template leftovers
(https://repo-url resolves nowhere; https://github.com/org/repo 404s). L55 is
in an [!IMPORTANT] callout selling the plugin's primary CLAUDE.md-poisoning
defense, so its one example command couldn't be run as written.
First example in each command block now uses a real, resolving self-demo URL
(https://git.fromaitochitta.com/open/llm-security), making the README a
working quickstart instead of an illustration. Remaining variants use an
unambiguous <owner>/<repo> placeholder that doesn't read as a real URL.
L14 (SSRN, 403) and L224 (sandbox-exec docs, 503) are untouched — bot-blocked
or transiently unavailable is not evidence of a dead link.
Order: 20260818T114340Z-186999443-from-.claude
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JCinQGjXauCrGBqKoFNs4u
25 H2 sections, no TOC — long README friction fix per AAA+ B-axis order 32.
Anchors validated against actual headings (see toc_check.py known-positive
control); flat list, mirrors config-audit/README.md pattern.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
new RegExp(pattern, 'i') never throws when pattern is a truthy non-string
(e.g. an object) — it ToString-coerces it first. The truthy-only guard
(`!rule.pattern`) let such a rule through as a real, compiled RegExp,
bypassing the try/catch meant to drop malformed rules. Worse than a silent
drop: `new RegExp("[object Object]", "i")` is parsed as a character class
over o/b/j/e/c/t/space, so the "dropped" rule instead becomes a
near-universal false-positive matcher. Same path for the built-in
commons-backed ruleset and the operator's sig.custom_rules_path (both
route through compileRules).
Fix: require typeof rule.pattern === 'string' before compiling. Verified
the golden dump pins no rule that exists only because of this coercion —
it regenerates byte-identically after the fix.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019iWrdLSVgRhgPzTB29rQGD
Scanning this repository with the SIG scanner produced 7 findings, 4 of them
on our own detection data: scanners/commons/CHANGELOG.md and
scanners/commons/signatures/malware-signatures.json. The ruleset that describes
xmrig and webshells is, byte for byte, a document containing those strings, so
the engine matched it as malware. EXCLUDED_PATH_RE already carried
knowledge/, tests/, docs/ and node_modules/ for exactly this reason; the
vendored commons arrived in v8 Phase 5 (bbada84) without being added.
One alternation branch closes it. Tests first: two cases added to
describe('signature-scanner: path exclusions'), both verified red against the
real scan() entry point before the regex changed.
Stated plainly, because it is a real cost and not a technicality: the branch is
`scanners\/commons` behind the existing `(^|\/)` prefix, so it matches that
two-segment path ANYWHERE in a target's relative path, not only at its root. A
webshell planted at vendor/scanners/commons/shell.php in a hostile repository is
therefore invisible to SIG. The second new test asserts that blind spot
deliberately, so it can never be discovered by accident. It is accepted because
anchoring at ^scanners/commons/ would miss the same payload one directory
deeper while re-opening the self-flag whenever the plugin is scanned from a
parent directory. TRG, AST, entropy and supply-chain still read these files;
only SIG identity-matching is blinded.
scanners/lib/supply-chain-data.mjs is NOT excluded. Its finding is a true
positive against real blocklist data.
Measured before: 7 findings. After: 3 (2 on STATE.md, 1 on
supply-chain-data.mjs). signature-scanner.test.mjs 23/23; custom-rules + e2e
54/54; golden-baseline 8/8 with suite-counts.json untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SBJVYzwch63Rpk1hii1cNM
Fifth and last consumer swap of v8 Phase 5 step 4. The seven known-bad-identity
signatures stop living in knowledge/signatures.json and are built from the
vendored commons artifact signatures/malware-signatures.json instead.
Measured before the swap over all seven positions -- id, family, severity,
pattern, description, provenance, key order, and recompilation identity under
the engine's unconditional `i` flag: zero divergences over 56 checks, in order.
The commons copy was extracted from this repository's own file at b0de0ca and
had not drifted.
knowledge/signatures.json is REMOVED rather than left in place. Keeping it would
have left two files spelling one table with nothing gating the drift, and its
golden `file:` pin would have gone on passing while pinning bytes no scanner
reads -- a gate reporting success without running. The pin is replaced by a
walked-module anchor over SIGNATURE_RULES, which is strictly stronger: the pin
covered the bytes on disk, the walk covers what `new RegExp` made of them.
Golden diff was exactly that and nothing else: 7 ADDED, 1 REMOVED, 0 CHANGED
(102/7/5 -> 109/7/4), each added source verified equal to the recompiled commons
pattern.
compileRules() moves into the new lib module and is exported, so the built-in
ruleset and the operator's sig.custom_rules_path path keep one implementation
rather than two copies of the defaulting logic.
Coverage by construction, not by memory: the probe table in the scanner test is
asserted against the LOADED ruleset, so a rule commons adds cannot arrive
without an end-to-end probe. Mutation of the vendored JSON fires in three
directions -- under-match (xmrig alternative dropped) reddens two scanner tests
plus golden; over-match (webshell rule widened to a bare `shell`) reddens the
clean-fixture false-positive probe plus golden; reorder reddens the declared-
order test plus golden.
Loud failure is contract: an unresolvable commons writes one line to stderr
rather than silently disabling known-malware detection, and never throws.
Suite 2247 / 2241 pass / 6 skipped / 0 fail. suite-counts.json untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0151x4FVg9Mn55C2LvHLpHKo
The 19 fixed credential shapes in pre-edit-secrets.mjs were regex literals;
they now come from signatures/secret-egress.json in the vendored commons via
a new scanners/lib/secret-egress.mjs. Policy-injected custom patterns (entries
20+) are unchanged and still appended by the hook.
Measured before the swap, not assumed: all 19 positions compared for order,
name, regex source and flags, plus recompilation identity, against the literal
table sliced out of the module text. Zero divergences. Commons had reported
the same result; that was their measurement, so this one was run anyway.
STATE's expectation that the golden gate would go red on both table records
and file sha256 was wrong: pre-edit-secrets.mjs is in neither PINNED_FILES nor
WALKED_MODULES, so the table had no golden coverage at all and the swap moved
nothing. Rather than leave the vendored data with only behavioural coverage,
secret-egress.mjs joins WALKED_MODULES — walked, not pinned, since it inlines
no regex of its own. Golden diff was 19 ADDED, 0 CHANGED, 0 REMOVED, each
source byte-identical to the pre-swap literal; re-blessed. suite-counts.json
untouched.
Tests: coverage is derived from the loaded table, so an entry commons adds
cannot arrive without an end-to-end probe. All 19 now block through the real
hook and are asserted by label, which also pins the ordering contract (a
Bearer-wrapped JWT must report as the header). Mutating the vendored JSON
fires in both directions plus reorder: under-match (AKIA quantifier) reddens
3 hook tests + golden; over-match (Anthropic key truncated to its prefix)
reddens the false-positive probe + golden; moving the JWT entry ahead of the
Bearer entry reddens the ordering test.
Suite 2231 tests / 2223 pass / 6 skipped. The two parallel-run failures
(pre-compact size-cap, benchmark) pass alone — the known timing flakes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MGMv5ZTUhVzZtCCwRrNZG5
catalog found commit 0b795b1 sitting on `feat/add-ms-ai-architect` in the
marketplace forge -- a monorepo-path docs commit
(`plugins/llm-security/README.md`) that was only partially carried across
the repo split. Their read was that the CVE reference and
`transfer.fsckObjects` were rewritten into main independently while the
Windows half was never brought over.
Verified here before acting: `AppContainer`, `Windows Sandbox` and
`AppArmor` each grep 0 in our README and 0 at the published v7.8.3 tag,
while `CVE-2024-32002` and `transfer.fsckObjects` are present. Partial
transfer confirmed, not a deliberate drop.
Recovers the three missing pieces:
- the per-platform sandbox matrix, including the Ubuntu 24.04+ AppArmor
caveat that explains why bwrap fails there
- the Windows options table (Windows Sandbox / Docker Desktop / WSL2 /
AppContainer) with isolation level and requirements
- the note on why Node's --permission model does not apply: it restricts
fs access within the Node process and does not sandbox child
processes, and git is a separate OS process
The layer table already in main is kept as-is -- it postdates the stranded
commit and is more accurate than the version there. Only the one-line
Windows sentence is replaced.
All 8 external links verified live (HTTP 200) rather than copied forward
on trust.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJxU3xuwfMq8W1mxtiGhLk
Removes this repo's GOVERNANCE.md and repoints its one reference to the
canonical file in repo-standard, both in the same commit so no
LINK-INTERNAL-MISSING window opens between the two.
Verified here rather than taken from the census message: the canonical
URL returns HTTP 200, md5 3df3603325d3d6937fd560c3f67b5a5d, 131 lines --
matching org-ops' measurement exactly -- and a diff against our copy
shows only the generalising "plugin"/"marketplace" -> "repository"/
"organisation" wording, same 131 lines, no content removed.
Our copy was byte-identical to the stated baseline
(md5 736fc9d6af84fbd83c9cc7f860d8c8b7). Grep confirms README.md:5 was
the only reference in this repo, matching the measured site list.
Nothing gates on the file for us: GOVERNANCE.md is in required_files for
the catalog class only, and this repo's class is plugin.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJxU3xuwfMq8W1mxtiGhLk
The v8.x-A whole-table gate times every exported pattern against a corpus
of 8 hand-written units and asserts its own coverage -- but the assertion
`covers every exported pattern` guards the pattern LIST, not the input
corpus. A pattern is only measured if some unit happens to carry its
leading literal; otherwise it fails on the first character and reports
green having measured nothing.
Measured on the pre-swap tables: 37 of the 45 prefix-bearing patterns
were never reached, including BOTH quadratic hybrid-xss rows this gate
was believed to cover. `<script ` and `<iframe ` appear in no unit, so
the two rows commons independently measured as quadratic ran their
literal-prefix check and stopped. Same defect class as all of v7.8.2:
reported success without running.
Hand-writing 37 more units does not fix it -- it re-arms the same trap at
the next pattern. Class 3 derives each attack unit from the pattern's OWN
literal prefix, so coverage is a function of the table rather than a list
someone must remember to extend. 64KB rather than the 512KB read cap for
the class-1 reason: a quadratic pattern met at 512KB stalls the run for
minutes instead of failing it.
Proven to fire, both directions, against the vendored file:
- gate written first, pre-swap: RED, naming script-tag 1429ms and
iframe-src 1161ms against a 150ms budget (exit 1)
- post-swap: GREEN, 17.7ms for all 45 probes (exit 0)
- vendored JSON mutated back to [^>]*: golden AND ReDoS gates both exit 1
- vendored JSON corrupted: golden exit 1, conformance 3 fail
- restored: all green
Clean-table margin at 64KB is ~700x: worst legitimate pattern 1.66ms.
Carried with the commons v0.4.3 subtree pull, which is what makes the
gate passable. v0.4.0 was the tag commons announced; v0.4.1-v0.4.3 came
after and touch no data table -- lexicon 0.8.0 and secret-egress
0.3.0/19 are identical across all four -- so v0.4.3 was taken for the
conformance manifest correction (302625e) they sent separately.
Golden re-blessed after a post-by-post diff: exactly 2 changed records,
both [^>]* -> [^><]*, 0 added, 0 removed, reference run 61/61 unchanged.
The file-sha256 layer did NOT move, contrary to the note in STATE: it
pins scanners/lib/injection-patterns.mjs, which has held no literals
since be14867. The vendored lexicon is covered by the regex layer only.
The script-tag tripwire pinned the old form and fired correctly. Updated
to the v0.4.x form and widened to the iframe row, which had no tripwire
while it was quadratic -- which is why nobody had named it.
Full suite 2193 pass / 6 skipped. The one red is the documented
pre-compact size-cap timing flake; passes alone (exit 0), as do
attack-simulator and the gate itself.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KJxU3xuwfMq8W1mxtiGhLk
7ce0ba7 docs(carriers): the third verdict exists, and publishing an alias is what takes it away
302625e fix(conformance): the tag carrier has no output: label, and our blocker claimed it did
fe5e6b2 docs(conventions): the merge button is off for a reason, and the reason now lives in the repo
2d86151 fix(divergence): our own iframe number read 3x low, and the reported cause was not the cause
daa7ba4 release(0.4.0): two values moved by two mechanisms, and the difference is the release
2eee7e1 feat(lexicon): both unbounded rows narrow to [^><]*, and the mechanism is new here
d467324 feat(signatures): the staleness we disclosed is closed by reading the module, not the message
4187715 docs(divergence): our own form has a number now, and it is quadratic
0e765a0 docs(security): the attack surface here is data, so the report route had to say where a wrong entry gets fixed
d96fbbf docs(divergence): the span row had one witness; llm-security measured five more
git-subtree-dir: scanners/commons
git-subtree-split: 7ce0ba706cadd032ec3c16622dcfdb5ce4dc32d6
Closes the gap STATE has been carrying since e1511f9. Section 1.1's runtime
behaviour has been correct since then -- the third verdict is real, the
declared set is one constant with two uses -- but what we PUBLISHED was a
console.log summary: the right facts in a format only its author could
parse, which is most of what section 1.1 exists to prevent. Commons
shipped a shape for it in v0.3.0
(schema/conformance-declaration.schema.json 0.1.0), so the artifact can
now exist.
Every field is counted from the run rather than restated. The cases record
their own verdict as they execute, keeping `failed` (ran and disagreed)
apart from `error` (could not run) on exactly the distinction section 1
turns on. `commons_commit` is read out of the subtree-pull subject in our
own history rather than transcribed into a constant that would drift at
the next pull, and it refuses to publish a coordinate it cannot determine
-- a fabricated commit is worse than no declaration. The artifact is
gitignored: a committed declaration keeps asserting what was true once,
and nothing makes it wrong out loud when it stops being.
Validated once against the vendored schema with a real 2020-12
implementation: VALID, and the validator proven discriminating by six
negative controls it rejected (dropped zero-count, unknown key,
out-of-enum source, non-integer count, missing enumeration, malformed
case id). Continuous validation would mean a Python dependency in a suite
that has none, so what stays is the cheap half that actually drifts -- the
two key sets, asserted exactly.
TWO DEFECTS FOUND BY MUTATING THIS GATE, both in its own first draft:
1. It lived in `after()`. Measured on Node 25.8.2: an assertion that fails
in an after hook prints under "failing tests" and marks the suite red,
but leaves `fail 0` and exit code ZERO. `npm test` and CI would have
read a falsified declaration as green. The gate against "reports
success without running" was itself reporting success without running.
It is now a test, declared last, and the verdict-count assertion is
what guards the ordering that makes "last" meaningful.
2. Nothing tied the PUBLISHED `declared_tables` to the runner's constant.
Substituting a literal list left every other assertion green, because
they all read the constant rather than what was published -- so
`declaration_source: derived-from-runner` could be a lie with no code
change to point at. Now asserted identical.
Seven mutations, all exiting non-zero: dropped zero-count, falsified
not_applicable, unpublished field, hand-maintained tables, lied-about
source, and a hardcoded `passed` combined with a genuine case failure.
Published this run: 90 total, 84 passed, 0 failed, 6 not-applicable,
0 error, at commons 4641a7b (v0.3.0). Full suite 2192 pass / 0 fail /
6 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017XDdiKC9ZXmcSUQ2m84s6y
Third consumer swap of v8 Phase 5 step 4, after codepoints and OWASP_MAP,
and the last one with a behavioural gate behind it. The 83 regex literals
leave injection-patterns.mjs; the four arrays are now built in
scanners/lib/injection-lexicon.mjs from the vendored
lexicon/injection-lexicon.json and re-exported unchanged, so every
consumer sees the same published surface.
Behaviour-preserving by measurement, not by intent. The proven recipe ran
in order: a differential over all 83 positions (regex source, flags,
label, aliases.llm_security) found 0 divergences BEFORE anything changed;
the golden dump was then diffed post-for-post rather than read as a 9000-
character assertion, and the ONLY changed record was the sha256 of
injection-patterns.mjs itself -- 83 regex posts, 7 table records and all
counts identical. That single file digest is the diff a swap MUST produce,
so the baseline was re-blessed rather than silenced.
Two deliberate departures from the two earlier swaps:
FAILURE IS LOUD. codepoints and owasp-map fail silently on purpose: an
empty codepoint table weakens normalization, an empty OWASP map mislabels
a report. An empty injection table is different in kind -- scanForInjection
returns found:false for every input, and the UserPromptSubmit scan, the
MCP output scan and the pre-compact scan all go blind while reporting
success. That is precisely the v7.8.2 defect class, which bit this plugin
four times in one release. An unresolvable commons therefore writes one
line to stderr naming the disabled capability. It still does not throw:
hooks run per-tool-call, and a module-load throw breaks the tool call
instead of degrading the scan. The warning is suppressed for an explicit
commonsRoot, so tests and dev checkouts stay quiet and the line keeps
meaning something.
ENTRIES COMPILE DEFENSIVELY. commons is vendored data, not code. An
uncompilable pattern or unknown flag would throw inside new RegExp at
module load -- in a hook. Malformed entries are dropped instead, the same
call owasp-map.mjs makes for a non-array value.
Gates proven by mutating the vendored JSON in BOTH directions, five ways,
all firing: re-adding the script-tag tail commons dropped (golden 1,
lexicon 2, corpus 1), dropping a critical pattern (2/1/3), stripping the
`m` flag off a spoofed-header anchor (2/1), adding a pattern commons never
published (2/2/85), and removing commons outright -- which produced the
stderr line, four empty tables and 5 red rather than a green suite over
zero patterns. Lexicon restored byte-identical after each.
Full suite 2191 pass / 0 fail / 6 skipped (2184 -> 2197).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017XDdiKC9ZXmcSUQ2m84s6y
The subtree pull carries exactly one detection-data change: commons
converged `hybrid-xss:script-tag` on our open-tag-only form
(`<script\b[^>]*>`), dropping the `[\s\S]*?<\/script>` tail that was a
recall hole. Measured before the pull, not taken on their word: a
throwaway differential compared all 83 positions of the v0.3.0 lexicon
against the live source tables on source, flags, label and alias --
0 divergences, in order. The four vendored files were then re-hashed
against `git show v0.3.0:<file>` upstream; all four byte-identical.
Everything else in v0.2.0..v0.3.0 is additive: the CHANGELOG, the
divergence doc, spec text, the new §1.1 declaration schema, and one new
conformance case.
That new case is why the corpus tripwire moves. `manifest.count` is now
90 and `count_by_scope['lexicon/injection-lexicon.json']` is 84, because
`hybrid-xss__script-tag--src-no-close` gives the script pattern a SECOND
case. The tripwire fired on its own (actual 90, expected 89) rather than
being adjusted pre-emptively, so it is proven live this session.
`aliasMap.size` deliberately stays 83: the case-to-pattern relation is
now many-to-one, and only the alias map is a bijection. The header
comment says so explicitly, so the next reader does not "fix" the 83
into an 84.
Corpus: 84/84 passed, 6 not-applicable. Golden gate untouched (8/8) --
no table is built from the lexicon yet, which is the next commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017XDdiKC9ZXmcSUQ2m84s6y
4641a7b release(0.3.0): a detection pattern changed value — that is new here
1482c0b feat(schema,spec): give the §1.1 MUST a shape, since v0.2.0 shipped it without one
25a2cf9 feat(conformance): the witness case, and the derivation rule that had no room for it
c1b2385 fix(lexicon): converge script-tag on its source — re-extraction, not revision
6f79a6e fix(lexicon,docs): the script-tag row reversed — commons is now the sole diverger
git-subtree-dir: scanners/commons
git-subtree-split: 4641a7b5184047460e3f10038b615a61e7a4ac21
Bare/unquoted legacy OpenAI keys (no label assignment, no Bearer prefix)
slipped past the pre-write secret-detection hook. Added a pattern anchored
on the T3BlbkFJ base64 "OpenAI" watermark (vendor-documented shape),
avoiding the collision-prone bare sk-+48alnum form. Failing tests first,
full suite green (2184/0/6).
The originally planned source for this fix — porting two entries from
commons' secret-egress.json — turned out to be a false premise: that file
is a byte-identical copy of this hook's own table, not a superset. The two
missing names existed only as prose in commons' conformance/manifest.json,
describing a different repo's (the guard's) unpublished Python table.
gcp-service-account-json was measured NOT to be a gap (already covered by
the existing PEM-block pattern); openai-api-key-legacy was the one real
gap, closed here with a locally-authored pattern rather than an invented
"port". Commons notified via coord-send that their secret-egress.json
(count: 18) is now stale.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PYJX35KLH3rpS6pi7LHj8u
Commons v0.2.0 (532d70d) grew a MUST: a runtime claiming conformance must
declare the set of commons data files it implements and publish that set with
its result. The corpus went 83 -> 89; the six new cases are scoped to
signatures/active-content.json, which this runtime does not implement. Under
1 they would be six permanent failures; under 1.1 they are not-applicable, a
third verdict distinct from 1's error.
Published shape: 83/83 passed, 6 not-applicable. The six are enumerated by
name, reported as skipped rather than passed, and stay in the denominator.
The declaration is DERIVED from the same constant the runner uses to accept a
scope (DECLARED_TABLES), so it cannot drift from what the suite actually runs
- this is what was promised to commons in reply 20260811T104628Z.
Anti-narrowing was NOT enforced by construction, contrary to the claim in the
reply. Measured: setting DECLARED_TABLES to the empty set turns all 89 cases
not-applicable and leaves the suite GREEN with zero cases run - exactly the
exit 1.1 forbids. "Visible as a code change" describes a reviewer, not a gate.
Closed with a derived floor rather than a second hand-maintained table list
(which would be the parallel declaration we promised not to keep): a commons
table whose aliases name llm_security has registered this runtime as a
consumer per 3.1, and a registered consumer that stops declaring the table is
withdrawing a published claim. The universe of tables comes from the manifest;
membership comes from each table's own aliases. Measured: only the injection
lexicon names us, so the floor is one table and the other three carry no
obligation.
Mutation-proven, all six firing:
declaration -> [] green, 0 cases -> 1 fail (the defect above)
declaration -> wrong table 7 fail
over-declare an unimplemented one 6 fail
count_by_scope 83 -> 82 1 fail
drop a case from manifest.cases 2 fail
strip our alias registration 85 fail
Suite 2173 -> 2181 (+2 gates, +6 not-applicable), 2174 pass, 0 real failures.
The one red under parallel load was pre-compact-scan size-cap, a known timing
flake: 358 ms alone against a 1000 ms cap, 1886 ms under load. Budget untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uj6iB8uUUUFHLgAya1hZt7
532d70d release(0.2.0): the contract gained a MUST, so the tag has to move
946f51d fix(active-content,conformance): cite line numbers per commit — they do not resolve at the pin
bdcb1f1 feat(conformance): ship the six active-content cases; the id space already existed
807c0d4 feat(spec): add not-applicable, so a single-runtime table stops reading as 7 defects
a1578e6 fix(conformance): record the guard's internal-surface position on _LEX_PAYLOADS
4d351d2 fix(mapping): state that three of four OWASP maps have no production consumer
f082a91 fix(lexicon,docs): retract the claim that the guard's port cites severity.mjs
git-subtree-dir: scanners/commons
git-subtree-split: 532d70d5ed2f9b23a8efad760ef490356cf52ada
Second consumer swap of step 4. OWASP_MAP stops being a hardcoded constant in
severity.mjs and is built from the vendored commons artifact
mapping/owasp-map.json by a new scanners/lib/owasp-map.mjs, re-exported from
severity.mjs so the published surface (which the golden gate walks as
severity:OWASP_MAP) is unchanged.
Scope is one of the four maps commons publishes, and the omission is measured,
not incidental. OWASP_MAP has a production consumer: owaspCategorize() reads it
as the per-scanner fallback, and that reaches real report output through
output.mjs's owasp_breakdown. OWASP_AGENTIC_MAP, OWASP_SKILLS_MAP and
OWASP_MCP_MAP have none - every reference tree-wide is a test or a golden
artifact - so they stay source literals, the same call already made for
cyrillic_confusables in the first swap. Porting them would move data no runtime
reads into the load path.
Measured byte-likeness before the swap, all four taxonomies: same 16 prefixes,
same insertion order, same code arrays. Loadable verbatim, unlike the injection
table.
Content preservation proven the same way as the codepoint swap: the golden dump
differs in exactly one record, the sha256 of severity.mjs, which changes by
construction when a table leaves the file. All 83 regex records and all 7 table
records including severity:OWASP_MAP are byte-identical; reference-run.json
unchanged at 61/61. patterns.json re-blessed for the file digest only.
New property, not just preservation: the golden gate now pins the vendored
commons data transitively for this table too. Mutation-proven in both
directions - changing one code value and deleting a whole prefix each turn
three independent gates red (golden table digest, the new owasp-map gate by
name, and the pre-existing severity behaviour tests).
Entries are validated rather than trusted: commons is vendored data, and a
value that is not an array of strings would be spread straight into
owaspCategorize's category list, so a malformed entry is dropped. Graceful-empty
on an unresolvable commons, matching commons-loader's contract - severity.mjs is
on the import path of output.mjs and every orchestrated scanner, so a load throw
would abort a scan rather than degrade it.
Suite 2164 -> 2173, all green.
First consumer swap of step 4. ZERO_WIDTH_CHARS (5), the Unicode Tag range,
BIDI_CHARS (9) and HOMOGLYPH_MAP (28) stop being hardcoded constants in
unicode-scanner.mjs and string-utils.mjs and are built from the vendored
commons artifact codepoints/carriers.json by the new lib/codepoints.mjs.
Started here rather than at injection-patterns, which the plan ordered first:
that table is the one table that cannot be loaded verbatim (the
hybrid-xss:script-tag divergence is directional, and loading the lexicon as-is
would reverse the 90f576f recall fix). The codepoint tables were measured
byte-equal to the source constants BEFORE the swap - same members, same
values, same insertion order on HOMOGLYPH_MAP - so they load verbatim.
Proof the swap is content-preserving: the golden dump differs in exactly one
record, the sha256 of string-utils.mjs, which changes by construction when a
table leaves the file. All 83 regex records and the
table:string-utils:HOMOGLYPH_MAP digest are byte-identical, and
reference-run.json is unchanged at 61/61. patterns.json is re-blessed for the
file digest alone.
The gate is proven red-capable against the SUBJECT, both directions:
- dropping U+00AD from the vendored zero_width table fails the new
codepoints gate by name, twice;
- altering one homoglyph value reddens the golden table digest AND a
behavioural homoglyph test.
That second direction is a property the swap creates rather than preserves:
the golden gate now transitively pins the vendored commons data, where before
it pinned a source literal and a commons mutation was invisible to it.
NOT ported: commons carries cyrillic_confusables (13), and unicode-scanner.mjs
declares a set by that name - but nothing reads it. The homoglyph-mixing
detector tests isCyrillic(cp), the whole U+0400-U+04FF block. Loading it would
move dead data into the load path, so the dead const stays where it is and is
recorded instead. The recorded v8.x-B i/x drift between that set and the
lexicon class is therefore latent, not live. commons' private_use table has no
constant behind it here at all.
Graceful-empty is kept deliberately: codepoints.mjs is on string-utils'
import path and hooks import string-utils in fresh per-tool-call processes, so
a module-load throw would break the tool call rather than degrade the scan.
The loud half is the test, which asserts exact per-table counts through the
real default commons root - the same shape as the lexicon load-assertion.
Drive-by, unavoidable: the deleted JSDoc carried the "~25 entries" claim for a
28-entry table (v8.x-C). It needed a re-bless of the same file digest this
swap already forces, so it closes here at no extra cost.
Suite 2158 -> 2164, all green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X7XEEFrAJsREqa9N4tpfm8
Vendored commons v0.1.0 carries a cross-runtime conformance corpus. Until now
it was measured by a throwaway script, which makes 83/83 a claim rather than a
gate - and step 4 swaps the very tables it constrains, so the measurement has
to survive into that step or it protects nothing.
Comparison is exact-within-scope per spec section 4: every listed finding must
be raised and no other lexicon finding may be. Findings are named by commons
pattern_id, which scanForInjection() does not carry - it returns our labels.
Section 3.1 permits a runtime registered in the lexicon's aliases object to
compare through it, and we are registered. Measured first, not assumed: the
map is a total bijection, 83 labels to 83 ids, no duplicates, family membership
agreeing throughout. Nothing here restates a pattern's id, severity or label,
so nothing here can drift from the lexicon.
Deliberately NOT done: adding an id field to our 83 table entries. It would
change the source file the golden gate pins by sha256, forcing a re-bless in
the middle of a behaviour-preservation measurement, and duplicate what step 4
does anyway when the table itself starts loading from commons JSON.
Case discovery is driven by manifest.cases and cross-checked against the
directories on disk, because section 1 requires every case to run and a
deleted case dir would otherwise shrink the gate silently. An unimplemented
match or scope throws rather than skips (section 4). Input bytes and sha256 are
both verified before scanning - two fixtures carry characters invisible on
screen.
Proven red-capable in both directions by mutating the subject, not the harness:
neutering one pattern failed exactly override__disregard; widening one to
[aeiou] failed 82 cases on extra findings. Source restored byte-identical after
each.
Suite 2158, 85 new. The one red in the parallel run is the known
pre-compact-scan size-cap flake (366 ms alone, 1060 ms under load).
Vendoring commons v0.1.0 under scanners/commons/ falsified the premise of
`degrades gracefully when commons has not been vendored yet`: it asserted the
default DEFAULT_COMMONS_ROOT did not exist. It was the only red test after the
subtree add (2072/2073).
The graceful-empty contract it guarded is covered twice over by the
missing-artifact and invalid-JSON cases, which drive the same code path through
an explicit commonsRoot. So the replacement asserts the direction that is now
uncovered and matters more: a non-zero record count through the real default
root, no override.
That is the positive load-assertion Phase 5 step 4 requires. Every other gate we
have treats a commons load failure as indistinguishable from a legitimately
empty table, so a total loss of the vendored corpus would leave the suite green.
Proven red-capable by moving scanners/commons aside: the assertion fires by
name, not as an incidental TypeError elsewhere.
Suite 2073/2073.
Two of the three confirmed v8.x-A evasions are closed; the third (attribute
padding) stays open by operator decision and is documented, not silently left.
Recall fix. hybrid-xss required a closing </script>, so `<script>alert(1)` and
`<script src=x.js>` both passed scanForInjection() with found: false, while the
closed form returned high. A src= tag has no body to close in the first place.
The opening tag alone is the signal, and matching it is strictly linear: one
negated-class run whose excluded character is its own terminator, so there is
no backtracking surface that could re-introduce v7.8.3 #24.
ReDoS gate. The #24 test covered six html-obfuscation patterns against the two
shapes that defect was found on. A catastrophically backtracking regex added
anywhere else in the four tables would have failed no test at all - which is
why #24 had to be found by hand. The new gate times every exported pattern and
asserts its own coverage, so it cannot be narrowed silently.
It has two classes because the blowup shapes need opposite inputs, and because
a synchronous RegExp.test() cannot be interrupted: an exponential pattern met
with a 512KB input would HANG the run rather than fail it. Class 1 uses a
28-char ambiguous-run ladder, where exponential costs ~1s and anything sane
costs microseconds; on a hit the 512KB sweeps refuse to run. Class 2 uses the
hook's real read cap to catch the polynomial #24 class. Proven to fire: an
injected exponential regex was named by label and the hang guard held.
Measured clean: worst single pattern 17ms, full large sweep 218ms.
Golden gate went red as expected and was re-blessed - diff is exactly two
lines, the regex source and the source-file digest. The 61-payload reference
run is byte-identical, so no showcase behaviour moved.
Suite 2073/2073 (was 2063; +5 recall tests, +5 gate tests).
Still open, unchanged: all 7 bounded HTML patterns evade on >256 chars of
padding, in two positions - before the attribute and inside the style value.
Wider than STATE recorded, which named only aria-label.
Advisor review on the prior commit (69cad7c) caught a real detection-kill
vulnerability before push: reading `commons.root` from the SCANNED
TARGET's .llm-security/policy.json let a hostile cloned repo redirect
llm-security's own detection corpus to an attacker-supplied (empty)
one, with graceful-empty fallback making the substitution silent — a
substitutive override, unlike sig.custom_rules_path's additive one.
Dropped the policy import entirely; commons location is this plugin's
own concern, resolved only from __dirname or an explicit test/dev
override, never from policy or the scan target.
Also fixed two issues the review surfaced:
- Default vendor path was repo-root `shared/`, which package.json's
`files` allowlist (bin/, scanners/, knowledge/) would never publish —
moved under scanners/commons/, inside the directory that actually
ships. Same defect class as 2fe2915 (green dev checkout, empty
detection tables once installed).
- Cache keyed success/failure together, so the first caller's
`fallback` shape (e.g. []) leaked to a second caller expecting a
different shape ({}) on the same missing artifact. Cache now stores
a load-failed sentinel and returns each caller's own fallback.
Loaded artifacts are also deep-frozen, since the cache hands out one
shared object by reference to every caller.
New/changed tests cover all four: a simulated hostile-target policy
file is ignored, the failure-cache no longer cross-contaminates
fallback shapes, and mutating a loaded artifact throws.
Golden baseline unchanged; full suite 2063/2063.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QAYkRaBXT6tmWXTQAi1ZBg
Thin, sync-read JSON artifact loader for the future vendored
llm-security-commons subtree, modeled on signature-scanner.mjs's
loadRules()/loadCustomRules() pair: process-cached, graceful-empty
fallback on any read/parse error, and policy-extensible via a
`commons.root` policy value (mirrors sig.custom_rules_path).
Unit-tested now against a local fixture — Phase 4 (commons repo
creation, gated on the operator creating the Forgejo remote) hasn't
run yet, so the default `shared/` vendor path doesn't exist in this
checkout. That "not vendored yet" case is itself asserted: the loader
must degrade to the caller's fallback, not crash.
Not wired to any consumer yet (that's Phase 5 step 4, table-by-table
behind the golden gate). No CLI/hook/scanner-visible behaviour exists
to document. Golden baseline unchanged; full suite 2063/2063 (2053 +
10 new).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QAYkRaBXT6tmWXTQAi1ZBg
The `coverage` block never looked at the 61 hook invocations. It probed every
payload STRING against every injection regex in-process, so `47/83` meant
"if you threw all 61 strings at all 83 patterns, 47 would match" - not "the
reference run exercised 47". The pre-bash-destructive payloads never reach
injection-patterns at all, yet their strings were in the probe set and could
mark a pattern exercised.
That number was asserted as reference-run coverage in three places that
instruct a future session: tests/golden/README.md, the STATE golden-gate
section, and the generator's summary line. In a repo whose v7.8.2 lesson was
"the check reported success without running", a figure that measures one
thing while labelled another is the same defect wearing a different hat.
Relabelled rather than re-measured - the probe still honestly bounds the gate
(an unreachable pattern is one the corpus cannot protect under ANY
attribution), it just has to say what it is:
coverage.kind = 'static-reachability', with the caveat in a `note` field.
patternsExercised -> patternsReachable
uncoveredPatterns -> unreachablePatterns
tablesExercised -> corpusContains (a payload CONTAINS a homoglyph; it
does not say the run folded one)
The gate now pins both the `kind` and the note, so the honest label cannot be
dropped quietly by a later edit.
Also surfaced the gap the old wording hid: the four OWASP maps have NO
behavioural coverage - they are scanner-side and no hook in this corpus
reaches them. They are precisely the tables the dump was widened to cover, so
the table digest is their only protection. Stated in the README next to the
47/83 line, where a reader was previously left to infer the reference run
backed them.
Flakiness check for the new gate (61 sequential spawns, ~12s, the most
process-heavy file in the suite, added to a suite npm test runs concurrently):
three consecutive full runs, 2053/2053, 0 fail. The three known
timing-sensitive files did not destabilise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
Records the reference artifacts Phase 5 swaps will be measured against, and
the gate that reads them. No extraction yet: this is the "before" picture,
and it had to land first or every later comparison would be confounded.
Typed test/ rather than feat/ deliberately - the only production change is
one widened export; the rest is gate, artifacts and generator.
Written failing-first (7 red on missing artifacts), then generated.
Three layers, because the plan's "assert .source/.flags of every regex" is
necessary but not sufficient:
1. regex records - .source/.flags off the COMPILED object. After a swap a
pattern is new RegExp(jsonString, flags), so the plan's named hazard
(JSON backslash-doubling on 83+18 regexes) is visible here and nowhere
else. Source-text comparison cannot see it.
2. table records - key/value digests. Most of what Phase 4 moves is not a
regex at all: HOMOGLYPH_MAP (x3, AS-IS), the typosquat tokens and the
four OWASP maps are char->char and string->string data. A regex-only
dump is blind to a broken homoglyph swap, i.e. to the bulk of the
payload. Operator decision: widen the dump.
3. file records - sha256 of the five moving-set sources. This dissolves
STATE's open question (how to enumerate every regex): it is complete by
construction, covering inline regexes in function bodies that no export
walk reaches, with no JS parser in a zero-dep repo. A lexical count
would have pinned a lie - severity.mjs scores 4 "regexes" that way and
exports none.
Both layers were proven to fire, not assumed to: mutating one HOMOGLYPH_MAP
entry reddens the table layer, and widening an inline regex inside
decodeHexEscapes (unreachable by any export walk) reddens the file layer.
HOMOGLYPH_MAP is now exported from string-utils.mjs. That export is a source
change the plan already flags as a surface hazard ("private tables become
loaded"), so it is pre-paid here rather than confounding the before/after.
Reference run: the 61 showcase payloads through the real hook entry points,
sequentially - array order is semantic and the plan forbids key-sorting, so
concurrency is removed rather than sorted away. 61/61 match expectation,
which also settles the plan's open assumption that payloads.json expectations
match current behaviour. Coverage is recorded, not assumed: 47/83 patterns,
with the other 36 listed by key so the gate never implies coverage it lacks.
Suite counts are per-file, each file run alone - a total is unattributable,
and the three known timing-sensitive files flake only under concurrency.
2045 pass / 0 fail across 91 files, matching the pre-existing count exactly.
The gate's own file is excluded (it reads the artifact the run produces) and
that exclusion is named in the artifact.
Suite: 2053/2053, 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
Closes Phase 3 (B11) of the v8.0.0 plan. Three parts, all with the failing
test written first.
riskScoreV1 removed. scanners/lib/severity.mjs drops riskScoreV1() and its
SEVERITY_WEIGHTS_V1 table - @deprecated since v7.0.0, kept for diff/comparison,
zero callers in code or tests (re-verified, not taken from the plan). The v1
weights are recorded in CHANGELOG so an old score stays re-derivable. riskScore
(v2) is untouched; a test pins that one critical still lands in the 70-95 tier
and that 50 lows score below it, which is exactly the case v1 collapsed to 100.
Posture category 12 no longer keys off an identifier name. The check was
/TRIFECTA_MODE/i over the session-guard source, which measured what a constant
was CALLED rather than whether enforcement was configurable. With the env-var
gone, that regex would have dropped every correctly-migrated project from PASS
to PARTIAL - the gate punishing the migration it exists to encourage. It now
matches getPolicyValue('trifecta', 'mode', ...) and still accepts a pre-v8
vendored guard reading the old env-var, because a third-party project carries
its own hook copy and is equally configurable either way; the evidence line
says which of the two was found. The PARTIAL finding recommended setting an
env-var that v8 ignores; it now names the policy key. The grade-a fixture hook
moves to the policy-era form.
Two never-implemented env-vars deleted from the docs. LLM_SECURITY_SCR_OFFLINE
(ci-cd-guide) and LLM_SECURITY_OFFLINE (supply-chain-attack example) were
documented as OSV.dev / npm-audit kill-switches. No code has ever read either -
verified by grep across scanners, hooks and scripts, which finds them only in
markdown. A promised kill-switch that does nothing is worse than a documented
absence: it is trusted precisely when the run is meant to be air-gapped. The
docs now say there is none and that egress must be blocked at the network
layer. The LLM_SECURITY_AUDIT_* wildcard is narrowed to the one real key.
Docs. Migration section in README + CHANGELOG with the env-var -> policy-key
table, the detection commands (env + shell rc + .envrc + workflows), and the
explicit warning that a removed variable is now INERT rather than an error -
which is the failure mode that loses a project its configuration silently. The
hardening-guide env table splits into surviving vars and a removed-vars
migration table; its "promote to block" runbook named two variables that no
longer exist. Also swept: CLAUDE.md hook table, scanner-reference, ci-cd-guide,
both lethal-trifecta example docs, mitigation-matrix, injection-research.
Test counts in README/CLAUDE.md synced 2034 -> 2045.
Suite 2045 tests, 0 fail (2039 + 4 posture-trifecta + 2 riskScoreV1). The two
known parallel-load flakes did not recur this run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BB4vXvwvtW4dxbPRd6vsez
