The corpus was blocked on a decision nobody had to take. The 13 divergent
lexicon patterns were measured on witness inputs -- an attribute run padded
past 256 characters, an interior '<', an unclosed <script> -- and the corpus
payloads contain none of those shapes. Run through both runtimes' public
entry points, all 83 patterns produce identical lexicon finding sets, the 13
included. No winner picked, because the question was never reachable from
these inputs.
Measured at each runtime's entry point (scanForInjection() at b0de0ca,
scan_output(source=OUTPUT) at 0bf0729), never at a rebuilt regex table --
the layer mistake the divergence document already had to retract once.
- conformance/<case>/{input.txt,expected.json} x83, plus manifest.json
- spec/conformance-corpus.md, normative: bytes not text, id-only findings,
exact-within-scope, and observed_out_of_scope as evidence not expectation
Verified by a harness that does not share the generator's knowledge: reads
only the case directories, re-runs both runtimes, checks every field
including digests -- 83 cases, 0 failures. Severity agrees with what commons
publishes 83/83. Deleting the middle third of each input breaks 76 of 83
expectations; the 7 survivors are the shortest payloads, a weak mutation
rather than a weak fixture, and are recorded as such.
Scope is 83 and not 94 for a different reason than expected: the 11
non-lexicon convertible cases have no ratified cross-runtime finding id, and
writing them would mint a contract unilaterally in the same stroke as the
tag. Named in manifest.json under scope_planned.
Docs corrected in place rather than edited away: extraction-plan and
lexicon-port-divergence both claimed the 13 blocked the corpus.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WhXDL82FRrQWedEmUg12Pj
The last missing data file. Seven known-bad-identity signatures over four
families - webshell, reverse_shell, cryptominer, hacktool - reproduced
verbatim from llm-security/knowledge/signatures.json at b0de0ca, key order
included. The file was generated from the parsed source rather than typed,
and provenance pins the source's byte length (2494) and SHA-256 so the
claim is checkable rather than asserted.
Note the family spellings: reverse_shell with an underscore, and
cryptominer rather than miner. The working note this file was planned from
had both wrong. They are policy keys - the engine filters on them and
interpolates them into every finding title - so a rename is a breaking
change, which is exactly why the table was read instead of recalled.
The rules were the easy half. The substance is engine_behaviour_not_data,
which draws the line between the table and the runtime around it. No rule
carries a flags field, because the engine compiles every pattern with `i`
unconditionally at signature-scanner.mjs:48 - so a consumer compiling these
case-sensitively silently under-matches all seven, and the dialect block
records that where a reader will hit it. Also engine, not data: matching
against five decode variants rather than raw bytes, the enabled-families
policy filter, per-file rule dedup, custom-rule merging, and a loader that
defaults four missing fields instead of rejecting a rule.
Two limits are stated as evidence limits rather than left implied. Seven
signatures are not malware coverage; a clean SIG result is not "no
malware", and the seed runtime's own header calls the table deliberately
tight. And three of the seven match on names - xmrig, mimikatz,
meterpreter - so a document discussing those tools matches. The seed
runtime hides that by excluding knowledge/, tests/, docs/ and
node_modules/, which is scan scoping and does not travel with the table.
Verified: 7/7 rule objects field-identical to source including key order,
no non-ASCII bytes, all seven compile in Node bare, i and iu (21/21) and in
Python re (7/7). Charter guard clean - no executable code in the repository.
README and CHANGELOG updated: the file moves out of "planned, not in
v0.1.0" and out of "not included".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SNMcqrfNyoLRQ7qXUFZnb9
Advisor review found six gaps the session's own checks did not cover. The one
that mattered was outward-facing: the divergence report told the guard repo
that two sub-agent bounds were undocumented. That was wrong. lexicon.py's
module docstring documents them explicitly under "Bounded token gaps", and
git log -S dates {0,12} to the original port commit f397cd9 and [^><] to the
ReDoS fix cff0437. Every divergence on the guard's side is documented and
traceable. The claim rested on two sed slices of one file; an absence claim
needs a search over the whole repository. Corrected here and by coord.
lexicon/injection-lexicon.json 0.2.0 -> 0.3.0:
- pattern_id_space.alias_evidence records the two aliases separately instead
of averaging them. llm_ingestion_guard is verified — coverage.py asserts on
that exact string, so it is demonstrably what a guard finding carries.
llm_security is not: it is the pattern table's name, the finding producer
was never supplied, and the known Node finding shape uses title, not label.
- normalisations[].affects now keys on id, with the prose names kept beside it
as affects_labels. An internal cross-reference on label was a second
identity space inside the file the id exists to unify.
Detection data unmoved again: labels, patterns, flags and the ids and aliases
added in 7b70f5b are all byte-identical in sequence; 166/166 Node compiles.
Also: the README lexicon row described thematic families the file does not
have (they are severity families; the theme is the id prefix), and the
conformance convertibility table gained its missing second condition — a case
is buildable only if the label it asserts maps to data this repository
publishes. Eleven cases fail that test (entropy, decoded, the sanitize rows,
the OKF scans), so the buildable set is ~94, not 105.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
The guard's coverage.py turned out to be readable from a session here — the
read boundary covers llm-security only. Counting it instead of estimating
corrected two published numbers and surfaced the real conformance blocker.
Counted, not estimated: CORE_CASES holds 134 cases (128 caught, 6 gap), not
the "126 classes + 4 gaps-must-hold" the extraction plan claimed. 105 are
convertible to static input/expected; 29 assert a runtime API surface this
repository does not own, and are named as out of scope rather than faked.
Measured while counting, and the reason the corpus is still blocked: the
commons lexicon and the guard's are two ports of one source file, and they
disagree. 83/83 patterns correspond, 64 byte-identical, 6 differ only by
escaping and are proven equivalent by string identity, and 13 behave
differently — with a witness input for each and misses in both directions.
Cause is two different ReDoS mitigations of the same table: the Node side
bounds the run ({0,256}), the guard excludes the anchor character ([^><]).
Each has a recall cost the other does not.
No data file changed. Behaviour preservation is a v0.1.0 invariant, so the
divergence is reported to the owning repositories, not fixed here. The 8
hybrid severities the guard's port assigns are deliberately NOT copied in —
a second-hand port is not the producing module.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
repo-standard's gate flagged 10 dead internal README links. Seven now
resolve because the files landed this session. The remaining three are
unlinked and marked "Planned, not in v0.1.0" rather than invented:
- signatures/malware-signatures.json — seed data not delivered
- spec/decode-pipeline.md — needs the decode implementation, which is engine
code and was not supplied. A normative spec guessed from a data dump would
be worse than an absent one: it would carry a Status: normative marker and
pass every convention check while saying nothing anyone verified.
- conformance/ — corpus form is still an open decision
The changelog claimed 0.1.0 shipped "a conformance corpus". It does not.
Rewritten to enumerate what is actually present, to state which files were
proven differentially and which one (calibration) carries verified: false,
and to name what is not included. Moved back under [Unreleased] because the
tag has not been cut.
README also now tells a reader that each data file records its own provenance
and verification strength, and which file currently says false.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
The risk-score tier constants, verdict thresholds, risk-band cutoffs and
posture grade thresholds. Constants only — the log scaling and the if/else
chains that consume them are engine and stay in the consumer.
This file is the exception in this repository and is marked as such in its
own verification block. Every other data file here was rebuilt from its
commons JSON and diffed against an imported module. This one arrived as a
human-written PROSE SUMMARY of severity.mjs, so there was nothing to import
and nothing to diff. verified: false, with the two skipped checks named
rather than a caveat attached to a pass.
The consequence inverts this repository's central rule and the file says so:
for every other file a disagreeing consumer is wrong; for this one, until the
module arrives in executable form, a disagreement is not automatically the
consumer's bug.
What COULD be checked was: bands are contiguous and non-overlapping across
0-100, the BLOCK trigger (65) is exactly the Critical band floor, the WARNING
trigger (15) is exactly the Medium band floor, and the per-tier reachable
minima recompute exactly (80/48/20/4, exact because log2(2) is 1).
Corrects README and the extraction plan in the same commit: both promised
entropy floors, scan caps and disposition ranks. None arrived — 0 occurrences
each across the whole dump. Named in the file under not_supplied so the
absence is visible rather than inferred, and both rows now describe what is
actually present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FaYqid3mejFmd9ZHsiHgp3
Data+contract+fixtures repository for runtime-neutral LLM/agent security
detection. Charter is hard: no engine code, no dependencies, no runtime —
JSON data, normative specs and conformance fixtures only.
Scaffolding only; the seed data files land in follow-up commits from
verified source dumps (never from recollection).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0191AKc2qW6tmXDFSx1xn53q
