feat(schema): close the finding contract against its producer
The schema was seeded from sarif-formatter.mjs, which consumes findings.
That could only ever establish a lower bound on the property set, so
additionalProperties had to stay open. The producer is now readable:
finding() in scanners/lib/output.mjs line 32, at b0de0ca. It returns an
object literal with exactly ten keys and no spread, so the set is complete
and the schema closes.
Added: id and evidence, the two keys a consumer-side reading could not see.
id gets its own definition, DS-<prefix>-<counter>, with pattern
^DS-[A-Za-z]+-[0-9]{3,} - the {3,} because padStart(3) is a minimum, so a
run past 999 findings produces four digits. It comes from a process-global
counter and is stable neither across runs nor across processes; the
definition says so before someone keys on it.
Nullability is evidence now, not convention. Five keys are emitted as null
rather than omitted, so a serialised finding always carries all ten. The
four assigned straight from opts are the exception: omit description and
the key is undefined and disappears from the JSON. Reproduced against the
real producer - ten keys in memory, nine serialised.
owasp is a string, not an array, and not one code. Multiple codes are
joined with ", ". Measured across the seed runtime: 31 distinct values over
157 sites, 13 multi-code, and four that mix taxonomies inside a single
value with no discriminator. That has a consequence nobody had written
down: sarif-formatter builds tags: [f.owasp], so "LLM06, ASI02" becomes ONE
tag with a comma in it and nothing filtering on LLM06 matches. Reproduced
end to end through the real finding() and toSARIF(), logged as
known_lossiness.owasp-tag-not-split. It is consumer behaviour, not data, so
it is reported rather than fixed here.
The JSONL profile is set to "not applicable" rather than "unspecified".
The distinction carries weight: unspecified would assert a profile exists
and has merely not been written down. There is no finding-JSONL - findings
are emitted only inside one JSON envelope. The single module that does
write JSONL, audit-trail.mjs, writes audit events under a different schema
where owasp is an ARRAY. Same field name, different type, same repository.
The profile records that trap instead of leaving a TODO.
Verified: valid Draft 2020-12, every finding built by the real producer
validates, and four negative controls are rejected.
One new open question left unpatched: the producer's JSDoc lists seventeen
scanner prefixes including IDE, while all four maps in owasp-map.json are
keyed on sixteen without it. An IDE finding has no taxonomy mapping
anywhere. Adding the key would be inventing detection data.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SNMcqrfNyoLRQ7qXUFZnb9
This commit is contained in:
parent
be84121604
commit
7e2e92eecf
2 changed files with 116 additions and 32 deletions
49
CHANGELOG.md
49
CHANGELOG.md
|
|
@ -18,7 +18,7 @@ change. **Not yet tagged** — see *Not included* below.
|
|||
### Added
|
||||
|
||||
- `schema/finding.schema.json` — the finding contract plus the SARIF output profile.
|
||||
Normative. The JSONL profile is deliberately left `unspecified`.
|
||||
Normative. Closed against the producer in 0.2.0; the JSONL profile is `not applicable`.
|
||||
- `signatures/active-content.json` — the EchoLeak class (CVE-2025-32711): 17 patterns,
|
||||
severities, opacity floors and pass order, from the Python guard.
|
||||
- `lexicon/injection-lexicon.json` — 83 prompt-injection patterns in four families
|
||||
|
|
@ -53,6 +53,53 @@ checks that were not run instead of attaching a caveat to a pass.
|
|||
|
||||
### Changed
|
||||
|
||||
- `schema/finding.schema.json` **0.1.0 → 0.2.0** — the schema is **closed**. It was seeded
|
||||
from `sarif-formatter.mjs`, which *consumes* findings, so its property list could only ever
|
||||
be a lower bound and `additionalProperties` had to stay open. The producer is now known —
|
||||
`finding()` in `scanners/lib/output.mjs`, line 32 — and it returns an object literal with
|
||||
**exactly ten keys and no spread**: `id`, `scanner`, `severity`, `title`, `description`,
|
||||
`file`, `line`, `evidence`, `owasp`, `recommendation`. `additionalProperties` is `false`,
|
||||
and the two keys the old schema never knew about (`id`, `evidence`) are added.
|
||||
|
||||
`id` gets its own definition: `DS-<prefix>-<counter>`, pattern `^DS-[A-Za-z]+-[0-9]{3,}$`.
|
||||
The `{3,}` is deliberate — `padStart(3, '0')` is a minimum, so a run emitting more than 999
|
||||
findings produces four digits. The id comes from a process-global counter, so it is stable
|
||||
neither across runs nor across processes, and the definition says so before someone keys on it.
|
||||
|
||||
Nullability is now evidence rather than convention. Five keys are emitted as `null` rather
|
||||
than omitted (`opts.x || null`), so a serialised finding always carries all ten. The
|
||||
exception is the four assigned straight from `opts`: omit `description` and the key is
|
||||
`undefined` and vanishes from the JSON. Verified by calling the real producer — ten keys in
|
||||
memory, nine after serialisation.
|
||||
|
||||
**`owasp` is a string, not an array, and not one code.** Multiple codes are joined with
|
||||
`, `. Measured across the seed runtime: 31 distinct values over 157 emission sites, 13 of
|
||||
them multi-code, and **four mix taxonomies inside a single value** (`LLM06, ASI02` and
|
||||
friends) with no discriminator saying which is which. That sharpens the edition problem
|
||||
`mapping/owasp-map.json` already records, and it has a consequence nobody had written down:
|
||||
`sarif-formatter.mjs` builds `tags: [f.owasp]`, so a finding anchored to two taxonomies
|
||||
produces **one** SARIF tag with a comma in it. Nothing filtering on `LLM06` will match.
|
||||
Reproduced end to end through the real `finding()` and `toSARIF()`, and logged as
|
||||
`known_lossiness.owasp-tag-not-split` — consumer behaviour in `llm-security`, not data, so
|
||||
it is reported rather than fixed here.
|
||||
|
||||
The **JSONL profile is `not applicable`, not `unspecified`** — the distinction is the point.
|
||||
`unspecified` would claim a profile exists and merely has not been written down. No
|
||||
finding-JSONL exists: findings are emitted only inside a single JSON envelope
|
||||
(`output.mjs:140`). The one module that does write JSONL, `audit-trail.mjs`, writes *audit
|
||||
events* under a different schema — where `owasp` is an **array**. Same field name, different
|
||||
type, same repository. A consumer reading both through one code path will be wrong about one
|
||||
of them, so the profile records the trap instead of leaving a TODO.
|
||||
|
||||
Verified: the schema is valid Draft 2020-12, every finding built by the real producer
|
||||
validates against it, and four negative controls (extra property, missing `id`, malformed
|
||||
`id`, unknown severity) are all rejected.
|
||||
|
||||
One new open question, unpatched by design: the producer's JSDoc lists **seventeen** scanner
|
||||
prefixes including `IDE`, while all four maps in `mapping/owasp-map.json` are keyed on
|
||||
**sixteen** without it. An `IDE` finding has no taxonomy mapping in any map. Adding the key
|
||||
would be inventing detection data.
|
||||
|
||||
- `lexicon/injection-lexicon.json` **0.4.0 → 0.5.0** — the last null in the file is filled and
|
||||
the id space is ratified. Two blockers close, no detection data moves.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue