feat(lexicon): count every match of a pattern, and give four gaps a measured reason
Gap 4 (count=1) is closed. scan_lexicon still gives one finding per pattern id at the first offset, and its count is now the number of matches: the most matches any one view of the text shows (raw, decoded, folded, rot13). Views are not summed, so count is a lower bound; the joiner row counts only the splits it reports. The matrix case moved from gap to caught (lexicon:count), and the gap register records it closed. Measured before and after on the same trees: the FP populations are identical line for line at 106/198, 196/2918, 108/389; all 3583 documents (with the 78 pinned OKF concepts) keep their disposition under both presets and the same labels and severities; OKF import stays 47/31. 53 documents now carry count > 1 (23 + 23 + 7 + 0), as measured before building. scan_lexicon over 3505 documents: 96.0 s before, 97.1 s after. Mutating count back to 1 turns 5 tests red. docs/LIMITATIONS.md: gaps 2, 3, 5 and 7 now state the rule tried and its cost (session 1 numbers): 2 fires on the true twin, 3 costs 48 on the trusted gate, 5 costs 23 of 569 READMEs, 7 catches 2 of 5 rewordings. Matrix 145/145 + 6/6 gaps, suite 1260 passed.
This commit is contained in:
parent
c4ecf0dba6
commit
736bf7882f
9 changed files with 217 additions and 26 deletions
|
|
@ -240,9 +240,15 @@ items; this is the full list, each with the mechanism.
|
|||
indented code block (a comment there is reported although it renders). The
|
||||
input side is unchanged: it strips every comment, code samples included.
|
||||
|
||||
- **Lexicon findings are deduplicated by pattern id** — `count=1` and the first
|
||||
offset are reported, so a class matched across several channels collapses to one
|
||||
finding at its first location: a deliberate readability tradeoff.
|
||||
- **A lexicon finding's `count` is a lower bound.** Each pattern id gives one
|
||||
finding, at the offset of its first match, so a report stays readable. Since
|
||||
1.9.0 `count` is the number of matches, not 1. The lexicon matches several views
|
||||
of the text (raw, decoded, homoglyph-folded, rot13), and one match shows in
|
||||
several of them, so the views are not added up: `count` is the most matches any
|
||||
one view shows. A plain match plus a homoglyph one counts 2, because the folded
|
||||
view shows both. A plain match plus a rot13-hidden one counts 1, because no
|
||||
single view shows both. `count` never reaches the disposition, so no document
|
||||
changes gate.
|
||||
- **Active-content severity grades on URL shape, so a pure *beacon* is only LOW.**
|
||||
Since 0.3.1 a URL that merely names a remote document (bare path, no query, no
|
||||
userinfo, no percent-escapes, no opaque segment) is LOW, and only a URL that can
|
||||
|
|
@ -796,19 +802,56 @@ items; this is the full list, each with the mechanism.
|
|||
returns *content* rejects at the cap, a surface that returns *findings*
|
||||
truncates and says so.
|
||||
|
||||
## The seven documented gaps (tracked by the coverage matrix)
|
||||
## The documented gaps (tracked by the coverage matrix)
|
||||
|
||||
These are asserted to *still hold* by `tests/test_coverage_matrix.py` — a closed gap
|
||||
fails the test, forcing this doc to be updated:
|
||||
Seven gaps were frozen on 2026-11-01 (`docs/threat-2026/gaps-frozen.tsv`). The open
|
||||
ones are asserted to *still hold* by `tests/test_coverage_matrix.py`, so a gap that
|
||||
closes fails the test and forces this doc to be updated. A gap is closed only when
|
||||
a rule catches the class without a new false alarm on clean documents. Otherwise it
|
||||
stands with the rule that was tried and what that rule cost, measured with
|
||||
[`docs/gap-sweep.py`](gap-sweep.py) on 2026-11-01 (v1.8.0). The clean documents
|
||||
are 3,583: the three false-positive corpora (198, 2,918 and 389) and the 78 OKF
|
||||
concepts at the pinned upstream commit. A "new" alarm is a document the gate passes
|
||||
today (WARN) that the rule would stop. Because those 3,583 hold no external image,
|
||||
569 package READMEs (from installed npm and PyPI packages, deduplicated by sha256)
|
||||
were measured too.
|
||||
|
||||
1. **Hex-wrapped secret egress** — `entropy` decodes base64 only (above).
|
||||
1. **Hex-wrapped secret egress** — `entropy` decodes base64 only (above). Open.
|
||||
Measured to close without new alarms; next in line.
|
||||
2. **Semantic / factual poisoning** — invisible to token analysis (above).
|
||||
**Measured reason:** the cheapest rule, a factual claim of the form "is made of"
|
||||
or "was completed in", gave 0 new alarms on the 3,583 and fired on 3 of the 569
|
||||
READMEs. But it fires on the true sentence exactly as on the false one ("made of
|
||||
iron and nickel", "completed in 1889"), and the guard gives the same findings
|
||||
(none) for both halves of such a pair. Whether a claim is true is not in its
|
||||
words, so no rule of this kind can see the class. The `grounding` seam is where a
|
||||
judge that checks claims against sources plugs in.
|
||||
3. **A lone HIGH in trusted prose → WARN** — the §4.7 trust-scaling design (above).
|
||||
4. **Lexicon dedup (`count=1`)** — first offset only, by design (above).
|
||||
**Measured reason:** letting a lone HIGH escape WARN under
|
||||
`PRESET_TRUSTED_SOURCE` would stop **48** of the 3,583 clean documents on the
|
||||
trusted gate (6, 11, 31 and 0 across the four populations) and 67 of the 569
|
||||
READMEs. The upload gate is not affected. Run a source that can carry attacker
|
||||
text as untrusted.
|
||||
4. **Lexicon dedup (`count=1`)** — **closed in 1.9.0.** `count` is now the number
|
||||
of matches (see "A lexicon finding's `count` is a lower bound" above). It never
|
||||
reaches the disposition, so the rule adds 0 alarms by construction; 53 of the
|
||||
3,583 documents get a lexicon `count` above 1.
|
||||
5. **Pure beaconing** — a bare-path remote image on a hostile host is LOW (above).
|
||||
**Measured reason:** grading every external image above LOW gives 0 new alarms on
|
||||
the 3,583, but that measurement is blind: none of them holds an external image.
|
||||
On the 569 READMEs it fires on 52 and would stop **23** of the 211 that pass
|
||||
today. The fetch itself is what an ordinary README does.
|
||||
6. **Short opaque URL segment (<24 chars)** — below what entropy can resolve (above).
|
||||
Open. The cheapest rule (a segment of 8 to 23 characters that base64-decodes to at
|
||||
least 6 printable bytes) fired on 4 of 14,943 README URLs, among them a size
|
||||
parameter in an image URL and the word "Pygments". It has to demand more of the
|
||||
decoded text and be measured again before it is built.
|
||||
7. **Unmarked prose task (toxic agent flow, A12)** — no address, deferral, memory or
|
||||
concealment to read (above).
|
||||
concealment to read (above). **Measured reason:** a rule for "gather it from
|
||||
private places" gave **6** new alarms. Adding "and put it into this document" in
|
||||
the same paragraph gave 0 and catches the A12 case, but only 2 of 5 rewordings of
|
||||
the same request (the first rule also 2 of 5). A rule that misses 3 of 5
|
||||
rewordings knows one sentence, not the class, so the gap is not counted as closed.
|
||||
|
||||
- **`Severity` still carries disposition intent on the detection side.** The
|
||||
0.5.0 axis separation split the *assessment* (`Risk`) from the *action*
|
||||
|
|
|
|||
31
docs/PLAN.md
31
docs/PLAN.md
|
|
@ -252,6 +252,37 @@ hver uke. Under en annen kjøring uten ssh-remote ville proben her svare
|
|||
|
||||
Ingen release, fordi ingenting i den utgitte pakken er endret.
|
||||
|
||||
## V.8 Økt 3, hull 4 lukket og fire begrunnelser (01.11.2026, release 1.9.0)
|
||||
|
||||
**Hull 4 (dedup):** `scan_lexicon` gir fortsatt ett funn per mønster-id med første
|
||||
offset, men `count` er nå antall treff. Leksikonet leser flere varianter av teksten
|
||||
(rå, dekodet, foldet, rot13), og samme treff vises i flere av dem. Derfor er `count`
|
||||
det høyeste antallet i én enkelt variant, og variantene summeres ikke. Det er en nedre
|
||||
grense: et rent treff pluss et rot13-skjult teller 1. For orddelingsraden telles bare
|
||||
skjøtene raden melder. Matrisesaken står nå som fanget (`lexicon:count`), og
|
||||
`tests/test_gap_register.py` har `CLOSED = {"dedup": "count"}`.
|
||||
|
||||
- **TDD:** fem tester i `tests/test_lexicon.py`. Tre var røde av riktig grunn
|
||||
(`count` 1, ventet 2), og to låser det som allerede holdt (ett treff gir 1, og
|
||||
rot13 pluss rent treff gir 1). Mutasjonssjekk: med `count=1` igjen ble 5 tester røde.
|
||||
- **Vaktregelen:** målt før og etter på samme trær. FP-korpusene er identiske
|
||||
linje for linje på 106/198 · 196/2918 · 108/389. Alle 3 583 dokumenter (de tre
|
||||
korpusene og de 78 OKF-konseptene ved pinnen) har samme disposisjon under begge
|
||||
forhåndsinnstillingene, og samme etiketter og alvorlighetsgrader. OKF-importen står
|
||||
på 47 fail-secure / 31 warn, uendret. **53** dokumenter får `count` > 1
|
||||
(23 + 23 + 7 + 0), som målt i § V.6, og høyeste `count` er 26. 0 nye alarmer.
|
||||
- **Ytelse:** `scan_lexicon` over de 3 505 dokumentene i de tre FP-korpusene tok
|
||||
96,0 s før og 97,1 s etter, og de tregeste dokumentene er de samme.
|
||||
- **Rad 2:** hull 4 er lukket. 2, 3, 5 og 7 står med målt begrunnelse i LIMITATIONS
|
||||
(tallene fra § V.6). 1 og 6 er åpne, og de er neste.
|
||||
|
||||
**Vakten** er kjørt på nytt 01.11 før release-commiten: 16 current, 0 behind, 0 watch,
|
||||
0 notVerified. `watch-last.json` er endret bare i `lastRun`.
|
||||
|
||||
**llm-security** har ikke svart på forespørselen om ukentlig kjøring. Meldingen
|
||||
ligger fortsatt ubehandlet i innboksen deres. Rad 1 er ikke framme før jobben deres
|
||||
har kjørt lista vår.
|
||||
|
||||
## Forrige retning (godkjent av operatøren 28.10.2026, nådd 31.10)
|
||||
|
||||
> «llm-ingestion-guard bygger på nyeste utgave av hver trusselkatalog den viser til,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue