Rows 1-3 hold on 1.8.0: 16/16 catalogues current, 0 stale catalogue numbers
(frozen denominator 95 at 6702d7a unchanged), 17 attacks caught and 2
documented gaps with none red. The project rules now say the remote is the
public open/ repository, and .claude/ is ignored so local session state cannot
be added by accident.
Session 7 of the 2026 threat plan (attack row 3): A08, A09, A12, A16 and
A19 were red. Their payloads are ordinary English with no marker.
- indirect:prose-instruction (MEDIUM): a paragraph that combines two of
four classes: it speaks to the model, it is meant to act later, it
writes to the model's memory, it hides from the user. One class alone is
ordinary prose. A keyword prefilter skips text where no non-address
class can match. Catches A08, A09, A16 and A19.
- A12 (a task with no marker) is the seventh documented gap; a rule that
caught it would be fitted to the payload's words.
- redos-sweep walked no dicts, so the new class patterns were swept by
nothing. It walks dicts now: 168 patterns, 0 flagged; positive control
flagged at x4.0.
Measured before building: 0 of 5494 benign documents fire (the three FP
corpora, 577 package READMEs, 1412 vendor-published agent skills). Guard
rule before/after: FP 106/198, 196/2918, 108/389 identical line for line;
OKF import 47/31 identical. Matrix 144/144 + 7/7 gaps, suite 1233 passed.
Row 3: 17 caught, 2 gaps, 0 open.
Session 6 of the 2026 threat plan (attack row 3): A05 output was red.
CamoLeak renders a secret as one image per character from a URL alphabet;
each URL is ordinary, so each image graded LOW and the gate WARNed.
- active:markdown-image-sequence (MEDIUM): at least 4 ordinary external
images, inline or reference-style, on one URL template whose varying
tokens are 2 chars or fewer, or digit runs. Digits that count up in
document order are a counter and carry nothing. The per-image grade is
unchanged; the finding sits beside it.
- Accepted twins (flag tables, figures 1a/1b) go to review; measured
evasions (word dictionary, 3-char symbols, counter increments, run of 3,
a template per symbol) are in LIMITATIONS.
Guard rule before/after: FP 106/198, 196/2918, 108/389 identical. Those
corpora hold no external markdown image, so the rule was also run on 577
package READMEs (312 with external images): 0 firings at run lengths 3-8.
OKF import 47/31 identical. Matrix 140/140, ReDoS 163 patterns 0 flagged,
suite 1179 passed + 5 xfailed. Row 3: 13 caught, 1 gap, 5 open.
Session 5 of the 2026 threat plan (attack row 3): A04 and A15 were red.
- html-obfuscation:tiny-font (HIGH): font below 4px. ShadowLeak's 1px passed
because the commons zero-font row only matches a leading 0.
- html-obfuscation:invisible-colour (HIGH): text within 8 per channel of the
background in the same style attribute, or transparent. White alone is not
a signal (email buttons); measured residual in LIMITATIONS.
- output:html-comment-present (MEDIUM): a comment in the artifact outside
fenced code and inline code. 253/254 comments in the reference corpus sit
in code.
Local to output.py; the vendored lexicon stays byte-equal with commons.
Guard rule before/after: FP 106/198, 196/2918, 108/389 identical; OKF 6/53
and 47/31 identical. Matrix 139/139, ReDoS 161 patterns 0 flagged, suite
1142 passed + 6 xfailed. Row 3: 12 caught, 1 gap, 6 open.
Row 1 of the 2026 done-criterion: 16 of 16 register entries on the newest
edition, each with a check date and a source link, re-read at the publisher
on 2026-10-31.
- README: new subsection mapping OWASP ASI06 and seven MITRE ATLAS v2026.10
techniques to what the guard stops at the write and what it does not;
a dated catalog table; the threat-model anchors are linked.
- ATLAS moved to v2026.10 (2026-10-28) after session 1; AML.T0057 no longer
exists there and is not linked.
- Register 1.1.0: `cited` beside `latest` on every entry; CleanBase added
(cited in BRIEF, unregistered); unverifiable sources kept as candidates.
- tests/test_catalog_register.py fails on a missing date or link, an older
cited edition, or a README ATLAS/ASI id the register has not verified.
- ZWJ measurement re-taken on Unicode 18.0 (1614 sequences, 122/122); the
"458 unassigned" figure came from Python's Unicode 16.0, 406 in 18.0.
OWASP's Top 10 for LLM Applications 2026 renumbered the list; only LLM01
and LLM02 kept their meaning. Every finding's `owasp` was written against
2025 with no year, so EchoLeak findings said LLM05, which in 2026 means
Data and Model Poisoning. Row 2 of the 2026 done-criterion counted 95
such sites at 6702d7a (82 old meaning, 13 wrong); it is now 0.
- Producers name the risk, never the number: one table (`_owasp.py`)
owns `LLMnn:2026`. The vendored lexicon stays byte-equal with commons
and gets its year where the finding is made; a commons number whose
meaning moved between editions raises instead of being guessed.
- Mechanical map from the 2025 meaning (verified against the publisher
in session 1): LLM05 -> LLM10:2026, LLM06 -> LLM03:2026,
LLM10 -> LLM06:2026, LLM08 -> LLM09:2026, LLM04 -> LLM05:2026.
The 13 wrong-meaning sites, judged:
- Factual poisoning (PoisonedRAG gap, and the test that follows it):
LLM05:2026. The 2026 text names "RAG Knowledge Base Poisoning" as its
example 6. It said LLM09, which was Misinformation.
- Fail-secure (README row, matrix row): no number. It is the guard's own
failure semantics; no edition describes it. Was LLM10.
- Parser-surface refusals (nested mapping, block scalar, quoted flow
element, off-allowlist key): no number. Refusing a shape the parser
would have to interpret is not resource consumption. Was LLM10. The
anchor and merge-key rows stay LLM06:2026 (alias expansion is DoS).
- `resource` URL and cross-link scheme validation (4): no number. They
validate received bundle metadata; LLM10:2026 is about model output
and says input validation belongs to LLM01. Was LLM05.
- `_raise_case` default (OKF path traversal, absolute path, reserved
name, non-.md, bundle escape): no number. None is prompt injection.
Was LLM01.
A doubtful number would be the misreading the year exists to prevent,
so `Case.owasp` is now Optional and the matrix prints "-".
Self-safety (input caps, ReDoS) keeps LLM06:2026: the 2026 text, read at
the publisher, covers "input flood ... exploiting processing
inefficiencies" and application-side rechunking. The README's LLM09
anchor (Misinformation) is dropped; it was the factual-poisoning row.
The permanent test (tests/test_catalog_numbers.py) fails on an old or
wrong number, on a number without the year outside the vendored
lexicon, and on any finding the coverage matrix drives whose `owasp`
lacks the newest year; "newest" is read from the catalog register. The
census holds its wrong-rules to their counts at the frozen revision
only, so the frozen listing (da1aa0fa...6f) is unchanged.
Variation-selector smuggling (A01) and Sneaky Bits (A02) gave 0 findings
on 1.4.1 and stayed in the sanitized text. The denominator is now the
frozen Unicode 18.0 set (DICP | Cf | Co, 141 674 codepoints); each one is
stripped and flagged unless a known legitimate use exempts it on context:
one VS15/16 after an emoji base, one ideographic selector after a CJK
ideograph, ZWNJ/ZWJ beside Arabic-script or Indic letters, LRM/RLM/ALM
in a document with RTL text, an Arabic number sign before a digit. A run
is always a carrier; private use is caught in full (operator decisions
2026-10-28). One predicate serves sanitize, the output gate and the
lexicon's word-split row, whose regex stays byte-identical with commons.
A01 and A02 are green; the FP populations and the pinned OKF corpus are
identical before and after.
Rows 1-3 of the done-criterion now carry frozen denominators: 15
catalogs (0 dated), 95 number sites, 19 attacks (8 caught, 1 documented
gap, 10 red). The 2025 -> 2026 mapping is re-verified against OWASP,
and the attack batches for sessions 5-7 are proposed.
Nineteen attacks that can reach an ingestion pipeline, verified against
primary sources, with four out of scope and their reasons. Each arrival
runs through the canonical wiring; twelve of 31 arrivals are red today
and marked strict xfail, so a fix fails until its marker is removed.
PoisonedRAG asserts that its documented gap still stands.
Fifteen catalogs and sources, each with the edition cited at v1.4.1, the
newest edition read from its publisher, the check date and the link.
Behind: OWASP LLM Top 10 (2025 -> 2026) and Unicode (17.0 -> 18.0).
The invisible-character denominator for the next session is Unicode
18.0's own: Default_Ignorable_Code_Point, format (Cf) and private use
(Co), 141 674 code points, built from sha256-pinned UCD files.
docs/catalog-census.py classifies every OWASP LLM number in tracked files:
old (2025 meaning, maps mechanically), wrong (a meaning no edition gave it,
each with its reason), same, current, history, migration-plan, self.
At 6702d7a the denominator is 95 sites in 23 files (82 old, 13 wrong);
the listing is frozen with its sha256 and a test reproduces it from git.
The approved direction heads the plan: every cited catalog on its newest
edition with a check date and source link, no catalog ID carrying an older
edition's meaning, and the known 2025-2026 attacks that can reach an
ingestion pipeline as test cases, each caught or documented as a gap.
Measured on v1.4.1 before anything is built: LLM03-LLM10 carry their 2025
meaning in 101 places across 26 tracked files, and 17 of 19 invisible or
deceptive code-point types (variation selectors among them) pass sanitize
with no finding.
Patch release for 639da03: a flow-sequence element admits '#' and ':'
where YAML reads them as text, so okf's own `references: [...]` lines
with #fragments and scheme:// links parse (0 / 5 467 raise on the
consumer's bundle, was 2 038). Also ships 79285e1, which brought the doc
surfaces up to 1.3.0/1.4.0 behaviour.
Unlike the 1.4.0 release commit, this one bumps every live version
surface: pyproject.toml, __init__.py, README badge + status + install
pin, ADOPTION-BRIEF status + "as of" + test count (893 -> 910), BRIEF
status, CHANGELOG heading. Provenance references to 1.4.0 are left as
they are.
No exported surface changed. Gates on this tree: 910 passed, coverage
exit 0 (130/130 + 6/6), redos-sweep exit 0, 45 LIMITATIONS entries.
The 1.4.0 release commit (d19de8c) dated the CHANGELOG heading and nothing
else, so the tag went out with README, ADOPTION-BRIEF and BRIEF still
naming 1.3.0 and the install pin still installing 1.3.0. This bumps the
badge, status line and install pin in README, the status and "as of"
lines in ADOPTION-BRIEF (test count 868 -> 893) and BRIEF's status line.
The tag itself is not moved.
Three claims that went false earlier are corrected against the code:
- README said `resource` is deliberately not on the frontmatter allowlist
and that a `sources` block list of mappings is refused. Both stopped
being true in 1.3.0. The paragraph now names the four mapping carriers
and says `resource` / `usage_window` are admitted inside a `sources`
entry only.
- README described the `resource` https-allowlist without its scope. It
covers the top-level `resource` only; `sources[].resource` is never
URL-validated (measured: `javascript:` there imports with no error).
- LIMITATIONS still listed `tags: [a, b]` among the routes that fail.
It parses since 1.4.0.
Gates after the edit: 893 passed, coverage exit 0, redos-sweep exit 0,
45 LIMITATIONS entries.
`tags: [a, b, c]` is the form SPEC.md 4.1's own frontmatter skeleton writes out,
and 9/53 upstream reference concepts use it. It raised on the `[` indicator.
It parses now, to the same list its block-sequence sibling already produced.
The predicate is character-level, inside `_parse_flow_sequence`: an element is a
plain scalar only if it is non-empty and carries none of `{ } [ ] : , " ' #`,
and it then passes the unchanged scalar-indicator rule. Everything that would
need YAML semantics to split or unquote still raises - a quoted element (quotes
are retained here, never stripped), a colon or comma inside an element, a
sequence inside a sequence, an empty element, an anchor, an alias. A sequence
may not mix scalars and mappings, the rule the block list already carries, and
the mixing verdict is reached before the element is parsed so the caller is told
about the mix rather than about a key the allowlist would have named instead.
The 1.3.0 `sources` flow-mapping carrier is unchanged and pinned against
regression. Depth 1 is not spent: the elements are leaves.
Measured with the denominator, against the pinned corpus (`_okf-upstream` @
3fcbb9f, 53 non-reserved documents) and the pinned SPEC (`_okf-canonical` @
ad30107). Baseline reproduced first, with a known-positive control, at 0/53.
After: 6/53, all six in acme_retail. It does not close the corpus - 44/53 still
stop on `generated` written as a top-level block mapping, which spends the
no-nesting-past-depth-1 rule and is a security decision, out of scope here.
P1 alone, per the operator decision of 08.09. The two neighbouring predicates
were measured and deliberately not built: a flush-left block sequence and a
folded plain scalar release 0/53 each on their own, and stacked on this one they
still measure 6/53. `_consume_block_list`, the `description` continuation and
the allowlist are untouched.
docs/LIMITATIONS.md's tags/description entry is rewritten against the
measurement: three of its claims were wrong. The parser does have a
sequence-value type (since 1.3.0 - what it lacks is the indentation the corpus
omits); the figure is 6/53, not the 1.2.0-era 4/53; and tags/description are
not the residual that blocks the corpus. README gains the sequence carrier in
the paragraph that already describes the mapping one.
Self-safety: the predicate compiles no regex, so docs/redos-sweep.py cannot see
it. Measured instead on the CPU clock - linear in element length (exponent
0.86-0.99) and in element count (0.97-1.05) over four doublings to 800_000 -
and pinned by two bounds in tests/test_okf.py.
Six rows that pinned the old refusal are re-aimed at the class that still
holds - the quoted element - the way the 1.3.0 rows were when the carrier
opened. One of them lives in src/llm_ingestion_guard/coverage.py, which is why
the src diff is three files rather than one.
Version 1.4.0 in the code only. The CHANGELOG entry stays under Unreleased and
no tag is cut: README's badge and install pin must keep naming a tag that
exists.
Gates after `git add`: 893 passed (was 868), coverage 130/130 + 6/6 gaps,
redos-sweep exit 0, LIMITATIONS still 45 entries.
Order 20260906T213322Z: measure and propose, no src, no bump. Measured against
SPEC _okf-canonical @ ad30107 and the corpus _okf-upstream @ 3fcbb9f (denominator
53, extracted at the pin because the work tree had moved on to 9a15b13).
Three claims in the tags/description entry are stale against the 1.3.0 parser.
The parser does have a sequence type -- _consume_block_list parses block lists of
scalars -- what it lacks is tolerance for the flush-left indentation 36/53 of the
corpus uses. Removing tags alone now lets 6/53 pass, not the 4/53 recorded, and
the entry's headline claim does not hold at all: description alone unblocks 0/53,
and with all three surface forms closed 44/53 still stop on generated as a
top-level block mapping. The entry oversells its own reach by a factor of seven.
The SPEC has no depth rule. no-nesting-past-depth-1 is entirely ours; the
conformance floor is only "a parseable YAML frontmatter block" (SS11.1).
Candidates were measured by normalizing the surface form onto a shape the parser
already accepts, then importing parse_frontmatter -- the predicate is never
re-implemented, only the input's spelling is rewritten. P1 (scalar flow sequence)
takes the corpus from 0/53 to 6/53 without spending depth-1; P2 and P3 buy 0/53
each and 6/53 stacked on P1. Recommendation is P1 alone, and the note says out
loud that this does not open the corpus.
The label "punkt 44" is not grep-able: it came from a count-after-insertion, and
the entry is today the 12th of 45 at docs/LIMITATIONS.md:126.
No src change, no version bump, no push. 868 passed, coverage 130/130 + 6/6 gaps,
redos sweep exit 0 -- all after git add.
Probed after 1.3.0 landed, on advisor challenge. Both came back clean --
these pin them so they cannot regress silently.
1. A `- ` item whose text begins with a YAML indicator AND carries a `": "`
could in principle route into the block-mapping path with a
half-validated key instead of into `_reject_dangerous_value`. It does
not: `&anchor id` fails `_KEY_RE` on the key side, so the item falls
through intact and raises on the indicator. One row each for & * ! %
backtick, plus the merge key inside an entry.
2. `_consume_block_mapping` ends at a blank line and at column zero, and
hands an index back to `_consume_block_list`, which hands one back to
`_parse_flat`. A line orphaned by that boundary -- a `resource:` left
over from a mapping that closed early -- must RAISE, not be dropped. A
pointer that vanishes rather than failing is this repo's failure class,
so it gets a test rather than a probe. Plus the return-index contract:
a top-level key after a multi-entry list is neither swallowed nor
re-read as an item.
868 green (was 859), 130/130 classes. Test-only; no source change.
Order 20260902T150716Z from .claude -- a K5 blocker in the OKF programme.
`parse_frontmatter` rejected `sources` in every form the spec and its
producers actually use. Measured 02.09 by two consumers independently:
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (one entry
as well as two), and the block sequence of block mappings -- SPEC.md 5.1's
OWN example -- raised "nested mappings are not supported". `resource` is
REQUIRED within a `sources` entry, so the whole provenance family was
unwritable and a bundle written the way the spec documents it was refused.
Measured against the spec before coding, not reasoned: 5.1's example block
is the canonical carrier for a REQUIRED field and 11.1 defines conformance
as parseable frontmatter, so refusing it refuses a conformant bundle. Both
carriers now parse to the same list of dicts.
The load-bearing change is not the carrier, it is WHO admits `resource`.
1.2.0 left it off the allowlist arguing the parser could not tell
`sources[].resource` (5.1, a citation) from `executor.resource` /
`attester.resource` (10, a pointer to code to be run -- the door-C route
closed in 1.1.0). That premise was false: the owning key is in scope at
every call site and was simply never threaded through. It is threaded now,
so the discrimination is structural, and door C stays shut through EVERY
carrier including the two this adds -- pinned by a new test that drives
`executor`/`attester` through all four.
Refusal stays the default elsewhere. A flow sequence of plain scalars
(`tags: [a, b]`) still raises: the sequence carrier is opened for the flow
mapping element and nothing else. A `sources` entry admits scalar leaves
only, so 5.1's optional PER-ENTRY `usage_window` is refused -- no nesting
past depth 1 is a security property and it was not spent here; registered
as a conformance gap rather than left as an oversight. A block list may not
mix scalars and mappings, because a consumer reading `entry.get("id")` over
one gets an AttributeError off the first str.
New residual registered: `sources[].resource` is scanned as text (T1) but
never URL-validated. T3's https allowlist cannot reach it without
over-blocking conformant bundles -- 5.1 permits bundle-relative paths and
scope descriptors, and the producers' own golden emits `resource: fixture`.
A consumer that dereferences it must call `validate_resource_url` itself.
Suite 834 -> 859 green. 25 new rows; four pre-existing rows changed because
this release changed the behaviour they pinned, two of them renamed since
their names asserted the old invariant (`exactly_one_route_to_a_mapping`,
`two_keys_per_item_is_where_the_block_list_hard_rejects`). Not "unchanged".
130/130 classes, 6/6 gaps hold, 44 -> 45 limitations, ReDoS 0/152 (the
sweep adds no evidence here -- this change adds no regex and the splitting
is linear). Six version surfaces bumped by hand, no sed. Re-measured alone
after the bump.
No exported surface changed; no detector behaviour and no calibration
changed.
Measured 08-23 (order 20260823T161935Z): the line-flat frontmatter parser has
no sequence-value type at all, so tags rejects 53/53 upstream concepts
regardless of flow or block form, and description's folded-scalar continuation
misreads as a nested mapping in 29/53. Independent of both the mapping-form
gap and the sources block-form gap already documented here -- closing either
moves nothing on this corpus. 43 -> 44 items; README count moved with it. No
code change, no release.
Order 20260823T175853Z from .claude: commit 5870483 (the G3 nine-key
allowlist fix) was untagged -- llm-ingestion-okf measured 4/9 concepts
through Doer C on it, blocked from 8/9 only by the absence of a
pinnable tag, on a suite already green (595/596) at that commit.
Six live version surfaces bumped by hand (no sed -- provenance is
never bumped): pyproject.toml, __init__.__version__, README badge +
Status + the pinned pip install tag, docs/BRIEF.md, docs/ADOPTION-BRIEF.md
Status, and CLAUDE.md. ADOPTION-BRIEF's snapshot line moves with it:
129/129 -> 130/130 classes, 802 -> 834 passing (the new row is the
off-allowlist-key coverage case added by the G3 fix itself).
NOT bumped, matching the 1.1.0 precedent: every "closed in 1.1.0" /
"the 1.1.0 defect" reference across CHANGELOG.md, CLAUDE.md,
docs/LIMITATIONS.md, docs/redos-sweep.py, src/llm_ingestion_guard/okf.py
and tests/ names a historical baseline, not the current version --
rewriting those would erase what they document. docs/BRIEF.md's
GuardLLM v1.1.0 reference is a third party's version, untouched.
Re-measured after the bump, alone: 834 passed, 130/130 classes, 6/6
gaps hold, 43 limitations, ReDoS sweep 0/152 candidates flagged.
No exported surface changed; no detector behaviour and no calibration
changed. Pure release action -- no functional code touched.
OKF v0.2 writes its whole trust and provenance layer as mappings, and T2 gave
the mapping class no expressible form. A consumer measured 0 of 53 upstream
concepts through the gate on 0.3.4, 1.0.0 and 1.1.0. That was a contract
collision, not a calibration setting: SPEC.md @ 62432a09 uses flow mappings in
its own 5.1/5.2 examples, and 11 carries a hard MUST for consumers ("MUST treat
a bare `verified` mapping as a one-element list") that presupposes they parse.
Admitted: a flow mapping, as a value or as a block-list item, whose every key is
on a nine-name allowlist and whose every leaf is a plain scalar run through the
unchanged dangerous-value and mapping-construct predicates. The form is safe
because the allowlist inspects every key -- the blanket refusal was the
enforcement, not the point.
Refused, each on its own rule and ground-truthed against PyYAML 6.0.3: a key off
the allowlist, a nested collection, a quoted leaf, a duplicate key, an empty or
unclosed mapping, trailing junk, and {a:b} (which PyYAML reads as the KEY a:b).
A refused mapping still raises rather than degrading into a string, so the 1.1.0
type-confusion defect is not reopened, and the block, dotted and inline-colon
routes still raise.
`resource` is deliberately off the allowlist though SPEC.md 5.1 names it inside
a `sources` entry: it is a pointer rather than a label and the only key T3
exists for, so admitting it would let `executor: { resource: skills/run.md }`
carry an executable-code pointer through in typed clothes -- the door-C route
closed in 1.1.0. It costs nothing today, because the conformant carrier for
sources[].resource is a block sequence of block mappings, which stays refused.
Mapping leaves are scanned like every other frontmatter value (T1). Coverage
matrix 130/130 (new row: the off-allowlist key). No exported surface, detector
behaviour or calibration changed.
docs/redos-sweep.py timed on time.monotonic() while every ReDoS bound in the
suite had moved to process CPU time, so the 1.5 ms sensitivity floor and the
"~23 s at the cap" figure in docs/LIMITATIONS.md were produced by a different
instrument than the bounds they support. The script imports scan_seconds now.
Re-derived on that instrument, the floor came back UNCHANGED at 1.5 ms. Twelve
full runs of all 2585 arms: median ratio 1.95-2.03 in every size bucket above
50 us, but two-point excursions past the 2.6 threshold survive at every
magnitude (p99 2.9-3.3 even above 1 ms). Flagged arms per run by floor: 6.9 at
0.5 ms, 1.1 at 1.0 ms, 0.33 at 1.5 ms. Four arms flagged across the twelve runs,
each in exactly one; six arms that have ever flagged re-measure at exponent
0.97-1.09 over six doublings, at most 1.2 s at the cap. Descheduling was never
what made this sweep noisy -- a ratio from two points is.
Measured before publishing and it cost a correction: an earlier draft of this
change said the sweep "reports 0 candidates". The next run reported 2. Nine of
twelve instrumented runs are clean and eight consecutive shipped-script runs
under load flagged 0-3, so neither a clean run nor a flagged one is evidence
on its own -- the doc says that now.
Also corrects the pattern count in the same bullet: 150 -> 152, the script's
own printed total. The 0.3.4 CHANGELOG entry keeps its 150 as a snapshot.
No exported surface, no detector behaviour, no calibration touched.
802 passed, 129/129 classes, 6/6 gaps, 43 limitations, gitleaks clean.
TRACK 2. All seven were tested and released; none had a line in
docs/LIMITATIONS.md. Written where each belongs topically, not appended.
36 -> 43 items; README count moved with it. No code change, no release.
- The carrier split LOOSENS a lone anchor, not only tightens. Measured as
shipped: <a href="https://ext.example/p"> alone is active:raw-html-link at
MEDIUM -> quarantine_review on PRESET_USER_UPLOAD (warn on trusted); through
0.6.1 it was HIGH -> fail_secure. An <img src> to the same host is untouched.
Only the tightening direction had been documented.
- The no-URL narrowing's name set is a judgement about affordance no test can
derive. A name wrongly placed in _URL_AFFORDANCE_TAGS goes silently inert,
and no corpus can catch it: what it produces is an absence.
- hybrid-xss:script-tag and active:raw-html do not match the same strings.
<script src="a<b"> raises active:raw-html and NO XSS label -- the lexicon's
[^><] is the 0.3.3 ReDoS fix, HTML_TAG_RE consumes quoted runs atomically.
Disposition never moves; the label does, so that id is not a script census.
- A green ReDoS row is evidence only if seen red, and three rows never were.
Corrected while writing: the third row is
test_gate_is_bounded_on_the_long_attribute_arm, killed by 0.7.0's own
narrowing (separation 1.0x, 0.028s, no findings) -- not active-url-attr-value,
which is a payload-design rule, not a dead row.
- Every suite bound runs on process CPU time (tests/redos_clock.py); the
residual is that a BLOCKING hang would hang the suite. docs/redos-sweep.py
still times on time.monotonic(), so the published 1.5ms floor at N=8000 and
the "~23s at the cap" figure come from a different instrument than the bounds
they justify. Not re-derived; stated as not directly comparable.
- Under DEFAULT_ACTION_MAP the assessment axis adds exactly one judgement over
the disposition (NONE vs LOW). Consequence for this document: the published
FP rates are a risk statement only while NONE+LOW are the WARN pre-image.
- The ZWJ context test is one predicate on two surfaces, and the symbol is
private -- output imports sanitize's _is_joiner_in_emoji_sequence, so a
consumer importing it pins a name 1.x does not promise.
Verified: 802 passed, 129/129 classes, 6/6 gaps hold, grep -c '^- \*\*' = 43,
and the doc scanned for stray format characters (one U+200D remains, the
pre-existing composed emoji -- an accidental U+200C of my own was removed).
okf measured that 0.7.0's no-URL narrowing dispositions four raw-HTML
carrier forms (<a aria-label>, bare </a>, <Frame>, <video />) to warn
with no active-content finding -- verified here directly against
screen_output(..., PRESET_USER_UPLOAD), which returns the library's own
"clean: no findings" reason string (disposition.py:264). At their door
(llm-ingestion-okf 0.7.0), warn is the persist floor (inbox.py:139/323),
and their adapter forwards the original extracted text because
screen_output never hands back sanitized bytes -- defanging is a
separate, deliberate neutralize() call. So the same four forms land
written into the bundle, carrier present verbatim. Named per operator
decision; CHANGELOG.md's "35 limitations" and GATE-G-v1.md's counts are
dated snapshots of past states and are left as measured, not bumped.
35 -> 36: README.md's summary line is the only other live count.
Both the LIMITATIONS bullet and the 1.1.0 CHANGELOG entry said "both now
FAIL_SECURE at T2, before the allowlist is reached" and stopped there. True,
and stronger than what shipped: the old bullet's closing clause about T3's
scope went out with the rewrite, so the text read as though a pointer can no
longer reach the consumer tree through frontmatter.
Measured, not reasoned:
attester: attesters/sql_equality.py -> WARN (unchanged)
resource: attesters/sql_equality.py -> FAIL_SECURE (unchanged)
T3 inspects `resource` and nothing else, so an honest string under another key
rides through exactly as before -- scanned under T1 like any other frontmatter
value, but never allowlisted. That is by design and is not what 1.1.0 changed.
Restored in both places.
Also corrects the row arithmetic: 13 added and 3 retired, not "11 added, 1
retired". Net +10 and 802 were measured and are right; the parenthetical was
not, and 2400 != 2401 is a locked convention here.
Tag v1.1.0 does not move: the code is correct, the prose about it was not.
802 passed, 35 limitations, :43 still the bullet's anchor line.
Six live version surfaces bumped by hand (no sed -- provenance is never
bumped): pyproject.toml, __init__.__version__, README badge + Status + the
pinned pip install tag, docs/BRIEF.md, docs/ADOPTION-BRIEF.md Status, and
CLAUDE.md. ADOPTION-BRIEF's test count 792 -> 802.
NOT bumped, and deliberately: SECURITY.md's two `1.0.0` references name the
freeze BASELINE, not the current version -- "a payload that disposes WARN on
1.0.0 may dispose FAIL_SECURE on a later 1.x" is the promise this release
instantiates, so rewriting it to 1.1.0 would erase what it promised. The
GATE-G and PLAN-v1 numbers are the 1.0.0 gate record. The Forge repo
description carries no version (verified against the API last session).
This is the case SECURITY.md and the 1.0.0 CHANGELOG entry described in
advance: the exported surface is frozen, detection behaviour is not. No
exported name moved. A document that disposed WARN on 1.0.0 may dispose
FAIL_SECURE here; a consumer whose frontmatter carries an unquoted ": " in a
value will see those concepts refused at import, and quoting it parses.
Re-measured after the bump, alone: 802 passed, 129/129 classes, 6/6 gaps hold,
35 limitations.
T2 gives the mapping class no expressible form by design. Two routes escaped
that by parsing "successfully" into the wrong TYPE instead of raising:
sources:\n - uri: https://e.com/a -> the string 'uri: https://e.com/a'
attester: resource: attesters/x.py -> the string 'resource: attesters/x.py'
Only the first was documented (LIMITATIONS.md:43). The inline second colon was
found by measurement while closing it -- a real YAML parser refuses that line
outright, ours accepted it. Shipping the list half alone would have left a
LIMITATIONS rewrite that overclaims.
Same consequence either way: a pointer parked in a degraded mapping rides
through in a key the `resource` allowlist never inspects, and mode-b
import_bundle wrote the merged concept verbatim (WARN). Both now FAIL_SECURE at
T2, before the allowlist is reached.
The boundary is where YAML puts it, ground-truthed against PyYAML 6.0.3 rather
than reasoned: ": " and a trailing ":" are exactly the two shapes where a plain
scalar becomes a mapping. A colon carrying neither a space nor a line end opens
no mapping -- domain:security and https://e.com:8443/a still parse -- and a
quoted scalar is still a scalar. Over-blocking a conformant bundle is itself a
failure mode, so the seven admitted shapes get rows of their own.
Iron Law: the four rejected rows and both import_bundle rows were written
first and seen red (7 failures, each DID NOT RAISE) before okf.py was touched.
Suite 792 -> 802. LIMITATIONS stays at 35: the bullet is reworded, not
retired -- the restricted grammar is still a limitation, the silent misparse
is no longer part of it.
Three durability gaps from the release, none of which block anything.
1. Session G had no LANDET marker. Every other landed session carries one
with its sha and tag; the release that froze the surface did not.
2. D4's rationale existed only in STATE.md, which is LOCAL-ONLY and gets
overwritten every session. The consumer-promise section says in its own
words that promises live in this file and not in STATE, 'fordi STATE er
LOCAL-ONLY' -- so a decision NOT to fire one belongs here too. A future
session reading promise 1 would otherwise find no notification and no
reason, which is indistinguishable from having forgotten it. The
counter-reading is recorded with it: the wording ('enhver endring')
pointed the other way, and it is a real argument, not a strawman.
3. 98ebc07's message says 'Nine current-state surfaces bumped by hand'.
Eight were bumped. The ninth is the Forge description, which carries no
version string and so was structurally invisible to the 421-hit sweep --
the same blind-spot class as the status badge and Development Status.
Verified against the Forgejo API instead: 178 codepoints, no version, no
'alpha'. It needed no change, but 'verified' is not 'bumped' and a pushed
commit message cannot be amended.
Nine current-state surfaces bumped by hand. The classification sweep ran
FIRST, before the first edit: 421 hits on 'v?0.N(.N)' across all tracked
files, each read and sorted current-state vs provenance. Provenance is
untouched -- 'New in v0.4.0', 'measured against 0.3.1', every '0.7.0'
in a code comment or a census candidate name still says what it measured.
The sweep found two surfaces the plan's nine-item list did not name:
README's status BADGE (still 'alpha' -- a version string grep cannot see
it) and ADOPTION-BRIEF's test count, which said 791 against a suite that
runs 792. Both corrected.
pyproject also moves Development Status :: 3 - Alpha -> 5 -
Production/Stable, likewise invisible to a version grep.
CHANGELOG [1.0.0] references [0.3.0] and [0.3.1] for the behaviour
changes rather than repeating them, and carries the freeze point itself:
what is frozen (the exported surface), what is deliberately NOT (all
detection calibration), the three conceded limitations, the one known
open defect (:43), and the runtime-coverage gap -- no external consumer
has run 0.7.0.
No code changed. Per docs/PLAN-v1.md the release gate is the whole suite
green, not a new test: 792 passed, coverage matrix 129/129 + 6/6, exit 0.
D3: :492 (Severity carries disposition intent) and :460 (the input-cap
asymmetry) are the two limits that can only be closed by changing an
exported symbol. Both now say 'conceded for 1.x', with the 2.0.0
consequence spelled out, instead of 'deferred deliberately'.
D5: :88's 'a calibration fix is pending' is gone. The concession is
narrower than the other two -- no fix is promised, but it is calibration,
so one may land in any 1.x release without breaking the contract.
SECURITY.md carries the part an outsider acts on: the support window is
rewritten off 'pre-1.0', the freeze is stated as a promise about the
Python surface and explicitly NOT about detection behaviour, and the two
permanent concessions join the documented-boundaries list.
0.7.0 shipped with its corpus numbers deliberately absent: the census had only
run on reference-corpus, which the 0.6.0 narrowing had already emptied of
raw-HTML drivers, so it bounded the change rather than showing it. Both wiki
corpora are now measured through the pinned instrument, in one session because
they are living populations.
Under PRESET_USER_UPLOAD, 0.6.0 as shipped -> 0.7.0, each population against its
own denominator (the two wiki corpora share content and are never summed):
reference-corpus 389 docs 54 -> 53 ceiling 53
vendor-harvest 187 docs 62 -> 20 ceiling 18 42 of 44 achievable
generated-notes 552 docs 59 -> 15 ceiling 13 44 of 46 achievable
PRODUCTION matched C1 + D field for field in every population, which is the
check that the instrument and the shipped predicate have not drifted.
Two things the measurement changed rather than confirmed:
The pair is superadditive by 13 documents in BOTH wiki corpora. Alone, the split
frees 8 in each and the narrowing 21 and 23; together they free 42 and 44.
Shipping either alone would have measured as barely worth the label.
The TIGHTENS column reads 0 on both tiers in all three populations, but that zero
is empirical and thin: the split measured ALONE tightens 13 documents on the
trusted tier in vendor-harvest and 14 in generated-notes, and the narrowing
cancels each one. LIMITATIONS now says so explicitly, so nobody reads the zero as
'cannot happen' -- the escalation is still constructed and pinned by
test_split_tightens_the_trusted_tier_when_both_carriers_are_present.
generated-notes counted 552 documents, not the 550 the scratchpad probe saw.
Living corpus, measured fresh.
791 passed; coverage 129/129, 6/6 documented gaps hold.
Two changes that had to ship together, because they co-occur.
`active:raw-html-link` (MEDIUM) splits the click-required carriers out of
`active:raw-html`. The same URL was LOW as `[t](url)` and HIGH as
`<a href="url">` — an asymmetry produced by syntax, not by affordance, on a
carrier the markdown path has graded MEDIUM since 0.3.1. The event-handler test
runs first, so `<a onclick=...>` stays HIGH. The url-attribute branch stays HIGH
too: a name outside the active set has unknown rendering, and grading
`<Card src=...>` as a link would be reasoning rather than measurement.
The no-URL narrowing makes `</a>`, `<Frame>`, `<video />` and `<img alt=...>`
without `src` inert — `<base />`'s argument from 0.6.0 applied to the rest of the
name branch. It tests for the URL attribute's PRESENCE, not for a readable value,
so the fail-secure gap `_url_attr_is_external` leaves open is not reopened here.
WHY TOGETHER: the narrowing strips a document's `</a>`/`<Frame>` and what remains
is the `<a href=...>` the split grades down, so each alone leaves the document
blocked by the other's residue.
`active_tag_class` is now the classification point and `is_active_tag` wraps it.
The census patches the former: a boolean could only express a narrowing, never a
regrade, so every carrier candidate would have measured equal to PRODUCTION —
silently, and in the direction that reads as "no change helps".
TWO COSTS, BOTH RECORDED RATHER THAN GLOSSED:
- The split TIGHTENS the trusted tier. One finding becomes two, and >=2 findings
at MEDIUM+ trip the compound overlay, so a document carrying both an `<img src>`
and an `<a href>` goes WARN -> quarantine_review on PRESET_TRUSTED_SOURCE. On
that preset it is the only direction the split can move anything. The census
now reports a TIGHTENS column on both trust tiers against the previously
shipped row — "frees N" without "tightens M" is a one-sided number.
- `count` drops on documents containing `</a>`, a published field moving under a
meaning that did not change.
MEASURED: reference-corpus (389) 54 -> 53 fail_secure, tightens 0/0, and the
census `PRODUCTION` row equals its `C1 + D` candidate row for row. The census
also reproduces 133/3/13/108/25 exactly, so it is calibrated against every
published historical number. The two wiki corpora are NOT yet re-measured; the
tree says so explicitly in the docstring, LIMITATIONS and CHANGELOG rather than
carrying probe numbers as fact.
791 tests (was 759), coverage 129/129, 6/6 documented gaps holding. Version
bumped to 0.7.0 across every surface; no tag is set until the measurement lands.
`docs/fp-sweep.py` and `docs/rawhtml-census.py` produce the numbers published in
`docs/LIMITATIONS.md`, and both reach past the public API into private module
state -- `active_content._ACTIVE_TAGS`, `_EVENT_ATTR_RE`, `_URL_ATTR_RE`,
`_has_external_target`, `calibration.RISK_RANK`. A rename inside `src/` broke
them while the suite stayed green, and the breakage would have surfaced months
later, at the moment someone tried to re-measure a published claim. This was the
last uncovered contract in the repo.
The coverage found a live one. `rawhtml-census.has_external_url_attr` parsed
attributes with its own pattern instead of calling the shipped reader, and the
copy had drifted on two shapes:
- a URL attribute reached through a prefix. `_URL_ATTR_RE` matches `src=`
inside `data-src=` on a word boundary, so the shipped predicate reads the
value and blocks; the census's own name table saw `data-src` and skipped it.
- a multi-candidate `srcset`. The shipped reader splits on `[,\s]+` so a
relative first candidate cannot mask an external one behind it; the census
tested the whole attribute value as a single URL.
Both made the `A` candidate rows free documents the shipped predicate keeps --
under-counting against the `PRODUCTION` row printed directly beside them, which
that row exists to expose. The census now delegates to
`active_content._url_attr_is_external`, so there is one reader, not two. This
repo already carries the general form of that lesson in `docs/URL-SHAPE.md`:
three consumers reconstructed a predicate from prose and each got a different
wrong answer.
NO PUBLISHED NUMBER MOVED. Re-measured against all three live populations after
the fix, in one session each: reference-corpus 389 docs (A frees 3, base-url 13,
both 25 -- the numbers in `docs/LIMITATIONS.md`, unchanged), vendor-harvest 187,
generated-notes 550. `PRODUCTION` equals `A + base-url` in all three (108/108,
98/98, 88/88), and vendor-harvest exercises the corrected branch for real (8
external `Card` attributes). The defect was latent, not published.
The tests are scoped to what a test here can honestly hold. The corpora live
outside this repo in private consumer repos, so neither script can be run end to
end from the suite and a stand-in corpus would only pin a fiction. What is
pinned: every imported name still exists with the shape used; `fp-sweep`'s
metric guard fails when the action map is re-mapped (a guard that cannot fail
protects nothing); `measure` still reads `.disposition` and `.assessment` and
excludes empty files from the denominator; the census's in-process patch point
still moves the gate, so a census patching a dead symbol cannot print six
identical rows and read as a finding; `PRODUCTION` equals `A + base-url` across
every branch of the predicate; and both scripts still refuse an argument-less
run rather than measuring nothing.
759 passing (was 736). Coverage matrix unchanged at 128/128 with 6/6 documented
gaps holding. The `736` in `docs/ADOPTION-BRIEF.md` is scoped "as of v0.6.1" and
is correct for that tag; it moves to 759 at the next version bump.
a59184b fixed a hard-block that reached every emoji-composed document, and
llm-ingestion-okf was told so -- while the fix sat untagged on main. A consumer
cannot pin what has no tag, so the notice was a promise the repo had not kept.
SURFACES MOVED (the eight the 0.5.0 sweep established, plus the ninth verified)
pyproject.toml 0.6.0 -> 0.6.1
__init__.py 0.6.0 -> 0.6.1
CHANGELOG.md [0.6.1] entry, [Unreleased] reset to "Nothing yet."
README.md badge, install tag @v0.6.1
docs/ADOPTION-BRIEF.md `v0.6.0` x2, and 727 -> 736 passing
CLAUDE.md one clause: ZWJ judged by context, not identity
SECURITY.md `0.6.x` -- still true at 0.6.1, verified, not moved
docs/BRIEF.md `v0.6 (alpha)` -- likewise
README.md:36 `v0.6`, alpha -- likewise
forge description re-verified THIS release: 178 codepoints, no version
claim. Not inherited from 0.6.0's check.
PATCH, NOT MINOR -- and it was asked, not assumed. 6bcb898 made "loosens the
upload door" the criterion for minor, and this loosens it too, so the reading
was put to the operator with the count rather than settled quietly. 0.6.0 was a
policy choice (`<base>` left the active name set by decision); this restores a
contract the module already published, against a class never meant to be
blocked. Measured loosening: 1 document of 1126 across the three FP populations
(reference-corpus 1/389, vendor-harvest 0/187, generated-notes 0/550), against
0.6.0's 25 + 2 + 2. Operator chose patch.
VERIFIED ON THE BUMPED TREE, NOT THE PRE-BUMP ONE
736 passed; coverage 128/128 recall, 6/6 documented gaps hold
`git grep '0\.6\.0'` returns provenance only -- LIMITATIONS history, the
census script, source docstrings, tests, README's pre-0.6.0 numbers
docs/LIMITATIONS.md: 34 items, README says 34
__version__ and pyproject agree at 0.6.1
no tracked file carries a stale current-state version or test count
The sweep tool itself was wrong first: `git grep -E '\b727\b'` returns nothing,
because POSIX ERE has no `\b`. An empty result read as "clean" when the claim
was there. Every sweep above is plain `git grep`.
Still to prove before this is announced: a scratch-venv install at the tag with
the resolved version asserted -- the README line above is only true once v0.6.1
resolves.
`_ZERO_WIDTH` (sanitize, input) and `_ZERO_WIDTH_CPS` (output,
`_scan_invisible_carriers`) tested U+200D on codepoint membership alone.
`disposition._CARRIER_LABELS` grades both as any-tier FAIL_SECURE with no
appeal, so any first-party document containing a ZWJ-composed emoji --
professions, families, skin tones, flag variants -- was hard-blocked forever.
Reported by ms-ai-architect; confirmed here against the code.
The strip was the worse half and was not in the report: sanitize *removed* the
joiner, silently decomposing the emoji into two unrelated ones. A module whose
contract is "only ever removes carriers" was corrupting content.
The fix is the one our own lexicon row `unicode:zero-width-in-word` (`\w[ZW]\w`)
already used: judge the joiner by CONTEXT, not identity. A ZWJ is exempt only
when BOTH neighbours are emoji-context codepoints. Half-context is not context,
so `a<ZWJ>{emoji}` stays a carrier and an attacker cannot buy exemption with a
single emoji.
Blocks, not an emoji table. Measured against Unicode 17.0's
`emoji-zwj-sequences.txt`: 1614 RGI sequences use 122 distinct codepoints
adjacent to a ZWJ, and the five ranges cover 122/122. The measurement earned
its keep -- the hand-reasoned candidate table missed U+2194, U+2195 and U+2B1B.
Shipping the RGI list itself would be exact on the day it landed and stale at
the next Unicode release, reopening this same false positive for every new
emoji; whole blocks carry the unassigned headroom (458 Cn codepoints) that
future emoji are allocated into, so the table does not age.
The predicate is defined once in sanitize and imported by output. A second copy
is how the input side stops flagging while the output side keeps blocking; the
cross-surface test asserts the two agree on six inputs.
Two residuals, both in LIMITATIONS (33 -> 34, README bumped): a ZWJ between two
emoji is now exempt and could carry a covert channel (one emoji per bit, cannot
split a word); and U+200C (ZWNJ) still has no context test, so Persian, Arabic
and Devanagari documents -- where it is orthographically required -- stay
blocked. That needs a script-based criterion and no corpus is here to verify it
against, so it is parked as a known FP class rather than guessed at.
736 green (was 727), coverage 128/128, 6/6 documented gaps hold.
The narrowing landed in 736f370 and its prose asserted `0.6.0` in thirteen places
-- README's public front page among them -- while every version surface still read
0.5.0. That is the same defect class the last three commits were spent correcting:
a published number no measurement backs. Two ways out, land it or neutralize the
references; operator chose to cut.
SURFACES MOVED (the eight the 0.5.0 sweep established, plus the ninth verified)
pyproject.toml 0.5.0 -> 0.6.0
__init__.py 0.5.0 -> 0.6.0
CHANGELOG.md [Unreleased] -> [0.6.0], fresh [Unreleased]
README.md badge, `Status: v0.6`, install tag @v0.6.0
SECURITY.md support window `0.5.x` -> `0.6.x`
docs/BRIEF.md `v0.5 (alpha)` -> `v0.6 (alpha)`
docs/ADOPTION-BRIEF.md `v0.5.0` x2, and 717 -> 727 passing
CLAUDE.md `v0.5 (alpha)` -> `v0.6`, plus what 0.6.0 changed
forge description 178 codepoints, carries no version claim -- verified,
not moved. A surface can be checked and stay still.
Measurement provenance is left alone, as in 0.5.0: `tests/test_disposition.py`'s
"0.5.0 axis separation", `disposition.py` and `calibration.py` docstrings,
LIMITATIONS' 0.5.0 reference, every dated claim in docs/PLAN-v1.md. Bumping those
falsifies the record rather than updating it.
MINOR, NOT PATCH: 0.6.0 loosens the upload door. A document whose only finding was
a doc-relative URL attribute on an inactive tag name, or an attribute-less
`<base />`, now WARNs where it was held -- 25 documents in the reference corpus, 2
in each wiki corpus.
VERIFIED BEFORE COMMITTING, NOT AFTER
727 passed; coverage 128/128 recall, 6/6 documented gaps hold
docs/LIMITATIONS.md: 33 items, README says 33
rawhtml-census PRODUCTION row equals `A + base-url` on all three populations --
the shipped predicate measured, not a hypothesis about it
redos-sweep: 0 candidates of 152 patterns
docs/fp-sweep.py still imports the private names it reaches into
no tracked file carries a stale current-state version claim
Still to prove before the tag: `git show` over this commit's README, and a clean
clone install at this sha.
`is_active_tag`'s URL-attribute branch was a presence test: any element carrying
`href=`/`src=`/`action=` graded HIGH regardless of where the URL pointed. An MDX
`<Card href="/en/agent-sdk/quickstart">` reaches no attacker-controlled host, and
neither does APIM policy XML's `<set-header>`. It now requires an external target
-- the rule the markdown paths have applied since 0.3.1. `<base>` left the active
name set in the same change: HTML's `<base>` has its whole affordance in an `href`
the attribute branch still catches, and APIM's attribute-less `<base />` is inert.
Measured before and after in ONE session against one corpus state, because two of
the three corpora are living and a split would mix this with re-harvest drift:
reference-corpus 389 docs 133 -> 108 (ceiling 107)
vendor-harvest 187 docs 100 -> 98 (ceiling 62)
generated-notes 550 docs 90 -> 88 (ceiling 49)
96% of the achievable reduction in reference-corpus, 5% in the wiki corpora. The
two classes had to be measured TOGETHER -- alone they free 3 and 13 documents,
together 25, because a document carrying one usually carries the other.
The second surface: `neutralize` imported `is_active_tag` by name, so this would
have silently narrowed the opt-in mutator too -- and no test discriminated the two
halves, since every `neutralize:raw-html` payload stays active under any narrowing
considered. That test is written first here. The predicates are now separate
symbols; the mutator keeps defanging anything, because over-defanging is auditable
and blocks nothing while under-defanging hands a human a live construct.
Behaviour change: a document whose only finding was one of these classes now WARNs
instead of holding. Detection is unchanged -- 128/128 classes, 6/6 gaps hold.
Self-safety: reading an attribute VALUE needs a pattern the presence test lacks. It
reuses the same literal alternation so no new run shape enters the table; its
`_REDOS_PAYLOADS` row denies the `=` the pattern requires, since a unit supplying it
matches at once and never exercises the run (the lexicon's `script-tag` row is the
cautionary case). 0.031-0.046s across five attack shapes at 100_000 chars against a
2.0s bound; `docs/redos-sweep.py` reports 0 candidates of 152. An attribute the
presence test saw but the value parser cannot read counts as external -- fail secure.
`docs/rawhtml-census.py` gains a PRODUCTION row that re-measures the shipped
predicate rather than a hypothesis, so a published number and the code cannot drift
apart unnoticed. README's limitation count moves 34 -> 33.
727 passed (was 717).
Yesterday's correction fixed a wrong causal claim and published a second one.
It reported that narrowing the URL-attribute branch frees 3 documents in
reference-corpus and left the reader to conclude the over-reach is cheap.
Measured together with the `<base>` name-branch fix it frees 25, against a
ceiling of 26 — 96% of what the detector costs that population. A document
blocked by two over-reach classes is freed by neither alone.
The same pair frees 2 of a 38-document ceiling in vendor-harvest and 2 of 41
in generated-notes. The over-reach is nearly the whole raw-html cost in Azure
APIM policy XML and nearly none of it in vendor documentation.
Three further corrections:
- `<base />` appears in 25 reference-corpus documents, not 30. The published
count came from `grep '<base'`, which also matched the literal
`<base64_string>` placeholder — not a tag this detector fires on.
- Any fix moves TWO surfaces: `neutralize` imports `is_active_tag` by name, so
narrowing it also stops the opt-in mutator defanging the same tags. No test
covers that half; the suite's `neutralize:raw-html` payloads stay active
under every narrowing considered.
- Adds `docs/rawhtml-census.py` so the ladder is reproducible instead of
living in a session scratchpad. Corpus roots are arguments, never hardcoded.
Measured under the real edit, not reasoned: the suite fails exactly one test,
`test_raw_html_overblocks_are_still_high[relative-href-on-inactive-name]`,
which exists to force this doc update when an over-block closes. Recall holds
at 128/128 and all 6 documented gaps still hold.
The "Measured, document by document" bullet attributed `active:raw-html` in
52 of vendor-harvest's 98 to the relative-URL-attribute over-reach. Measured
by re-running the gate with that branch narrowed to external targets only, it
frees 1 document there, 1 in generated-notes, 3 in reference-corpus. The
over-reach is real and nearly free; the rate is driven by the NAME branch
(`<a>` 298, `<frame>` 94, `<img>` 63 per wiki corpus; `<base>` 113 in
reference-corpus).
Two further corrections the census forced:
- The over-reach is not only MDX. In reference-corpus it lands on Azure APIM
policy XML (`<set-header>` 50, `<ip-filter>` 3, `<set-query-parameter>` 2).
- New bullet: APIM's attribute-less `<base />` collides with HTML's `<base>`,
whose whole affordance is `href`. 30 of reference-corpus's 389 documents.
Also pins corpus provenance. "Every population was swept twice and reproduced
its counts exactly" is false for two of the three: vendor-harvest and
generated-notes live in claude-code-llm-wiki, which re-harvests per Claude
Code release. At its commit aba87e2 they now give 100/187 and 90/550;
reference-corpus is static and reproduced 133/389 exactly. The published table
keeps its original counts — measurement provenance is never silently bumped —
and gains the provenance it was missing.
README limitations count 33 -> 34.
TWO STALE PREMISES IN THE SAME SECTION, BOTH NOW MEASURED
1. The pin. Session G's gate was written against okf pinning the guard
`>=0.2,<0.3` with `[tool.uv.sources]` tag `v0.2.0` — true when written
(their HEAD 4ea00a9), and the whole reason the gate says a green answer
proves nothing: `<0.3` excluded the versions the fixtures were meant to
exercise, so uv resolved v0.2.0 and came back green for free.
Read from their pyproject.toml today, not from their coord message: they are
on `>=0.3,<0.4` with tag `v0.3.4`. That inverts the consequence — an
unchanged tree now resolves v0.3.4, which is inside what the gate asks for,
so the null result is gone.
Two things follow and neither is settled here. `<0.4` excludes 0.4.0 and
0.5.0, so the axis separation and the input-cap refusal are outside anything
okf can measure today. And the gate says "passes against guard 0.3.1" while a
run today measures 0.3.4 — whether that counts as satisfied is an operator
decision, deliberately not taken. The transitive-arrival bullet is corrected
the same way: an accidental consumer lands on v0.3.4 now, not v0.2.0.
2. The file list. It named four release surfaces. The 0.5.0 sweep found five
more carrying a version claim that no release had ever touched — and one of
those five was `**Status:** v0.3`, which this very line already named, and
0.4.0 missed anyway. Writing down a trap is not applying it.
Replaced with the nine current-state surfaces, plus the rule that made the
sweep safe: sort every hit into current-state or measurement provenance
BEFORE editing, because a version sweep also hits "verified identical on
0.2.0 and 0.3.1" and bumping that falsifies the record. So it can never be a
sed pass.
The key-assumption test went with it. "grep all four files" presupposed the
list it was supposed to verify. It is now a git grep over all tracked files
with each hit classified, plus `git show <pre-tag-sha>:README.md` — the check
that proves what the tag will carry, run while the tag does not yet exist.
That ordering is the fix; v0.4.0 verified afterwards and carries a stale
README permanently as a result. SECURITY.md gets a note that at 1.0.0 its
sentence is rewritten, not bumped: "pre-1.0" stops being true.
0.5.0 is the axis separation `de09711` built: `Risk` (assessment) alongside
`Disposition` (action), `Policy.action_map` as the supported override, and the
fail-closed path pinned to both axes. Additive and measured to be so — 717
passing with no test changed, matrix 128/128 with 6/6 documented gaps, the
`PRESET_USER_UPLOAD` grading table unchanged row by row. Plus the field FP
measurement (`d1bff60`) and the 0.3.3 behaviour-change correction (`d3d0928`).
WHY THIS COMMIT TOUCHES EIGHT FILES AND 0.4.0's TOUCHED THREE
0.4.0's release commit updated CHANGELOG, pyproject.toml and __init__.py, and
deferred README deliberately: the install block should not name a tag before a
clean-venv install had proven it resolved. Sound reasoning, and the proof step
never ran — so tag v0.4.0 permanently advertises v0.3.4. The tag is not moved.
The ordering is.
Sweeping every tracked file for a version claim, instead of ticking the four
surfaces the checklist named, found five more that no release had ever touched:
SECURITY.md "pre-1.0 (0.2.x)" — the one with a consequence for an
outsider: it named a support window two minor lines
behind the code.
README.md "**Status:** v0.3" — the front page, stale since 0.4.0.
docs/BRIEF.md "v0.2 (alpha)" — stale since 0.3.0.
CLAUDE.md "v0.2 (alpha)" and "12 moduler" where src/ has 15.
docs/ADOPTION-BRIEF "703 passing" where the suite is at 717.
Measurement provenance is deliberately left alone: "New in v0.4.0", "verified
identical on 0.2.0 and 0.3.1", "measured against the v0.3.1 tag", every
"post-0.4.0 tree" in LIMITATIONS. Bumping those falsifies the record instead of
updating it, which is why this cannot be a sed sweep — the surfaces have to be
sorted into current-state and provenance before a single edit.
Found because llm-ingestion-okf took our report of this defect class as a
hypothesis about their own repo, measured it, found a worse instance on their
public front page, and sent back the generalization: writing down a trap is not
the same as applying it.
VERIFIED BEFORE COMMITTING, NOT AFTER
717 passed; coverage 128/128 recall, 6/6 documented gaps hold
docs/LIMITATIONS.md: 33 items, README says 33
fp-sweep reproduced all three published numbers exactly on the bumped tree —
vendor-harvest 98/185 (53.0%), generated-notes 88/547 (16.1%),
reference-corpus 133/389 (34.2%) — and self-docs runs clean, so the
untested script survived the bump it imports names from
forge description: 178 codepoints, under the 180 cap
no tracked file carries a stale current-state version claim
Still to prove before the tag: a clean-venv install from this commit's sha, and
`git show <sha>` over the README. The install proves the package builds; only
the grep proves the text the tag will carry is right. That second check is the
one the old ordering could not perform, because by then the tag existed.
