A consumer measured our frontmatter gate against 0.2.0 and asked us to confirm
against a newer guard, correctly noting that a divergence would be a version
difference rather than a contradiction. Measured at 0.3.1: no divergence. Both
their results reproduce exactly.
Two things are sharper than what we documented yesterday:
ALL THREE ROUTES TO A MAPPING FAIL, each on a different rule -- flow {k: v} on the
disallowed value-start indicator, block on the nested-mapping check, dotted keys on
the key pattern. So the mapping CLASS has no expressible form; it is not a choice
between two shapes where one is better. That matters because OKF v0.2's `generated`
IS a mapping (`by` is required when it is present), so it cannot be expressed at
all. Pinned by a test asserting the three failures stay distinct.
T2 RUNS ON DOOR C ONLY. parse_frontmatter is referenced nowhere in the door A/B
persist path, so the same frontmatter that FAIL_SECUREs through import_bundle
passes screen_output unremarked. The grammar bounds what a consumer can RECEIVE,
never what a producer can EMIT -- which is the question a consumer's emitter design
was blocked on.
Also corrects one of our own rows: we reported "sources block list" as a single
FAIL_SECURE case. The parser distinguishes three shapes -- flat scalars parse
correctly, one key per item misparses silently to a string, two keys per item hard-
rejects. A one-element `verified:` block list passes today.
631 -> 642 passed. README item count synced.
Three consumers reconstructed is_ordinary_url from prose we sent in coordination
messages and each produced a different wrong number on a real corpus: one omitted
the base64 20-char floor and fired on path words like /blog/; one omitted the
opaque-token condition entirely and undercounted; one computed Shannon entropy
over whole filenames instead of tokens and concluded the 4.4 floor over-blocks
ordinary documents. Same cause each time -- our prose described the rules without
their tokenizer.
docs/URL-SHAPE.md states the algorithm in order, spells out the separator class
and all three length floors, and lists the three reconstruction errors as worked
counter-examples. Its example table is parsed and asserted against the real
predicate by tests/test_url_shape_doc.py, so the reference cannot drift from the
code -- all 18 rows verified load-bearing.
LIMITATIONS.md brought current with the field measurements:
- Percent-escape is no longer zero. Two English corpora measured 0; a 389-file
Norwegian/Microsoft corpus found 10, all Norwegian (%C3%B8, %C3%A5 are just
o-slash and a-ring). It is a non-ASCII-language tax, and both zero-measuring
corpora being English was a sampling bias invisible from inside.
- The query over-block now has THREE disjoint benign populations: utm_* tracking,
content identity (?v=, ?channel_id=), and Microsoft Learn's ?view= version
selector. No parameter-level remedy covers any two, which moves this from a
conclusion to a settled constraint on 0.4.0.
- Legitimate CDN asset ids trip the hex branch permanently; the branch is otherwise
precise (no other FP in 2401 distinct URLs) and stays.
- Raw HTML with a relative URL attribute is HIGH though it reaches no external
host, and end tags are counted.
- OKF frontmatter: a one-key block-sequence item is silently misparsed to a string
where two keys hard-reject, so a pointer can ride past the resource allowlist.
Consequence: a conformant OKF v0.2 concept cannot traverse door C at all, since
both backward-breaking migration targets are nested. Fail-secure, but a
compatibility wall that needs a deliberate parse-safety decision.
- A persist gate cannot cover execution risk, and that boundary is unowned.
New behaviour claims are pinned by tests so a closed concession fails and forces
this doc to be updated. 593 -> 631 passed.
Two consumers measured the 0.3.1 URL-shape rule against real corpora on the
same day. Record what they found, and pin the dispositions both of them
inferred wrongly.
Percent-escape FP: zero occurrences across both corpora (0/347 external URLs
in a 527-document vendor-docs corpus, 0/81 in a capture store). This bounds
the shape rather than closing it -- a consumer that slugs filenames from
titles produces %20 systematically, and that corpus is still unmeasured.
Query FP: the over-block that actually occurs. The two corpora hit disjoint
benign populations -- 16/16 publisher-authored utm_* tracking versus 35/35
content identity (?v=, ?channel_id=) where the parameter IS the resource.
An allowlist keyed on tracking-parameter names resolves the first entirely
and the second not at all, so no parameter-level remedy covers both. That
is input to the 0.4.0 axis-separation scope, not a fix here.
Both consumers reported dispositions they had inferred rather than run, and
both were wrong: a query-carrying link is MEDIUM, so it is held or warned,
never hard-failed. tests/test_wiring.py now pins that, plus the active-tag
gate -- counting raw HTML tags overcounts what the gate flags, since
formatting markup is inert and only name/on*=/URL-attr tags are active.
593 passed.
v0.3.0 made the untrusted upload path unusable: measured on both doors, an
ordinary remote image fail_secure'd and an ordinary link/autolink/refdef
quarantined, so only documents without external references persisted.
Two independent defects compounded; neither fix works alone:
1. `markdown-image: HIGH` fired on any external image. The exfil primitive is a
URL that moves bytes outward, not an image. `is_ordinary_url` now grades on
shape - http(s)/protocol-relative, no query, no userinfo, no percent-escape,
no opaque host label or path segment -> LOW; anything data-carrying keeps the
carrier's severity. raw-html and data: URIs stay HIGH unconditionally.
Opacity reuses entropy's primitives; floors calibrated against real doc URLs
(worst legit token H=4.08, exfil segments 4.36-4.54) and frozen in
calibration.
2. The quarantine_default floor fired on ANY finding, a premise that broke when
every ordinary link became a finding. It now fires at MEDIUM+ - a no-op for
every detector that shipped before 0.3.0 (no LOW/INFO exists), which is what
makes this a patch rather than a minor.
The corpus blind spot that let this pass 522 green tests is closed: the FP
corpus carries realistic markdown and is asserted on the OUTPUT gate under
PRESET_USER_UPLOAD, with a counter-corpus of exfil-shaped URLs that must still
block. Beaconing and short opaque segments are conceded in LIMITATIONS and
asserted by the coverage matrix rather than papered over.
No new public API; no new preset (0.4.0 work); allow_reserved default unchanged.
Answers the gap that the README said what it does NOT stop (a long limitations
section) but never plainly listed what it DOES. Add a 'What it protects against'
section high up: attack classes grouped by OWASP anchor (LLM01 injection + 83
lexicon classes + carriers, LLM02 egress, LLM05 EchoLeak, LLM06 agency, LLM10
fail-secure, OKF T1-T6, container front-end), each driven by a live coverage-matrix
payload. Move the full honest-limitations list to docs/LIMITATIONS.md; README keeps
a high-impact summary + link. Net: protection and limits read in balance, 261 -> 216
lines. Coverage 126/126 and 522 tests unchanged; every class listed is real.
