# Honest limitations Conceding these plainly is itself a control — it prevents the false assurance that a green scan means safe content. The README carries a summary of the highest-impact items; this is the full list, each with the mechanism. - **Structural unsolvability at the text layer.** Pattern/lexicon detection is bypassable in isolation; character-injection and novel phrasings evade it. The *contract* (tool-less transform, capability isolation, fail-secure) carries the security — the lexicon is defense-in-depth, not a wall. - **A lone HIGH finding in trusted prose disposes to WARN, not quarantine.** Under `PRESET_TRUSTED_SOURCE`, trust-scaling downgrades a single HIGH to WARN, and one HIGH is not "compound" (escalation needs ≥2 findings at MEDIUM+). So a HIGH injection reproduced verbatim under a *trusted* policy persists with a WARN. This is by design: if your "trusted" sources can carry attacker-influenced text, run them as untrusted (or add a quarantine floor). - **The quarantine floor fires at MEDIUM+, and is a no-op under the shipped upload preset.** Through 0.3.0 `quarantine_default` floored *any* finding to QUARANTINE_REVIEW. That premise ("a finding is the exception") broke when the active-content detector made every ordinary markdown link a finding, so 0.3.1 raised the floor to MEDIUM+. Under `PRESET_USER_UPLOAD` (untrusted) a MEDIUM already escalates on trust alone, so the floor still changes no outcome there; it is live only for a caller-defined *trusted* policy that opts into `quarantine_default`. Documented so the preset is not over-read: a LOW finding on an upload now disposes WARN. - **Semantic / factual poisoning is invisible** to lexicon + entropy: a false claim in clean prose carries no suspicious token. **Highest impact for a wiki.** The `grounding` module ships only a `SourceGroundingCheck` *seam* — the deterministic core does not judge semantics; a `[judge]` implementation must be plugged in. - **Adversarial-ML evasion** can survive normalization; **tokenizer mismatch** between scanner and model leaves gaps. **Latent / dormant memory poisoning** is not judgeable at write time. - **Dormant / broken-link injection** in a linked corpus (e.g. an OKF bundle): a link to a not-yet-existing target passes a per-concept write-time scan clean — the payload is planted later, when that target is written. `link_graph` surfaces the *dangling* edge as the signal, but catching the payload needs cross-write re-scan over time (the caller's disposition call). - **OKF reserved files (`index.md` / `log.md`).** In a *received* bundle these are legitimate structure, so mode-b `import_bundle` scans their body and frontmatter (an injection in a directory listing is caught) rather than path-rejecting the conformant bundle. A front-end materialising individual uploads keeps the opposite rule (`allow_reserved=False`): a reserved basename is a listing-shadow and refused. - **OKF frontmatter is a restricted grammar: the mapping class has exactly one expressible form.** Gate T2 accepts a line-oriented subset deliberately — full YAML is a larger parse-attack surface than a write-time gate needs. Flow sequences (`[a, b]`) and nested mappings are *rejected outright*, which fails secure. **Three of the four routes to a mapping fail, each on a different rule** — block (`k:\n sub: v`) on the nested-mapping check, dotted keys (`k.sub: v`) on the key pattern, and the inline second colon (`k: sub: v`) on the mapping-construct check. **The fourth, the flow form, is admitted only when every key is on an allowlist** (`by`, `at`, `from`, `to`, `id`, `title`, `author`, `usage_count`, `last_modified` — the keys SPEC.md @ `62432a09` §5.1/§5.2 names inside a mapping) and every leaf is a plain scalar, itself run through the same value predicates as a top-level scalar. Nested collections, quoted leaves, duplicate keys, an empty or unclosed mapping, and `{a:b}` (which PyYAML 6.0.3 reads as the *key* `a:b`, not as a scalar) all raise. The form is expressible, never trusted: the allowlist inspects every key, which is the property that carried the security when the blanket refusal was doing the enforcing. **`resource` is deliberately off the allowlist** although §5.1 names it inside a `sources` entry — it is a pointer rather than a label and the only key T3 exists for, so admitting it would let `executor: { resource: skills/run.md }` carry an executable-code pointer through a key the https allowlist never inspects. What else survives is scalars and flat lists of strings. **Two routes used to degrade into a string instead of failing, and that defect is closed in `1.1.0`**: a block-sequence item carrying exactly one key (`sources:\n - uri: https://e.com/a` yielded the *string* `'uri: https://e.com/a'`) and the inline second colon (`attester: resource: attesters/sql_equality.py`, which a real YAML parser refuses outright). Both parsed "successfully" into the wrong *type*, and a pointer parked in one rode through in a key the `resource` allowlist never inspects — mode-b `import_bundle` returned WARN and wrote the merged concept verbatim. Both now FAIL_SECURE at T2, before the allowlist is reached. **What closed is the type confusion, not pointer-smuggling as a class:** T3 still inspects `resource` and nothing else, so an honest *string* under another key rides through exactly as before — `attester: attesters/sql_equality.py` is WARN, while the same path under `resource:` FAIL_SECUREs. That is by design (the string is scanned like any other frontmatter value under T1) and it is not what `1.1.0` changed. **The boundary is where YAML puts it**, ground-truthed against PyYAML 6.0.3: `": "` and a trailing `":"` open a mapping and are refused; a colon carrying neither a space nor a line end (`domain:security`, `https://e.com:8443/a`) does not and still parses, as does a quoted scalar (`- "uri: x"`). Quotes are retained rather than stripped — a divergence from YAML that remains, pinned in `tests/test_okf.py`. - **T2 constrains import, not emission.** The frontmatter grammar runs on `okf.import_bundle` (door C) only — `parse_frontmatter` is referenced nowhere in the door A/B persist path, so frontmatter that fails secure on import passes `screen_output` unremarked. The grammar therefore bounds what a consumer can *receive*, never what a producer can *emit*. Verified identical on 0.2.0 and 0.3.1. - **An OKF v0.2 concept traverses the external-import path only if its `sources` are flat.** The wall used to be total: both of v0.2's backward-breaking migration targets are mappings — `timestamp` → `generated.at`, and body `# Citations` → a `sources` block list of mappings — and a consumer measured **0 of 53** upstream concepts through the gate. The trust and provenance layer now passes in its spec form (`generated`, `verified` bare or listed, `usage_window`), so `generated.at` is no longer a wall. **`sources` still is**: SPEC.md writes each entry as a block mapping under a block sequence (`- id: …\n resource: …`), and that carrier stays refused — it is the shape whose one-key degradation smuggled a pointer before `1.1.0`, and reopening it is a separate parse-safety decision, not a corollary of the flow form. A concept whose `sources` are flat strings, or absent, imports. The dangling-or-substituted `executor`/`attester` pointer question stays out of reach for the same reason: both are mappings whose payload key is `resource`. - **A persist gate cannot cover execution risk.** OKF v0.2 introduces concepts whose purpose is to *name code to be run* (`runtime`, `executor.resource`, `attester.resource`). This library answers "is this safe to **store**"; executable code carries its risk at **run**. A file that is harmless to persist can be harmful to point at. Upstream defers the attester ABI and sandboxing to a future revision, so there is no runtime contract to gate against — the execution boundary is *unowned* across the stack rather than covered by anyone's roadmap, and no tightening of a write-time scanner would change that. - **A document that *describes* attacks is a false positive.** Content documenting prompt-injection payloads (security notes, this project's own corpus) trips carrier-strip / fail-secure. At the text layer "*about* an attack" and "*carrying* an attack" are indistinguishable; such content needs a deliberate, explicitly escaped path, never a silent allow. - **Bilingual text trips the Cyrillic/Latin homoglyph rule.** `homoglyph:cyrillic-latin-mix` (MEDIUM) flags a Latin letter adjacent to a Cyrillic look-alike, so genuine bilingual prose → MEDIUM → under untrusted → QUARANTINE_REVIEW — a real false positive for an inbox that expects multilingual content. **Conceded for 1.x: no fix is promised.** The rule fires on codepoint adjacency, which genuine bilingual prose produces as readily as a substitution attack does. Narrowing it is a calibration question, not an API one, so a fix may land in any 1.x release without breaking the contract — but none is scheduled, and a caller that ingests multilingual prose should raise its untrusted-tier threshold rather than wait for one. `SECURITY.md` lists this as a documented boundary, not a vulnerability. - **Insider in-place edits** by a trusted author are out of the untrusted-content threat model. - **Text-only.** The core is `text -> findings`: it parses no files (no `pypdf`/`python-docx`/archive deps). Extract text first, then scan it with the high-untrust upload provenance. OCR-embedded instructions and multimodal stego are out of scope beyond the sanitizer's character-layer stripping. - **Uploaded files: only the *extracted text* is scanned.** The dev-scoped OKF inbox showcase (`tests/test_okf_inbox_uploads.py` + `tests/inbox_frontend.py`; parsers in the `[dev]` extra, never core `dependencies`) reads `.txt`/`.md`/`.csv`/`.docx`/`.pptx`/`.xlsx`, folders and `.zip`, materializes an OKF bundle, then guards it. What survives extraction is **out of scope**: macros, OLE/embedded objects, OCR-needing images, font/render stego, encrypted files — the binary layer needs a separate scanner. The front-end owns the container threats it *can* see (zip-slip → path gate, zip-bomb → size cap, symlink refusal, CSV/XLSX formula-lead cells). **`.pdf` is a deliberate concession:** a top-level `.pdf` is *refused as unsupported* rather than half-scanned (a PDF parser is disproportionate for a dev showcase, and the OCR/stego it would smuggle is already out of scope). One known gap: the numeric `-`/`+` CSV false positive (a typed XLSX numeric cell does not trip it). - **Lexicon findings are deduplicated by pattern id** — `count=1` and the first offset are reported, so a class matched across several channels collapses to one finding at its first location: a deliberate readability tradeoff. - **Active-content severity grades on URL shape, so a pure *beacon* is only LOW.** Since 0.3.1 a URL that merely names a remote document (bare path, no query, no userinfo, no percent-escapes, no opaque segment) is LOW, and only a URL that can move bytes outward keeps HIGH/MEDIUM. The deliberate hole: `![x](https:// evil.test/pixel.png)` on an attacker-controlled host still *fetches* when a renderer touches it, leaking reader IP, user-agent and timing. Grading the fetch itself would re-block every ordinary document, which is precisely the 0.3.0 regression this replaced — so beaconing is conceded, not covered. - **Short opaque URL segments slip through the same grading.** Opacity is decided by `entropy`'s primitives: base64 that decodes to text (≥20 chars), a hex id (≥32 chars), or Shannon entropy ≥4.4 at ≥24 chars. A shorter payload segment — `https://evil.test/aGVsbG8gd29ybGQ` — cannot be told from a name, because entropy is bounded by `log2(length)` at short lengths. Mitigation in depth, not in this detector: a literal credential in a URL is still caught by the LLM02 egress patterns in the same `scan_output` pass, whatever severity the carrier gets. - **Percent-escapes count as data-carrying — a `%20` in a path is a false positive.** An ordinary link with an encoded space grades as carrying and reaches QUARANTINE_REVIEW / FAIL_SECURE on an untrusted upload. Obfuscated encoding is a core exfil primitive and the ambiguous case is put on the review side deliberately; it is listed here because it is the same *class* of over-block that 0.3.1 fixed, in a rarer shape. **It is a non-ASCII-language tax, and that is the finding.** Three consumer corpora measured it (2026-07-25/26). The two English ones found zero — 0 of 347 external URLs in a 527-document vendor-docs corpus, 0 of 81 in a capture store. The third, a 389-file Norwegian/Microsoft reference corpus, found 10 distinct real escapes and **every one of them Norwegian**: `%C3%B8` and `%C3%A5` are simply *ø* and *å* in UTF-8, and legal/government sources turn titles into paths (`lovdata.no/…/kap2/%C2%A710`). "Accepted false positive" reads differently as "URLs in your own language grade above LOW". **Two distinct axes, which an earlier revision of this file conflated.** For *third-party link* corpora — URLs an ingester collects from other people's sites — language does predict: the corpus of Norwegian legal/government sources carried escapes where two link corpora of mostly English-language domains carried none. For *generated paths* the predictor is not language but **slugger class**: a whitelist slugger (`[^a-z0-9]+ → "-"`) cannot emit an escape in any language, because it discards the character before anything encodes it — measured structurally, not statistically, by a consumer whose content is Norwegian and whose slugger output is `"Løkkene i produksjonslinja" → "l-kkene-i-produksjonslinja"`. A path built with `encodeURIComponent` produces escapes systematically the moment titles are non-ASCII. So non-ASCII language is a *confounder* for encode-vs-whitelist, and the slugger class is the testable thing at a consumer — a one-line code read, not a corpus census. **Both consumer sluggers we have now read are whitelists**, so their generated-path exposure is structurally zero rather than measured-zero; the second was read on 2026-07-31 and reduces the same Norwegian input to the same output. What this does *not* give us is the other half of the axis: we have never seen an `encodeURIComponent`-class slugger in the field, so "produces escapes systematically" above remains a prediction from the transform, not an observation. One sub-class worth separating: `{tenant}`/`{agent-id}` template placeholders in API code samples encode to `%7B`/`%7D` (26 of that corpus's 36 hits — an artifact of harvesting, not of prose). On the *generated-path* side those same placeholders cannot reach that shape at all: braces and `/` are outside the grammar, so `"{tenant}/{agent-id} mal"` reduces to `tenant-agent-id-mal`. - **A percent-escaped path also defeats tokenization, which feeds the entropy branch.** `%` is not in the separator class, so `NSMs%20Grunnprinsipper%20for%20IKT` is one 34-character token where the same title with literal spaces would be four short ones. Measured at H=4.04 — under the 4.4 floor, and moot in practice because the `%` rule already disqualifies the URL. It is recorded because it means the length floor does less work in non-ASCII paths than the calibration assumed. **A field re-measurement (2026-07-31) sharpens this by roughly 3x.** The highest-entropy *legitimate* token in a consumer's 2400-URL corpus is not the one above but a 78-character percent-escaped lovdata title, `Fra%20%C3%A5ndsverk%20…`, at H=4.301 — leaving **0.099 of headroom to the 4.4 floor, not 0.36**. Re-run here through the real `_URL_TOKEN_RE` and `shannon_entropy` rather than a reconstruction: our tokenizer does emit it as one 78-character token. Still moot per URL (`is_ordinary_url` is False on the `%` rule) and still not a reason to move the threshold — but the margin this bullet reports is much thinner than 4.04 implies. Note the corpus count: 2400 is a rebuilt harvester's, where the 2401 in the next bullet is the earlier run's. Same knowledge base, one URL apart. An earlier version of this note offered a second moving variable — a corpus that "grew from 389 files to 394" — and that explanation is **withdrawn**: the consumer corrected it the same day, and the counts reproduce here against their tree. 394 is every `.md` under `skills/`; 389 is the `references/**` path the measurement actually scoped to; the 5 `SKILL.md` files between them contributed no unique URLs. That is one snapshot counted two ways, not two snapshots. So only the script moved, and the one-URL gap stays *unexplained* — the original is gone from a scratchpad and nobody has chased it. The counts remain non-interchangeable: never summed, never quoted as one figure. - **Legitimate CDN content-asset ids trip the hex branch, permanently.** A ≥32-char hex path segment is opaque by design, and several public CMSes mint asset URLs that way (measured: 9 distinct on `regjeringen.no`, `ks.no`, `datatilsynet.no`). This is the "legitimate build hash or doc id" case the calibration predicted, now confirmed present in the field. The class does not decay — it is how those systems generate URLs — so it is a standing false positive rather than a transient one. The branch is otherwise precise (no other false positives in 2401 distinct URLs) and stays. - **A non-empty query is graded as data-carrying — the over-block that actually occurs in the field.** Three corpora have now measured it, and each found a *disjoint* benign population: **(1)** 16 of 16 query-carrying external URLs in a vendor-docs corpus were publisher-authored campaign tracking (`utm_*` on the publisher's own domains); **(2)** 28 of 28 in a capture store were content identity (`?v=`, `?channel_id=`, `?all=true`) where the parameter *is* the resource; **(3)** 149 of 1694 distinct `learn.microsoft.com` URLs in a reference corpus carried `?view=`, Microsoft Learn's own documentation-version selector, plus 23 `?api-version=`. **No parameter-level remedy covers any two of them, let alone all three:** an allowlist keyed on tracking-parameter names resolves (1) entirely and (2) and (3) not at all; stripping the query is lossless for (1), dereferences nothing for (2), and silently changes *which document is cited* for (3) — the worst failure mode of the three, because the result stays plausible. This is a settled constraint on any future middle tier, not a hypothesis. The cost is bounded — a query-carrying *link* is MEDIUM, so it disposes QUARANTINE_REVIEW under `PRESET_USER_UPLOAD` and WARN under `PRESET_TRUSTED_SOURCE`: held or warned, never hard-failed (pinned in `tests/test_wiring.py`, because both consumers inferred a hard block rather than running it). An *image* keeps HIGH, and that is the carrier where this would bite — the two corpora that reported a carrier breakdown contained zero remote images and the third did not report one, so the image row of this limitation remains unmeasured in the field. - **What `active:raw-html` still costs benign documentation is mostly ordinary HTML, not over-reach — and that residual is not a narrowing away.** Two over-reach classes were closed in 0.6.0: the URL-attribute branch now requires an *external* target (an element outside the active name set carrying `href="/en/agent-sdk/quickstart"` reaches no attacker-controlled host), and `` left the active *name* set (APIM's attribute-less `` means "run the inherited policy"; HTML's `` has its whole affordance in an `href` the attribute branch still catches). `` appears in **25 of reference-corpus's 389** documents — count it with `grep -rlE ''`, because a loose `grep '` placeholder, which is not a tag this detector ever fired on. Measured before and after in one session against one corpus state — `docs/rawhtml-census.py`, whose `PRODUCTION` row re-measures the shipped predicate rather than a hypothesis: reference-corpus 133 → **108** of 389, vendor-harvest 100 → **98** of 187, generated-notes 90 → **88** of 550. Against a ceiling of 26 / 38 / 41 (raw-HTML detection switched off entirely) that is **96% of the achievable reduction in reference-corpus and 5% in the two wiki corpora** — the over-reach was nearly the whole raw-html cost in APIM policy XML and nearly none of it in vendor documentation. **The classes had to be measured together.** Alone, the URL-attribute branch frees 3 documents in reference-corpus and `` frees 13; together they free 25. A document carrying one usually carries the other, so closing either alone leaves it blocked by its twin — one-at-a-time measurement understates both, and reading "frees 3" as "this over-reach is cheap" is how this document got the attribution wrong for three releases. **What remains is real HTML in vendor documentation**, flagged by the *name* branch, correctly: per wiki corpus `` 298 occurrences, `` 94, `` 63 — identical in both because 184 of their documents share ancestors. `` is the one arguable member: names are lower-cased and `frame` is in the active set for legacy HTML framesets, which appear in essentially no modern documentation, while `Frame` is a common MDX component name. Case is not an available discriminator: HTML is case-insensitive, so PascalCase cannot be treated as "component, not tag". Recovering the rest is **not** a further narrowing — it needs a carrier split and a new label (`active:raw-html-link`), because raw HTML has no ordinary form and the class collapses to one finding per document. **The external-target test now gates one more surface, and it is a literal one.** `_has_external_target` matches `^(?:[A-Za-z][A-Za-z0-9+.\-]*:|//)`, so a backslash-separated authority — `href="\\evil.example/x"` — reads as relative and now grades down, although WHATWG URL parsing normalizes backslashes to slashes for special schemes and a browser would resolve it externally. This is inherited, not introduced: the markdown paths have applied the same predicate since 0.3.1. It is recorded here rather than fixed because the fix belongs to the predicate, not to the raw-HTML branch that newly depends on it. **The scanner and the mutator no longer share a predicate.** Until 0.6.0 `neutralize` imported `is_active_tag` by name, so any narrowing moved the opt-in mutator too, and no test discriminated the two halves. They are now `is_active_tag` and `is_defangable_tag`; the mutator kept the broader behaviour deliberately, pinned by `tests/test_neutralize.py::test_mutator_still_defangs_what_the_scanner_now_lets_pass`. - **The carrier split TIGHTENS the trusted tier when both carrier classes are present.** 0.7.0 is sold as a loosening of the upload door, and on that door it is one. But splitting one class into two means a document carrying both an `` and an `` now emits *two* findings at MEDIUM+ where it emitted one, which trips the compound overlay (`>=2 findings at MEDIUM+ -> escalated one tier`). Such a document was WARN through 0.6.1 and is `quarantine_review` from 0.7.0 under `PRESET_TRUSTED_SOURCE`. On the trusted preset nothing was hard-failed to begin with, so this is the *only* direction the split can move it. Pinned by `tests/test_wiring.py::test_split_tightens_the_trusted_tier_when_both_carriers_are_present`, and `docs/rawhtml-census.py` now reports a `TIGHTENS` column against the previously-shipped row on both trust tiers — "frees N" without "tightens M" is a one-sided number. **Measured on all three populations, as shipped: 0 documents tightened, on both trust tiers — reference-corpus (389), vendor-harvest (187), generated-notes (552).** **That zero is empirical, not structural, and the census shows exactly how thin it is.** The split measured *alone* tightens **13** documents on the trusted tier in vendor-harvest and **14** in generated-notes. Adding the no-URL narrowing takes each of them back to 0: in these populations the document's second, HIGH-class carrier was itself a tag naming no target, which the narrowing makes inert, so the compound overlay never sees two findings. That is the census reporting a cancellation, not this repo proving one — a population whose second carrier is a real `` would still escalate, which is precisely the case `test_split_tightens_the_trusted_tier_when_both_carriers_are_present` constructs and pins. Read the zero as "not observed in any of the three populations, each counted against its own denominator", never as "cannot happen". - **The split also LOOSENS the upload door for a lone anchor — the direction it was built for, and the one with a residual worth naming.** Measured as shipped: `t` on its own emits one `active:raw-html-link` at MEDIUM and disposes `quarantine_review` under `PRESET_USER_UPLOAD` (`warn` under `PRESET_TRUSTED_SOURCE`); through 0.6.1 the same document graded HIGH and `fail_secure`d. An `` to the same host is untouched — `active:raw-html`, HIGH, `fail_secure`. So an anchor pointing at an attacker-controlled host, arriving on an untrusted upload, is now a human decision rather than a halt. The trade is deliberate and it removes an asymmetry that came from syntax rather than affordance: following an anchor needs a click, exactly like the markdown inline link that has graded MEDIUM since 0.3.1, so the same URL no longer grades two different ways depending on which syntax carries it. It is recorded here so 0.7.0's "frees N documents" is not read as free — what was freed is the click-required class, and MEDIUM is a real grade drop on the door where every finding is trust-escalated. - **Raw-HTML findings no longer count end tags, and that moved a published field.** Through 0.6.1 `` was active by name on its own, so `count` ran roughly 1.6× the opening-tag total (measured on one corpus) and a start/end pair counted 2. 0.7.0's no-URL narrowing makes an end tag inert — it names no target — so `count` is now the opening-tag total. A consumer reading `count` will see it *drop* for every document carrying ``, on a field whose meaning did not change. The finding count is unaffected: the class still collapses to one finding per class per document, and `count` was never a document count. - **Which tags the no-URL narrowing may render inert is a judgement about affordance, and no test in this repo can derive it.** `_URL_AFFORDANCE_TAGS` holds the nine names whose entire active affordance *is* the URL they name — `a`, `area`, `img`, `video`, `audio`, `source`, `track`, `frame`, `frameset` — so carrying no URL attribute they name no target and grade inert. Every other name in the active set stays active with no attributes at all, because it does something a URL cannot describe: ``, so a document *about* XSS is flagged alongside a document that *carries* it. This is a deliberate trade made twice over: requiring the closing tag was a fail-open (an unclosed `