1
0
Fork 0
Commit graph

157 commits

Author SHA1 Message Date
eeb7bf07f6
feat(active): report a run of images whose order carries a value
Session 6 of the 2026 threat plan (attack row 3): A05 output was red.
CamoLeak renders a secret as one image per character from a URL alphabet;
each URL is ordinary, so each image graded LOW and the gate WARNed.

- active:markdown-image-sequence (MEDIUM): at least 4 ordinary external
  images, inline or reference-style, on one URL template whose varying
  tokens are 2 chars or fewer, or digit runs. Digits that count up in
  document order are a counter and carry nothing. The per-image grade is
  unchanged; the finding sits beside it.
- Accepted twins (flag tables, figures 1a/1b) go to review; measured
  evasions (word dictionary, 3-char symbols, counter increments, run of 3,
  a template per symbol) are in LIMITATIONS.

Guard rule before/after: FP 106/198, 196/2918, 108/389 identical. Those
corpora hold no external markdown image, so the rule was also run on 577
package READMEs (312 with external images): 0 firings at run lengths 3-8.
OKF import 47/31 identical. Matrix 140/140, ReDoS 163 patterns 0 flagged,
suite 1179 passed + 5 xfailed. Row 3: 13 caught, 1 gap, 5 open.
2026-10-31 19:33:51 +01:00
d00d1b0658
chore(release): 1.6.0 v1.6.0 2026-10-31 17:04:02 +01:00
0ebe8abdbd
feat(output): report hidden markup — tiny font, invisible colour, comments outside code
Session 5 of the 2026 threat plan (attack row 3): A04 and A15 were red.

- html-obfuscation:tiny-font (HIGH): font below 4px. ShadowLeak's 1px passed
  because the commons zero-font row only matches a leading 0.
- html-obfuscation:invisible-colour (HIGH): text within 8 per channel of the
  background in the same style attribute, or transparent. White alone is not
  a signal (email buttons); measured residual in LIMITATIONS.
- output:html-comment-present (MEDIUM): a comment in the artifact outside
  fenced code and inline code. 253/254 comments in the reference corpus sit
  in code.

Local to output.py; the vendored lexicon stays byte-equal with commons.
Guard rule before/after: FP 106/198, 196/2918, 108/389 identical; OKF 6/53
and 47/31 identical. Matrix 139/139, ReDoS 161 patterns 0 flagged, suite
1142 passed + 6 xfailed. Row 3: 12 caught, 1 gap, 6 open.
2026-10-31 17:01:39 +01:00
8f08b1906e
docs(catalogs): wire ASI06 and ATLAS v2026.10 into the README, date every source
Row 1 of the 2026 done-criterion: 16 of 16 register entries on the newest
edition, each with a check date and a source link, re-read at the publisher
on 2026-10-31.

- README: new subsection mapping OWASP ASI06 and seven MITRE ATLAS v2026.10
  techniques to what the guard stops at the write and what it does not;
  a dated catalog table; the threat-model anchors are linked.
- ATLAS moved to v2026.10 (2026-10-28) after session 1; AML.T0057 no longer
  exists there and is not linked.
- Register 1.1.0: `cited` beside `latest` on every entry; CleanBase added
  (cited in BRIEF, unregistered); unverifiable sources kept as candidates.
- tests/test_catalog_register.py fails on a missing date or link, an older
  cited edition, or a README ATLAS/ASI id the register has not verified.
- ZWJ measurement re-taken on Unicode 18.0 (1614 sequences, 122/122); the
  "458 unassigned" figure came from Python's Unicode 16.0, 406 in 18.0.
2026-10-31 16:29:35 +01:00
3eb8e9e9a9
chore(release): 1.5.0 v1.5.0 2026-10-31 14:28:48 +01:00
1c6816cdfe
fix(owasp): qualify every catalog number with the 2026 edition
OWASP's Top 10 for LLM Applications 2026 renumbered the list; only LLM01
and LLM02 kept their meaning. Every finding's `owasp` was written against
2025 with no year, so EchoLeak findings said LLM05, which in 2026 means
Data and Model Poisoning. Row 2 of the 2026 done-criterion counted 95
such sites at 6702d7a (82 old meaning, 13 wrong); it is now 0.

- Producers name the risk, never the number: one table (`_owasp.py`)
  owns `LLMnn:2026`. The vendored lexicon stays byte-equal with commons
  and gets its year where the finding is made; a commons number whose
  meaning moved between editions raises instead of being guessed.
- Mechanical map from the 2025 meaning (verified against the publisher
  in session 1): LLM05 -> LLM10:2026, LLM06 -> LLM03:2026,
  LLM10 -> LLM06:2026, LLM08 -> LLM09:2026, LLM04 -> LLM05:2026.

The 13 wrong-meaning sites, judged:
- Factual poisoning (PoisonedRAG gap, and the test that follows it):
  LLM05:2026. The 2026 text names "RAG Knowledge Base Poisoning" as its
  example 6. It said LLM09, which was Misinformation.
- Fail-secure (README row, matrix row): no number. It is the guard's own
  failure semantics; no edition describes it. Was LLM10.
- Parser-surface refusals (nested mapping, block scalar, quoted flow
  element, off-allowlist key): no number. Refusing a shape the parser
  would have to interpret is not resource consumption. Was LLM10. The
  anchor and merge-key rows stay LLM06:2026 (alias expansion is DoS).
- `resource` URL and cross-link scheme validation (4): no number. They
  validate received bundle metadata; LLM10:2026 is about model output
  and says input validation belongs to LLM01. Was LLM05.
- `_raise_case` default (OKF path traversal, absolute path, reserved
  name, non-.md, bundle escape): no number. None is prompt injection.
  Was LLM01.
A doubtful number would be the misreading the year exists to prevent,
so `Case.owasp` is now Optional and the matrix prints "-".

Self-safety (input caps, ReDoS) keeps LLM06:2026: the 2026 text, read at
the publisher, covers "input flood ... exploiting processing
inefficiencies" and application-side rechunking. The README's LLM09
anchor (Misinformation) is dropped; it was the factual-poisoning row.

The permanent test (tests/test_catalog_numbers.py) fails on an old or
wrong number, on a number without the year outside the vendored
lexicon, and on any finding the coverage matrix drives whose `owasp`
lacks the newest year; "newest" is read from the catalog register. The
census holds its wrong-rules to their counts at the frozen revision
only, so the frozen listing (da1aa0fa...6f) is unchanged.
2026-10-31 14:20:35 +01:00
1241075ef8
chore(release): 1.4.2 v1.4.2 2026-10-29 08:31:59 +01:00
c72e648a94
fix(sanitize): catch every invisible codepoint, exempt only on context
Variation-selector smuggling (A01) and Sneaky Bits (A02) gave 0 findings
on 1.4.1 and stayed in the sanitized text. The denominator is now the
frozen Unicode 18.0 set (DICP | Cf | Co, 141 674 codepoints); each one is
stripped and flagged unless a known legitimate use exempts it on context:
one VS15/16 after an emoji base, one ideographic selector after a CJK
ideograph, ZWNJ/ZWJ beside Arabic-script or Indic letters, LRM/RLM/ALM
in a document with RTL text, an Arabic number sign before a digit. A run
is always a carrier; private use is caught in full (operator decisions
2026-10-28). One predicate serves sanitize, the output gate and the
lexicon's word-split row, whose regex stays byte-identical with commons.

A01 and A02 are green; the FP populations and the pinned OKF corpus are
identical before and after.
2026-10-29 08:31:43 +01:00
08ca8f61bc
docs(plan): fix the attack batches at three sessions, 5-7
The operator approved the proposed split on 2026-10-28: hidden markup,
sequence-borne exfiltration, and instructions in plain prose.
2026-10-28 21:30:06 +01:00
69e80f26a1
docs(plan): record session 1 measurements and the scope decisions
Rows 1-3 of the done-criterion now carry frozen denominators: 15
catalogs (0 dated), 95 number sites, 19 attacks (8 caught, 1 documented
gap, 10 red). The 2025 -> 2026 mapping is re-verified against OWASP,
and the attack batches for sessions 5-7 are proposed.
2026-10-28 21:27:33 +01:00
e4e18b38d7
test(threat-2026): one case per known 2025-2026 attack, red where not caught
Nineteen attacks that can reach an ingestion pipeline, verified against
primary sources, with four out of scope and their reasons. Each arrival
runs through the canonical wiring; twelve of 31 arrivals are red today
and marked strict xfail, so a fix fails until its marker is removed.
PoisonedRAG asserts that its documented gap still stands.
2026-10-28 21:27:33 +01:00
e702505312
docs(threat-2026): register the threat catalogs at their newest editions
Fifteen catalogs and sources, each with the edition cited at v1.4.1, the
newest edition read from its publisher, the check date and the link.
Behind: OWASP LLM Top 10 (2025 -> 2026) and Unicode (17.0 -> 18.0).

The invisible-character denominator for the next session is Unicode
18.0's own: Default_Ignorable_Code_Point, format (Cf) and private use
(Co), 141 674 code points, built from sha256-pinned UCD files.
2026-10-28 21:27:32 +01:00
2e3d33efb4
test(threat-2026): freeze the catalog-number census at v1.4.1
docs/catalog-census.py classifies every OWASP LLM number in tracked files:
old (2025 meaning, maps mechanically), wrong (a meaning no edition gave it,
each with its reason), same, current, history, migration-plan, self.
At 6702d7a the denominator is 95 sites in 23 files (82 old, 13 wrong);
the listing is frozen with its sha256 and a test reproduces it from git.
2026-10-28 21:27:31 +01:00
61475686bf
docs(plan): keep the repository private until the upgrade, then move it to open/ 2026-10-28 21:06:14 +01:00
f19f434b54
docs(plan): plan the move to the 2026 threat catalogs
The approved direction heads the plan: every cited catalog on its newest
edition with a check date and source link, no catalog ID carrying an older
edition's meaning, and the known 2025-2026 attacks that can reach an
ingestion pipeline as test cases, each caught or documented as a gap.

Measured on v1.4.1 before anything is built: LLM03-LLM10 carry their 2025
meaning in 101 places across 26 tracked files, and 17 of 19 invisible or
deceptive code-point types (variation selectors among them) pass sanitize
with no finding.
2026-10-28 20:55:41 +01:00
6702d7a569
docs(readme): the package index is the range-pinning channel from 1.4.1
1.4.1 is published to the Forgejo package index (operator decision
2026-09-23), so README's install section stops promising the index as a
future event. It now gives the index install line with a range, states
which versions the index carries (1.1.0 and 1.4.1; 1.2.0-1.4.0 were never
published there), and keeps the tag pin as the first option.

Measured anonymously after the upload: the simple index lists 2 wheels,
1.1.0 and 1.4.1, and `pip install "llm-ingestion-guard>=1.4.1,<2"` from
it resolves 1.4.1 in a clean venv.
2026-09-23 17:06:10 +02:00
57c91aeb11
chore(release): 1.4.1 v1.4.1
Patch release for 639da03: a flow-sequence element admits '#' and ':'
where YAML reads them as text, so okf's own `references: [...]` lines
with #fragments and scheme:// links parse (0 / 5 467 raise on the
consumer's bundle, was 2 038). Also ships 79285e1, which brought the doc
surfaces up to 1.3.0/1.4.0 behaviour.

Unlike the 1.4.0 release commit, this one bumps every live version
surface: pyproject.toml, __init__.py, README badge + status + install
pin, ADOPTION-BRIEF status + "as of" + test count (893 -> 910), BRIEF
status, CHANGELOG heading. Provenance references to 1.4.0 are left as
they are.

No exported surface changed. Gates on this tree: 910 passed, coverage
exit 0 (130/130 + 6/6), redos-sweep exit 0, 45 LIMITATIONS entries.
2026-09-23 12:46:32 +02:00
639da03f8c
fix(okf): a flow sequence admits '#' and ':' where YAML reads them as text
1.4.0 refused a flow-sequence scalar element carrying '#' or ':' anywhere.
okf writes `references: [...]` as a flow sequence of doc links, and an
entry may carry a #fragment or a scheme://. claude-code-llm-wiki measured
parse_frontmatter raising on 2 038 of 5 467 concepts of a bundle okf
produced (2026-09-17); reproduced here on the same bundle, 2 038 / 5 467
before and 0 / 5 467 after (0 / 4 466 and 0 / 5 530 on the two other
builds).

The two characters are now refused by position, where PyYAML 6.0.3 gives
them a meaning: '#' opening the element or following whitespace (a
comment), ':' opening or ending the element or preceding whitespace (a
mapping). Elsewhere they are scalar characters. The rest of the refused
set, the indicator rule and the no-mixing rule are unchanged.

Chosen over admitting the characters wholesale because `[a: b]` is a
mapping and `[a #b]` a comment to YAML; admitting those would hand the
consumer a value YAML does not read. The position checks are substring
tests, no regex: CPU time stays linear in element count and length
(100k -> 800k), about 10 % over 1.4.0, inside the existing 2 s bound.

Pinned upstream corpus unmoved at 6 / 53. 910 passed (+17), coverage
exit 0, redos-sweep exit 0, 45 LIMITATIONS entries.
2026-09-23 12:44:09 +02:00
79285e1265
docs: finish the 1.4.0 release on the doc surfaces
The 1.4.0 release commit (d19de8c) dated the CHANGELOG heading and nothing
else, so the tag went out with README, ADOPTION-BRIEF and BRIEF still
naming 1.3.0 and the install pin still installing 1.3.0. This bumps the
badge, status line and install pin in README, the status and "as of"
lines in ADOPTION-BRIEF (test count 868 -> 893) and BRIEF's status line.
The tag itself is not moved.

Three claims that went false earlier are corrected against the code:

- README said `resource` is deliberately not on the frontmatter allowlist
  and that a `sources` block list of mappings is refused. Both stopped
  being true in 1.3.0. The paragraph now names the four mapping carriers
  and says `resource` / `usage_window` are admitted inside a `sources`
  entry only.
- README described the `resource` https-allowlist without its scope. It
  covers the top-level `resource` only; `sources[].resource` is never
  URL-validated (measured: `javascript:` there imports with no error).
- LIMITATIONS still listed `tags: [a, b]` among the routes that fail.
  It parses since 1.4.0.

Gates after the edit: 893 passed, coverage exit 0, redos-sweep exit 0,
45 LIMITATIONS entries.
2026-09-23 12:25:33 +02:00
d19de8cb38 chore(release): 1.4.0 v1.4.0 2026-09-08 06:10:53 +02:00
3e324a1f86 feat(okf): a flow sequence of plain scalars parses, and the corpus number is 6/53
`tags: [a, b, c]` is the form SPEC.md 4.1's own frontmatter skeleton writes out,
and 9/53 upstream reference concepts use it. It raised on the `[` indicator.
It parses now, to the same list its block-sequence sibling already produced.

The predicate is character-level, inside `_parse_flow_sequence`: an element is a
plain scalar only if it is non-empty and carries none of `{ } [ ] : , " ' #`,
and it then passes the unchanged scalar-indicator rule. Everything that would
need YAML semantics to split or unquote still raises - a quoted element (quotes
are retained here, never stripped), a colon or comma inside an element, a
sequence inside a sequence, an empty element, an anchor, an alias. A sequence
may not mix scalars and mappings, the rule the block list already carries, and
the mixing verdict is reached before the element is parsed so the caller is told
about the mix rather than about a key the allowlist would have named instead.
The 1.3.0 `sources` flow-mapping carrier is unchanged and pinned against
regression. Depth 1 is not spent: the elements are leaves.

Measured with the denominator, against the pinned corpus (`_okf-upstream` @
3fcbb9f, 53 non-reserved documents) and the pinned SPEC (`_okf-canonical` @
ad30107). Baseline reproduced first, with a known-positive control, at 0/53.
After: 6/53, all six in acme_retail. It does not close the corpus - 44/53 still
stop on `generated` written as a top-level block mapping, which spends the
no-nesting-past-depth-1 rule and is a security decision, out of scope here.

P1 alone, per the operator decision of 08.09. The two neighbouring predicates
were measured and deliberately not built: a flush-left block sequence and a
folded plain scalar release 0/53 each on their own, and stacked on this one they
still measure 6/53. `_consume_block_list`, the `description` continuation and
the allowlist are untouched.

docs/LIMITATIONS.md's tags/description entry is rewritten against the
measurement: three of its claims were wrong. The parser does have a
sequence-value type (since 1.3.0 - what it lacks is the indentation the corpus
omits); the figure is 6/53, not the 1.2.0-era 4/53; and tags/description are
not the residual that blocks the corpus. README gains the sequence carrier in
the paragraph that already describes the mapping one.

Self-safety: the predicate compiles no regex, so docs/redos-sweep.py cannot see
it. Measured instead on the CPU clock - linear in element length (exponent
0.86-0.99) and in element count (0.97-1.05) over four doublings to 800_000 -
and pinned by two bounds in tests/test_okf.py.

Six rows that pinned the old refusal are re-aimed at the class that still
holds - the quoted element - the way the 1.3.0 rows were when the carrier
opened. One of them lives in src/llm_ingestion_guard/coverage.py, which is why
the src diff is three files rather than one.

Version 1.4.0 in the code only. The CHANGELOG entry stays under Unreleased and
no tag is cut: README's badge and install pin must keep naming a tag that
exists.

Gates after `git add`: 893 passed (was 868), coverage 130/130 + 6/6 gaps,
redos-sweep exit 0, LIMITATIONS still 45 entries.
2026-09-08 05:34:55 +02:00
6e7c8d2b98 docs(okf): measure the tags/description gap against the pinned SPEC and corpus
Order 20260906T213322Z: measure and propose, no src, no bump. Measured against
SPEC _okf-canonical @ ad30107 and the corpus _okf-upstream @ 3fcbb9f (denominator
53, extracted at the pin because the work tree had moved on to 9a15b13).

Three claims in the tags/description entry are stale against the 1.3.0 parser.
The parser does have a sequence type -- _consume_block_list parses block lists of
scalars -- what it lacks is tolerance for the flush-left indentation 36/53 of the
corpus uses. Removing tags alone now lets 6/53 pass, not the 4/53 recorded, and
the entry's headline claim does not hold at all: description alone unblocks 0/53,
and with all three surface forms closed 44/53 still stop on generated as a
top-level block mapping. The entry oversells its own reach by a factor of seven.

The SPEC has no depth rule. no-nesting-past-depth-1 is entirely ours; the
conformance floor is only "a parseable YAML frontmatter block" (SS11.1).

Candidates were measured by normalizing the surface form onto a shape the parser
already accepts, then importing parse_frontmatter -- the predicate is never
re-implemented, only the input's spelling is rewritten. P1 (scalar flow sequence)
takes the corpus from 0/53 to 6/53 without spending depth-1; P2 and P3 buy 0/53
each and 6/53 stacked on P1. Recommendation is P1 alone, and the note says out
loud that this does not open the corpus.

The label "punkt 44" is not grep-able: it came from a count-after-insertion, and
the entry is today the 12th of 45 at docs/LIMITATIONS.md:126.

No src change, no version bump, no push. 868 passed, coverage 130/130 + 6/6 gaps,
redos sweep exit 0 -- all after git add.
2026-09-06 23:58:02 +02:00
44e2b31afd test(okf): pin the two boundary classes the new carriers introduced v1.3.0
Probed after 1.3.0 landed, on advisor challenge. Both came back clean --
these pin them so they cannot regress silently.

1. A `- ` item whose text begins with a YAML indicator AND carries a `": "`
   could in principle route into the block-mapping path with a
   half-validated key instead of into `_reject_dangerous_value`. It does
   not: `&anchor id` fails `_KEY_RE` on the key side, so the item falls
   through intact and raises on the indicator. One row each for & * ! %
   backtick, plus the merge key inside an entry.

2. `_consume_block_mapping` ends at a blank line and at column zero, and
   hands an index back to `_consume_block_list`, which hands one back to
   `_parse_flat`. A line orphaned by that boundary -- a `resource:` left
   over from a mapping that closed early -- must RAISE, not be dropped. A
   pointer that vanishes rather than failing is this repo's failure class,
   so it gets a test rather than a probe. Plus the return-index contract:
   a top-level key after a multi-entry list is neither swallowed nor
   re-read as an item.

868 green (was 859), 130/130 classes. Test-only; no source change.
2026-09-02 17:26:08 +02:00
a965e8ac5b feat(okf): sources becomes expressible, and the parent key is what admits resource
Order 20260902T150716Z from .claude -- a K5 blocker in the OKF programme.
`parse_frontmatter` rejected `sources` in every form the spec and its
producers actually use. Measured 02.09 by two consumers independently:
`sources: [{ id: a, resource: x }]` raised on the `[` indicator (one entry
as well as two), and the block sequence of block mappings -- SPEC.md 5.1's
OWN example -- raised "nested mappings are not supported". `resource` is
REQUIRED within a `sources` entry, so the whole provenance family was
unwritable and a bundle written the way the spec documents it was refused.

Measured against the spec before coding, not reasoned: 5.1's example block
is the canonical carrier for a REQUIRED field and 11.1 defines conformance
as parseable frontmatter, so refusing it refuses a conformant bundle. Both
carriers now parse to the same list of dicts.

The load-bearing change is not the carrier, it is WHO admits `resource`.
1.2.0 left it off the allowlist arguing the parser could not tell
`sources[].resource` (5.1, a citation) from `executor.resource` /
`attester.resource` (10, a pointer to code to be run -- the door-C route
closed in 1.1.0). That premise was false: the owning key is in scope at
every call site and was simply never threaded through. It is threaded now,
so the discrimination is structural, and door C stays shut through EVERY
carrier including the two this adds -- pinned by a new test that drives
`executor`/`attester` through all four.

Refusal stays the default elsewhere. A flow sequence of plain scalars
(`tags: [a, b]`) still raises: the sequence carrier is opened for the flow
mapping element and nothing else. A `sources` entry admits scalar leaves
only, so 5.1's optional PER-ENTRY `usage_window` is refused -- no nesting
past depth 1 is a security property and it was not spent here; registered
as a conformance gap rather than left as an oversight. A block list may not
mix scalars and mappings, because a consumer reading `entry.get("id")` over
one gets an AttributeError off the first str.

New residual registered: `sources[].resource` is scanned as text (T1) but
never URL-validated. T3's https allowlist cannot reach it without
over-blocking conformant bundles -- 5.1 permits bundle-relative paths and
scope descriptors, and the producers' own golden emits `resource: fixture`.
A consumer that dereferences it must call `validate_resource_url` itself.

Suite 834 -> 859 green. 25 new rows; four pre-existing rows changed because
this release changed the behaviour they pinned, two of them renamed since
their names asserted the old invariant (`exactly_one_route_to_a_mapping`,
`two_keys_per_item_is_where_the_block_list_hard_rejects`). Not "unchanged".
130/130 classes, 6/6 gaps hold, 44 -> 45 limitations, ReDoS 0/152 (the
sweep adds no evidence here -- this change adds no regex and the splitting
is linear). Six version surfaces bumped by hand, no sed. Re-measured alone
after the bump.

No exported surface changed; no detector behaviour and no calibration
changed.
2026-09-02 17:21:26 +02:00
0184df9ed9 docs(limits): tags and description block the whole OKF corpus before sources is even read
Measured 08-23 (order 20260823T161935Z): the line-flat frontmatter parser has
no sequence-value type at all, so tags rejects 53/53 upstream concepts
regardless of flow or block form, and description's folded-scalar continuation
misreads as a nested mapping in 29/53. Independent of both the mapping-form
gap and the sources block-form gap already documented here -- closing either
moves nothing on this corpus. 43 -> 44 items; README count moved with it. No
code change, no release.
2026-08-25 08:27:22 +02:00
9aeceb02c0 release: 1.2.0 -- the OKF mapping-form fix becomes pinnable v1.2.0
Order 20260823T175853Z from .claude: commit 5870483 (the G3 nine-key
allowlist fix) was untagged -- llm-ingestion-okf measured 4/9 concepts
through Doer C on it, blocked from 8/9 only by the absence of a
pinnable tag, on a suite already green (595/596) at that commit.

Six live version surfaces bumped by hand (no sed -- provenance is
never bumped): pyproject.toml, __init__.__version__, README badge +
Status + the pinned pip install tag, docs/BRIEF.md, docs/ADOPTION-BRIEF.md
Status, and CLAUDE.md. ADOPTION-BRIEF's snapshot line moves with it:
129/129 -> 130/130 classes, 802 -> 834 passing (the new row is the
off-allowlist-key coverage case added by the G3 fix itself).

NOT bumped, matching the 1.1.0 precedent: every "closed in 1.1.0" /
"the 1.1.0 defect" reference across CHANGELOG.md, CLAUDE.md,
docs/LIMITATIONS.md, docs/redos-sweep.py, src/llm_ingestion_guard/okf.py
and tests/ names a historical baseline, not the current version --
rewriting those would erase what they document. docs/BRIEF.md's
GuardLLM v1.1.0 reference is a third party's version, untouched.

Re-measured after the bump, alone: 834 passed, 130/130 classes, 6/6
gaps hold, 43 limitations, ReDoS sweep 0/152 candidates flagged.

No exported surface changed; no detector behaviour and no calibration
changed. Pure release action -- no functional code touched.
2026-08-23 20:13:50 +02:00
58704834b6 feat(okf): the mapping class gets one expressible form, typed and allowlisted
OKF v0.2 writes its whole trust and provenance layer as mappings, and T2 gave
the mapping class no expressible form. A consumer measured 0 of 53 upstream
concepts through the gate on 0.3.4, 1.0.0 and 1.1.0. That was a contract
collision, not a calibration setting: SPEC.md @ 62432a09 uses flow mappings in
its own 5.1/5.2 examples, and 11 carries a hard MUST for consumers ("MUST treat
a bare `verified` mapping as a one-element list") that presupposes they parse.

Admitted: a flow mapping, as a value or as a block-list item, whose every key is
on a nine-name allowlist and whose every leaf is a plain scalar run through the
unchanged dangerous-value and mapping-construct predicates. The form is safe
because the allowlist inspects every key -- the blanket refusal was the
enforcement, not the point.

Refused, each on its own rule and ground-truthed against PyYAML 6.0.3: a key off
the allowlist, a nested collection, a quoted leaf, a duplicate key, an empty or
unclosed mapping, trailing junk, and {a:b} (which PyYAML reads as the KEY a:b).
A refused mapping still raises rather than degrading into a string, so the 1.1.0
type-confusion defect is not reopened, and the block, dotted and inline-colon
routes still raise.

`resource` is deliberately off the allowlist though SPEC.md 5.1 names it inside
a `sources` entry: it is a pointer rather than a label and the only key T3
exists for, so admitting it would let `executor: { resource: skills/run.md }`
carry an executable-code pointer through in typed clothes -- the door-C route
closed in 1.1.0. It costs nothing today, because the conformant carrier for
sources[].resource is a block sequence of block mappings, which stays refused.

Mapping leaves are scanned like every other frontmatter value (T1). Coverage
matrix 130/130 (new row: the off-allowlist key). No exported surface, detector
behaviour or calibration changed.
2026-08-21 21:04:51 +02:00
7cb4553301 docs(llms): add llms.txt per org D2b decision (21.08)
Minimal file: H1 project name, one-line blockquote summary, and the
install command copied verbatim from README's first code block —
llms.txt cites the README's start command, never replaces it.
2026-08-21 11:24:00 +02:00
01f6f382c4 test(docs): pin docs/redos-sweep.py under test.py's contract
It was the only docs/ measurement script without a row here, and this
repo just made it load-bearing for two published LIMITATIONS.md numbers
(the 1.5 ms floor, the 2.6 flag ratio). Pins structure only, per the
sweep's own docstring: the suite clock (not a reimplemented wall clock),
the constants against the doc's prose, and the 152-pattern/11-table
collector count. Never pins a timing outcome -- that would be red
several runs in twenty.
2026-08-20 23:10:25 +02:00
74656123f9 docs(redos): the sweep now measures on the clock its own numbers justify
docs/redos-sweep.py timed on time.monotonic() while every ReDoS bound in the
suite had moved to process CPU time, so the 1.5 ms sensitivity floor and the
"~23 s at the cap" figure in docs/LIMITATIONS.md were produced by a different
instrument than the bounds they support. The script imports scan_seconds now.

Re-derived on that instrument, the floor came back UNCHANGED at 1.5 ms. Twelve
full runs of all 2585 arms: median ratio 1.95-2.03 in every size bucket above
50 us, but two-point excursions past the 2.6 threshold survive at every
magnitude (p99 2.9-3.3 even above 1 ms). Flagged arms per run by floor: 6.9 at
0.5 ms, 1.1 at 1.0 ms, 0.33 at 1.5 ms. Four arms flagged across the twelve runs,
each in exactly one; six arms that have ever flagged re-measure at exponent
0.97-1.09 over six doublings, at most 1.2 s at the cap. Descheduling was never
what made this sweep noisy -- a ratio from two points is.

Measured before publishing and it cost a correction: an earlier draft of this
change said the sweep "reports 0 candidates". The next run reported 2. Nine of
twelve instrumented runs are clean and eight consecutive shipped-script runs
under load flagged 0-3, so neither a clean run nor a flagged one is evidence
on its own -- the doc says that now.

Also corrects the pattern count in the same bullet: 150 -> 152, the script's
own printed total. The 0.3.4 CHANGELOG entry keeps its 150 as a snapshot.

No exported surface, no detector behaviour, no calibration touched.
802 passed, 129/129 classes, 6/6 gaps, 43 limitations, gitleaks clean.
2026-08-18 16:56:01 +02:00
4472f209a4 docs(limits): seven shipped behaviours were only in the code, and one of them was written down wrong
TRACK 2. All seven were tested and released; none had a line in
docs/LIMITATIONS.md. Written where each belongs topically, not appended.
36 -> 43 items; README count moved with it. No code change, no release.

- The carrier split LOOSENS a lone anchor, not only tightens. Measured as
  shipped: <a href="https://ext.example/p"> alone is active:raw-html-link at
  MEDIUM -> quarantine_review on PRESET_USER_UPLOAD (warn on trusted); through
  0.6.1 it was HIGH -> fail_secure. An <img src> to the same host is untouched.
  Only the tightening direction had been documented.
- The no-URL narrowing's name set is a judgement about affordance no test can
  derive. A name wrongly placed in _URL_AFFORDANCE_TAGS goes silently inert,
  and no corpus can catch it: what it produces is an absence.
- hybrid-xss:script-tag and active:raw-html do not match the same strings.
  <script src="a<b"> raises active:raw-html and NO XSS label -- the lexicon's
  [^><] is the 0.3.3 ReDoS fix, HTML_TAG_RE consumes quoted runs atomically.
  Disposition never moves; the label does, so that id is not a script census.
- A green ReDoS row is evidence only if seen red, and three rows never were.
  Corrected while writing: the third row is
  test_gate_is_bounded_on_the_long_attribute_arm, killed by 0.7.0's own
  narrowing (separation 1.0x, 0.028s, no findings) -- not active-url-attr-value,
  which is a payload-design rule, not a dead row.
- Every suite bound runs on process CPU time (tests/redos_clock.py); the
  residual is that a BLOCKING hang would hang the suite. docs/redos-sweep.py
  still times on time.monotonic(), so the published 1.5ms floor at N=8000 and
  the "~23s at the cap" figure come from a different instrument than the bounds
  they justify. Not re-derived; stated as not directly comparable.
- Under DEFAULT_ACTION_MAP the assessment axis adds exactly one judgement over
  the disposition (NONE vs LOW). Consequence for this document: the published
  FP rates are a risk statement only while NONE+LOW are the WARN pre-image.
- The ZWJ context test is one predicate on two surfaces, and the symbol is
  private -- output imports sanitize's _is_joiner_in_emoji_sequence, so a
  consumer importing it pins a name 1.x does not promise.

Verified: 802 passed, 129/129 classes, 6/6 gaps hold, grep -c '^- \*\*' = 43,
and the doc scanned for stray format characters (one U+200D remains, the
pre-existing composed emoji -- an accidental U+200C of my own was removed).
2026-08-17 12:15:00 +02:00
2b03b8d643 docs(security): add contact email, https scheme on Forgejo URL
Scorecard Security-Policy scored 4/10: no email address, bare
git.fromaitochitta.com URL without scheme. Both are worth 6 of 10 points
per Scorecard 5.5.0's model. Repo-specific policy text kept as-is.
2026-08-16 21:15:10 +02:00
246e1fc1d3 docs(readme): add table of contents
ORDRE 32, AAA+ B-axis round 2.
2026-08-16 16:16:57 +02:00
45c2b06315 docs: "warn, clean" is graded, never cleaned bytes -- measured at okf's door
okf measured that 0.7.0's no-URL narrowing dispositions four raw-HTML
carrier forms (<a aria-label>, bare </a>, <Frame>, <video />) to warn
with no active-content finding -- verified here directly against
screen_output(..., PRESET_USER_UPLOAD), which returns the library's own
"clean: no findings" reason string (disposition.py:264). At their door
(llm-ingestion-okf 0.7.0), warn is the persist floor (inbox.py:139/323),
and their adapter forwards the original extracted text because
screen_output never hands back sanitized bytes -- defanging is a
separate, deliberate neutralize() call. So the same four forms land
written into the bundle, carrier present verbatim. Named per operator
decision; CHANGELOG.md's "35 limitations" and GATE-G-v1.md's counts are
dated snapshots of past states and are left as measured, not bumped.

35 -> 36: README.md's summary line is the only other live count.
2026-08-13 23:24:38 +02:00
869f9058f7 docs: the fix closed a type confusion, not pointer-smuggling as a class
Both the LIMITATIONS bullet and the 1.1.0 CHANGELOG entry said "both now
FAIL_SECURE at T2, before the allowlist is reached" and stopped there. True,
and stronger than what shipped: the old bullet's closing clause about T3's
scope went out with the rewrite, so the text read as though a pointer can no
longer reach the consumer tree through frontmatter.

Measured, not reasoned:

  attester: attesters/sql_equality.py   -> WARN          (unchanged)
  resource: attesters/sql_equality.py   -> FAIL_SECURE   (unchanged)

T3 inspects `resource` and nothing else, so an honest string under another key
rides through exactly as before -- scanned under T1 like any other frontmatter
value, but never allowlisted. That is by design and is not what 1.1.0 changed.
Restored in both places.

Also corrects the row arithmetic: 13 added and 3 retired, not "11 added, 1
retired". Net +10 and 802 were measured and are right; the parenthetical was
not, and 2400 != 2401 is a locked convention here.

Tag v1.1.0 does not move: the code is correct, the prose about it was not.
802 passed, 35 limitations, :43 still the bullet's anchor line.
2026-08-13 23:05:52 +02:00
ca4f97c8c9 release: 1.1.0 -- the first behaviour change shipped under the freeze v1.1.0
Six live version surfaces bumped by hand (no sed -- provenance is never
bumped): pyproject.toml, __init__.__version__, README badge + Status + the
pinned pip install tag, docs/BRIEF.md, docs/ADOPTION-BRIEF.md Status, and
CLAUDE.md. ADOPTION-BRIEF's test count 792 -> 802.

NOT bumped, and deliberately: SECURITY.md's two `1.0.0` references name the
freeze BASELINE, not the current version -- "a payload that disposes WARN on
1.0.0 may dispose FAIL_SECURE on a later 1.x" is the promise this release
instantiates, so rewriting it to 1.1.0 would erase what it promised. The
GATE-G and PLAN-v1 numbers are the 1.0.0 gate record. The Forge repo
description carries no version (verified against the API last session).

This is the case SECURITY.md and the 1.0.0 CHANGELOG entry described in
advance: the exported surface is frozen, detection behaviour is not. No
exported name moved. A document that disposed WARN on 1.0.0 may dispose
FAIL_SECURE here; a consumer whose frontmatter carries an unquoted ": " in a
value will see those concepts refused at import, and quoting it parses.

Re-measured after the bump, alone: 802 passed, 129/129 classes, 6/6 gaps hold,
35 limitations.
2026-08-13 22:58:36 +02:00
da30211bc7 fix(okf): a mapping construct no longer degrades into a string, and there were two routes
T2 gives the mapping class no expressible form by design. Two routes escaped
that by parsing "successfully" into the wrong TYPE instead of raising:

  sources:\n  - uri: https://e.com/a   -> the string 'uri: https://e.com/a'
  attester: resource: attesters/x.py   -> the string 'resource: attesters/x.py'

Only the first was documented (LIMITATIONS.md:43). The inline second colon was
found by measurement while closing it -- a real YAML parser refuses that line
outright, ours accepted it. Shipping the list half alone would have left a
LIMITATIONS rewrite that overclaims.

Same consequence either way: a pointer parked in a degraded mapping rides
through in a key the `resource` allowlist never inspects, and mode-b
import_bundle wrote the merged concept verbatim (WARN). Both now FAIL_SECURE at
T2, before the allowlist is reached.

The boundary is where YAML puts it, ground-truthed against PyYAML 6.0.3 rather
than reasoned: ": " and a trailing ":" are exactly the two shapes where a plain
scalar becomes a mapping. A colon carrying neither a space nor a line end opens
no mapping -- domain:security and https://e.com:8443/a still parse -- and a
quoted scalar is still a scalar. Over-blocking a conformant bundle is itself a
failure mode, so the seven admitted shapes get rows of their own.

Iron Law: the four rejected rows and both import_bundle rows were written
first and seen red (7 failures, each DID NOT RAISE) before okf.py was touched.

Suite 792 -> 802. LIMITATIONS stays at 35: the bullet is reworded, not
retired -- the restricted grammar is still a limitation, the silent misparse
is no longer part of it.
2026-08-13 22:58:27 +02:00
6cd4694613 docs(plan): Session G landed, D4's rationale lives where the promise does, and the ninth surface was verified not bumped
Three durability gaps from the release, none of which block anything.

1. Session G had no LANDET marker. Every other landed session carries one
   with its sha and tag; the release that froze the surface did not.

2. D4's rationale existed only in STATE.md, which is LOCAL-ONLY and gets
   overwritten every session. The consumer-promise section says in its own
   words that promises live in this file and not in STATE, 'fordi STATE er
   LOCAL-ONLY' -- so a decision NOT to fire one belongs here too. A future
   session reading promise 1 would otherwise find no notification and no
   reason, which is indistinguishable from having forgotten it. The
   counter-reading is recorded with it: the wording ('enhver endring')
   pointed the other way, and it is a real argument, not a strawman.

3. 98ebc07's message says 'Nine current-state surfaces bumped by hand'.
   Eight were bumped. The ninth is the Forge description, which carries no
   version string and so was structurally invisible to the 421-hit sweep --
   the same blind-spot class as the status badge and Development Status.
   Verified against the Forgejo API instead: 178 codepoints, no version, no
   'alpha'. It needed no change, but 'verified' is not 'bumped' and a pushed
   commit message cannot be amended.
2026-08-13 22:45:11 +02:00
98ebc07b56 release: 1.0.0 -- the exported Python surface is frozen under semver v1.0.0
Nine current-state surfaces bumped by hand. The classification sweep ran
FIRST, before the first edit: 421 hits on 'v?0.N(.N)' across all tracked
files, each read and sorted current-state vs provenance. Provenance is
untouched -- 'New in v0.4.0', 'measured against 0.3.1', every '0.7.0'
in a code comment or a census candidate name still says what it measured.

The sweep found two surfaces the plan's nine-item list did not name:
README's status BADGE (still 'alpha' -- a version string grep cannot see
it) and ADOPTION-BRIEF's test count, which said 791 against a suite that
runs 792. Both corrected.

pyproject also moves Development Status :: 3 - Alpha -> 5 -
Production/Stable, likewise invisible to a version grep.

CHANGELOG [1.0.0] references [0.3.0] and [0.3.1] for the behaviour
changes rather than repeating them, and carries the freeze point itself:
what is frozen (the exported surface), what is deliberately NOT (all
detection calibration), the three conceded limitations, the one known
open defect (:43), and the runtime-coverage gap -- no external consumer
has run 0.7.0.

No code changed. Per docs/PLAN-v1.md the release gate is the whole suite
green, not a new test: 792 passed, coverage matrix 129/129 + 6/6, exit 0.
2026-08-13 22:20:48 +02:00
e9d8fb2b9d docs(limits): two limits are conceded for 1.x, and the last 'pending' is retired
D3: :492 (Severity carries disposition intent) and :460 (the input-cap
asymmetry) are the two limits that can only be closed by changing an
exported symbol. Both now say 'conceded for 1.x', with the 2.0.0
consequence spelled out, instead of 'deferred deliberately'.

D5: :88's 'a calibration fix is pending' is gone. The concession is
narrower than the other two -- no fix is promised, but it is calibration,
so one may land in any 1.x release without breaking the contract.

SECURITY.md carries the part an outsider acts on: the support window is
rewritten off 'pre-1.0', the freeze is stated as a promise about the
Python surface and explicitly NOT about detection behaviour, and the two
permanent concessions join the documented-boundaries list.
2026-08-13 22:15:02 +02:00
ab6000b1af docs(gate): the surface claim covers okf too, and one branch of D4 is already a defect 2026-08-13 22:07:18 +02:00
aff35118bf docs(gate): the v1.0 freeze is countable, and two of thirty-five limits are the gate 2026-08-13 22:02:28 +02:00
2466d260d3 test(redos): the dead row had the wrong payload, and the last wall clock is retired
Both rows that could not go red are decided, each by measurement.

test_lexicon.py::test_redos_pathological_subagent_input_returns_fast is REVIVED,
not retired. The row was not dead because the seed form is safe -- it was dead
because both earlier payloads made the prefix match at ONE start position, and
the cost is per-prefix-match. Repeating `spawn an agent that ` instead makes it
match K times, each driving its own O(N) lazy scan for a keyword never supplied:
K x O(N) against the seed's `(?:.*?\s+)?`, K x O(1) against the shipped
`{0,12}?` bound. Measured through scan_lexicon at 1500/3000/6000/12000 words:
seed 0.091/0.283/1.085/4.091s (exponent 1.92), shipped 0.047/0.051/0.094/0.190s
(exponent 1.01). Verified red with the seed form patched in: 4.21s against the
2.0s bound. The nesting the old comment blamed was a red herring.

test_output.py::test_pathological_input_returns_within_a_bound moves to CPU time
with a 20.0s bound, and the "or a hang" half of its claim is retired. The wall
clock was kept because a blocking hang burns no CPU -- true in general, and
inapplicable to a path with no open(), socket, subprocess, thread, lock or sleep
anywhere on it. Same payload, idle vs ~4x oversubscription: wall 3.30 -> 21.63s
(2x over the old 10.0s bound), cpu 3.30 -> 7.62s. It guarded a mode it could not
have while paying the full false-red premium. No in-repo vulnerable form can
turn this row red, so the bound was proved live against what it actually guards
-- a future pattern quadratic on long runs, `A+\s*EXFILTRATE` -- which failed it
at 64.77s CPU, 3.2x over.

redos_clock.py and the clock's pin test both documented this row as the
deliberate wall-clock holdout; both corrected.

792 passed, 6/6 documented gaps hold.
2026-08-13 21:47:43 +02:00
c48a2923ac test(redos): one CPU clock for every bound, and a second row measured dead
`5667063` moved test_output.py's ReDoS bounds off the wall clock, because a
loaded machine steals wall seconds without adding any cycles and two rows
failed at 2.24s / 3.66s against a 2.0s bound while census had the CPU. The
remaining ten bounds in five other files still ran on `time.monotonic()` and
carried the same defect. They now share ONE clock.

The clock is IMPORTED, not copied: `tests/redos_clock.py`. Five private copies
would leave four of them unpinned -- the instrument test
(test_the_redos_clock_ignores_time_this_process_did_not_spend) can only pin the
implementation it calls, and the suite already holds that rule for the code it
measures.

Every ported row was verified the only way a time bound can be: the vulnerable
form patched back in, red demanded, `git checkout --` after. Measured against
the 2.0s bound (3.0s for the url arm):

  active_content long-attr   `{0,63}` -> `*`        RED
  neutralize     long-attr   same patch             RED
  output gate    long-attr   same patch             12.41s
  okf link graph  `[^\]\[]` -> `[^\]]`               6.91s
  sanitize comment  str.find -> `<!--.*?-->`        17.56s
  lexicon md-link-anchor-text                      319.14s
  lexicon md-link-anchor-url                         8.55s
  lexicon md-link-ref-comment                       37.82s

Two rows did not go red, for two different reasons.

test_sanitize.py::test_legitimate_comment_heavy_document is the legitimate SIDE
of a separation, not a second pin on the defect: closed comments never withhold
the required literal, so the lazy form runs it in 0.016s. Recorded in place.

test_lexicon.py::test_redos_pathological_subagent_input_returns_fast is DEAD --
the same zero-signal shape the `<a ` carrier had, found by the same method. The
seed form is `(?:.*?\s+)?` (llm-security 7.8.0, injection-patterns.mjs:84) and
this repo has never carried it: the bounded `{0,12}?` port is in the pattern
table's first commit. Patched in by hand at the row's own size: shipped 0.135s
vs seed 0.113s, separation 1.2x. Not the keyword gate either -- a variant that
reaches the inner branch stays linear over four doublings (exponent ~1.0),
because the nesting is one lazy run inside an OPTIONAL group, never a repeated
one. Left standing with the measurement written into it; picking a new carrier
is an operator call, like the wall-clock row above it.

The dead sibling row named in STATE is fixed: test_active_content.py's
long-attribute row swaps carrier `<a ` -> `<script `, for the reason `5667063`
established on its composed-gate twin -- 0.7.0's own no-URL narrowing put `<a>`
in `_URL_AFFORDANCE_TAGS`, so the tag returns inert BEFORE its body reaches the
arm the row guards. Re-measured here, not inherited: `<a ` 0.041s and NO
findings against the vulnerable form; `<script ` 19.349s against 0.052s
shipped, 373x apart.

`test_pathological_input_returns_within_a_bound` deliberately keeps its wall
clock (operator decision): it claims to catch a hang, and only a wall clock
catches one.

792 tests, 129/129, 6/6.
2026-08-13 21:25:36 +02:00
566706360a test(output): the ReDoS bounds move to CPU time, and the long-attribute row was dead
Two rows failed at 2.24s against a 2.0s wall-clock bound while two census
processes had the CPU, and passed 3/3 on an idle machine. Reproduced under
artificial oversubscription before touching anything (16 logical / 8 physical
cores), `lexicon-script-tag` shipped form, idle vs 2x vs 4x:

    wall  0.74s -> 3.55s -> 8.13s   (11x, still climbing with load)
    cpu   0.74s -> 1.42s -> 1.50s   (2.0x, flat from 2x to 4x)

Wall-clock inflation tracks how many other processes want the CPU and has no
ceiling; process CPU inflation is bounded by SMT and memory contention. The
bound itself is UNCHANGED at 2.0s: on an idle machine the two clocks are the
same number (measured ratio 1.00), so every figure in the derivation comment
stays true as a CPU-time figure. Raising the bound instead was rejected by
measurement -- the vulnerable `[^>]` script-tag form runs 4.0s idle, so any
wall bound loose enough to survive load would let the defect pass.

Signal verified by patching the pre-fix forms back in: both rows go red
(4.49s and 18.85s against 2.0s) and green again with the shipped forms.

That verification found the second, worse defect. 0.7.0's own no-URL narrowing
killed `test_gate_is_bounded_on_the_long_attribute_arm`: `<a>` is in
`_URL_AFFORDANCE_TAGS`, so the bare `<a ` + 100k + `>` payload is now inert and
returns BEFORE the body reaches `URL_IN_TEXT_RE` -- the arm the row exists to
guard. Measured against the vulnerable form through `scan_active_content`:

    <a ...>       0.028s and NO findings   <- dead: separation 1.0x
    <script ...> 12.475s
    <a href=x …> 12.719s
    <form ...>   17.092s

The carrier is now `<script `: active by NAME with no attributes, so no future
URL-shaped narrowing can make it inert the same way. 0.53s shipped vs 18.85s
vulnerable through `scan_output` -- 35x apart, bound 3.8x above the shipped side.

`test_pathological_input_returns_within_a_bound` deliberately keeps its wall
clock: it claims to catch "a hang or a blowup", and only a wall clock catches
the first. New instrument test pins the clock via the same helper the bounds
use, so the choice cannot drift silently.

792 tests, 129/129 coverage, 6/6 gaps. The ReDoS block passes 3/3 under the
4x oversubscription that produced 8.13s wall.
2026-08-13 21:02:31 +02:00
be9759b4b3 docs(census): the two wiki corpora are measured, and the tightening is thinner than zero looked v0.7.0
0.7.0 shipped with its corpus numbers deliberately absent: the census had only
run on reference-corpus, which the 0.6.0 narrowing had already emptied of
raw-HTML drivers, so it bounded the change rather than showing it. Both wiki
corpora are now measured through the pinned instrument, in one session because
they are living populations.

Under PRESET_USER_UPLOAD, 0.6.0 as shipped -> 0.7.0, each population against its
own denominator (the two wiki corpora share content and are never summed):

  reference-corpus   389 docs   54 -> 53   ceiling 53
  vendor-harvest     187 docs   62 -> 20   ceiling 18   42 of 44 achievable
  generated-notes    552 docs   59 -> 15   ceiling 13   44 of 46 achievable

PRODUCTION matched C1 + D field for field in every population, which is the
check that the instrument and the shipped predicate have not drifted.

Two things the measurement changed rather than confirmed:

The pair is superadditive by 13 documents in BOTH wiki corpora. Alone, the split
frees 8 in each and the narrowing 21 and 23; together they free 42 and 44.
Shipping either alone would have measured as barely worth the label.

The TIGHTENS column reads 0 on both tiers in all three populations, but that zero
is empirical and thin: the split measured ALONE tightens 13 documents on the
trusted tier in vendor-harvest and 14 in generated-notes, and the narrowing
cancels each one. LIMITATIONS now says so explicitly, so nobody reads the zero as
'cannot happen' -- the escalation is still constructed and pinned by
test_split_tightens_the_trusted_tier_when_both_carriers_are_present.

generated-notes counted 552 documents, not the 550 the scratchpad probe saw.
Living corpus, measured fresh.

791 passed; coverage 129/129, 6/6 documented gaps hold.
2026-08-13 20:22:18 +02:00
72de0e0c15 docs(claude-md): the linking example published one machine's home directory
org-ops census sweep 07 flagged CLAUDE.md:56 as LINK-FILE-URL (ERROR): the
example link carried file:///Users/ktg/..., which is dead for every reader but
the author and publishes the operator's directory layout on a public surface.

The rule itself is unchanged — absolute file:// paths with a descriptive name
are still what a response should use. Only the example is now a placeholder,
with a note saying why it has to stay one.

Note for the verify step: 'grep -n file:///' over this file cannot go empty by
construction — line 48 specifies the link syntax and must contain the literal.
The distinguishing check is 'grep -n file:///Users/', which is now empty.
2026-08-13 20:08:12 +02:00
fcfaee4589 feat(active-content): raw HTML graded on carrier, and a tag naming no target is inert
Two changes that had to ship together, because they co-occur.

`active:raw-html-link` (MEDIUM) splits the click-required carriers out of
`active:raw-html`. The same URL was LOW as `[t](url)` and HIGH as
`<a href="url">` — an asymmetry produced by syntax, not by affordance, on a
carrier the markdown path has graded MEDIUM since 0.3.1. The event-handler test
runs first, so `<a onclick=...>` stays HIGH. The url-attribute branch stays HIGH
too: a name outside the active set has unknown rendering, and grading
`<Card src=...>` as a link would be reasoning rather than measurement.

The no-URL narrowing makes `</a>`, `<Frame>`, `<video />` and `<img alt=...>`
without `src` inert — `<base />`'s argument from 0.6.0 applied to the rest of the
name branch. It tests for the URL attribute's PRESENCE, not for a readable value,
so the fail-secure gap `_url_attr_is_external` leaves open is not reopened here.

WHY TOGETHER: the narrowing strips a document's `</a>`/`<Frame>` and what remains
is the `<a href=...>` the split grades down, so each alone leaves the document
blocked by the other's residue.

`active_tag_class` is now the classification point and `is_active_tag` wraps it.
The census patches the former: a boolean could only express a narrowing, never a
regrade, so every carrier candidate would have measured equal to PRODUCTION —
silently, and in the direction that reads as "no change helps".

TWO COSTS, BOTH RECORDED RATHER THAN GLOSSED:

- The split TIGHTENS the trusted tier. One finding becomes two, and >=2 findings
  at MEDIUM+ trip the compound overlay, so a document carrying both an `<img src>`
  and an `<a href>` goes WARN -> quarantine_review on PRESET_TRUSTED_SOURCE. On
  that preset it is the only direction the split can move anything. The census
  now reports a TIGHTENS column on both trust tiers against the previously
  shipped row — "frees N" without "tightens M" is a one-sided number.
- `count` drops on documents containing `</a>`, a published field moving under a
  meaning that did not change.

MEASURED: reference-corpus (389) 54 -> 53 fail_secure, tightens 0/0, and the
census `PRODUCTION` row equals its `C1 + D` candidate row for row. The census
also reproduces 133/3/13/108/25 exactly, so it is calibrated against every
published historical number. The two wiki corpora are NOT yet re-measured; the
tree says so explicitly in the docstring, LIMITATIONS and CHANGELOG rather than
carrying probe numbers as fact.

791 tests (was 759), coverage 129/129, 6/6 documented gaps holding. Version
bumped to 0.7.0 across every surface; no tag is set until the measurement lands.
2026-08-12 00:42:44 +02:00
0df7e87c2f test(docs-scripts): the census re-derived the URL reader it measures, and nothing would have caught the drift
`docs/fp-sweep.py` and `docs/rawhtml-census.py` produce the numbers published in
`docs/LIMITATIONS.md`, and both reach past the public API into private module
state -- `active_content._ACTIVE_TAGS`, `_EVENT_ATTR_RE`, `_URL_ATTR_RE`,
`_has_external_target`, `calibration.RISK_RANK`. A rename inside `src/` broke
them while the suite stayed green, and the breakage would have surfaced months
later, at the moment someone tried to re-measure a published claim. This was the
last uncovered contract in the repo.

The coverage found a live one. `rawhtml-census.has_external_url_attr` parsed
attributes with its own pattern instead of calling the shipped reader, and the
copy had drifted on two shapes:

  - a URL attribute reached through a prefix. `_URL_ATTR_RE` matches `src=`
    inside `data-src=` on a word boundary, so the shipped predicate reads the
    value and blocks; the census's own name table saw `data-src` and skipped it.
  - a multi-candidate `srcset`. The shipped reader splits on `[,\s]+` so a
    relative first candidate cannot mask an external one behind it; the census
    tested the whole attribute value as a single URL.

Both made the `A` candidate rows free documents the shipped predicate keeps --
under-counting against the `PRODUCTION` row printed directly beside them, which
that row exists to expose. The census now delegates to
`active_content._url_attr_is_external`, so there is one reader, not two. This
repo already carries the general form of that lesson in `docs/URL-SHAPE.md`:
three consumers reconstructed a predicate from prose and each got a different
wrong answer.

NO PUBLISHED NUMBER MOVED. Re-measured against all three live populations after
the fix, in one session each: reference-corpus 389 docs (A frees 3, base-url 13,
both 25 -- the numbers in `docs/LIMITATIONS.md`, unchanged), vendor-harvest 187,
generated-notes 550. `PRODUCTION` equals `A + base-url` in all three (108/108,
98/98, 88/88), and vendor-harvest exercises the corrected branch for real (8
external `Card` attributes). The defect was latent, not published.

The tests are scoped to what a test here can honestly hold. The corpora live
outside this repo in private consumer repos, so neither script can be run end to
end from the suite and a stand-in corpus would only pin a fiction. What is
pinned: every imported name still exists with the shape used; `fp-sweep`'s
metric guard fails when the action map is re-mapped (a guard that cannot fail
protects nothing); `measure` still reads `.disposition` and `.assessment` and
excludes empty files from the denominator; the census's in-process patch point
still moves the gate, so a census patching a dead symbol cannot print six
identical rows and read as a finding; `PRODUCTION` equals `A + base-url` across
every branch of the predicate; and both scripts still refuse an argument-less
run rather than measuring nothing.

759 passing (was 736). Coverage matrix unchanged at 128/128 with 6/6 documented
gaps holding. The `736` in `docs/ADOPTION-BRIEF.md` is scoped "as of v0.6.1" and
is correct for that tag; it moves to 759 at the next version bump.
2026-08-11 23:01:58 +02:00
0903785187 release(0.6.1): the ZWJ fix gets a tag a consumer can pin v0.6.1
a59184b fixed a hard-block that reached every emoji-composed document, and
llm-ingestion-okf was told so -- while the fix sat untagged on main. A consumer
cannot pin what has no tag, so the notice was a promise the repo had not kept.

SURFACES MOVED (the eight the 0.5.0 sweep established, plus the ninth verified)

  pyproject.toml           0.6.0 -> 0.6.1
  __init__.py              0.6.0 -> 0.6.1
  CHANGELOG.md             [0.6.1] entry, [Unreleased] reset to "Nothing yet."
  README.md                badge, install tag @v0.6.1
  docs/ADOPTION-BRIEF.md   `v0.6.0` x2, and 727 -> 736 passing
  CLAUDE.md                one clause: ZWJ judged by context, not identity
  SECURITY.md              `0.6.x` -- still true at 0.6.1, verified, not moved
  docs/BRIEF.md            `v0.6 (alpha)` -- likewise
  README.md:36             `v0.6`, alpha -- likewise
  forge description        re-verified THIS release: 178 codepoints, no version
                           claim. Not inherited from 0.6.0's check.

PATCH, NOT MINOR -- and it was asked, not assumed. 6bcb898 made "loosens the
upload door" the criterion for minor, and this loosens it too, so the reading
was put to the operator with the count rather than settled quietly. 0.6.0 was a
policy choice (`<base>` left the active name set by decision); this restores a
contract the module already published, against a class never meant to be
blocked. Measured loosening: 1 document of 1126 across the three FP populations
(reference-corpus 1/389, vendor-harvest 0/187, generated-notes 0/550), against
0.6.0's 25 + 2 + 2. Operator chose patch.

VERIFIED ON THE BUMPED TREE, NOT THE PRE-BUMP ONE

  736 passed; coverage 128/128 recall, 6/6 documented gaps hold
  `git grep '0\.6\.0'` returns provenance only -- LIMITATIONS history, the
    census script, source docstrings, tests, README's pre-0.6.0 numbers
  docs/LIMITATIONS.md: 34 items, README says 34
  __version__ and pyproject agree at 0.6.1
  no tracked file carries a stale current-state version or test count

The sweep tool itself was wrong first: `git grep -E '\b727\b'` returns nothing,
because POSIX ERE has no `\b`. An empty result read as "clean" when the claim
was there. Every sweep above is plain `git grep`.

Still to prove before this is announced: a scratch-venv install at the tag with
the resolved version asserted -- the README line above is only true once v0.6.1
resolves.
2026-08-11 22:30:32 +02:00