1
0
Fork 0
Commit graph

15 commits

Author SHA1 Message Date
6bcb898632 release(0.6.0): the raw-html narrowing ships, and the version claims it already made become true
The narrowing landed in 736f370 and its prose asserted `0.6.0` in thirteen places
-- README's public front page among them -- while every version surface still read
0.5.0. That is the same defect class the last three commits were spent correcting:
a published number no measurement backs. Two ways out, land it or neutralize the
references; operator chose to cut.

SURFACES MOVED (the eight the 0.5.0 sweep established, plus the ninth verified)

  pyproject.toml           0.5.0 -> 0.6.0
  __init__.py              0.5.0 -> 0.6.0
  CHANGELOG.md             [Unreleased] -> [0.6.0], fresh [Unreleased]
  README.md                badge, `Status: v0.6`, install tag @v0.6.0
  SECURITY.md              support window `0.5.x` -> `0.6.x`
  docs/BRIEF.md            `v0.5 (alpha)` -> `v0.6 (alpha)`
  docs/ADOPTION-BRIEF.md   `v0.5.0` x2, and 717 -> 727 passing
  CLAUDE.md                `v0.5 (alpha)` -> `v0.6`, plus what 0.6.0 changed
  forge description        178 codepoints, carries no version claim -- verified,
                           not moved. A surface can be checked and stay still.

Measurement provenance is left alone, as in 0.5.0: `tests/test_disposition.py`'s
"0.5.0 axis separation", `disposition.py` and `calibration.py` docstrings,
LIMITATIONS' 0.5.0 reference, every dated claim in docs/PLAN-v1.md. Bumping those
falsifies the record rather than updating it.

MINOR, NOT PATCH: 0.6.0 loosens the upload door. A document whose only finding was
a doc-relative URL attribute on an inactive tag name, or an attribute-less
`<base />`, now WARNs where it was held -- 25 documents in the reference corpus, 2
in each wiki corpus.

VERIFIED BEFORE COMMITTING, NOT AFTER

  727 passed; coverage 128/128 recall, 6/6 documented gaps hold
  docs/LIMITATIONS.md: 33 items, README says 33
  rawhtml-census PRODUCTION row equals `A + base-url` on all three populations --
    the shipped predicate measured, not a hypothesis about it
  redos-sweep: 0 candidates of 152 patterns
  docs/fp-sweep.py still imports the private names it reaches into
  no tracked file carries a stale current-state version claim

Still to prove before the tag: `git show` over this commit's README, and a clean
clone install at this sha.
2026-08-11 18:01:36 +02:00
f669479777 release(0.5.0): the assessment axis ships, and every version surface moves with it
0.5.0 is the axis separation `de09711` built: `Risk` (assessment) alongside
`Disposition` (action), `Policy.action_map` as the supported override, and the
fail-closed path pinned to both axes. Additive and measured to be so — 717
passing with no test changed, matrix 128/128 with 6/6 documented gaps, the
`PRESET_USER_UPLOAD` grading table unchanged row by row. Plus the field FP
measurement (`d1bff60`) and the 0.3.3 behaviour-change correction (`d3d0928`).

WHY THIS COMMIT TOUCHES EIGHT FILES AND 0.4.0's TOUCHED THREE

0.4.0's release commit updated CHANGELOG, pyproject.toml and __init__.py, and
deferred README deliberately: the install block should not name a tag before a
clean-venv install had proven it resolved. Sound reasoning, and the proof step
never ran — so tag v0.4.0 permanently advertises v0.3.4. The tag is not moved.
The ordering is.

Sweeping every tracked file for a version claim, instead of ticking the four
surfaces the checklist named, found five more that no release had ever touched:

  SECURITY.md          "pre-1.0 (0.2.x)" — the one with a consequence for an
                       outsider: it named a support window two minor lines
                       behind the code.
  README.md            "**Status:** v0.3" — the front page, stale since 0.4.0.
  docs/BRIEF.md        "v0.2 (alpha)" — stale since 0.3.0.
  CLAUDE.md            "v0.2 (alpha)" and "12 moduler" where src/ has 15.
  docs/ADOPTION-BRIEF  "703 passing" where the suite is at 717.

Measurement provenance is deliberately left alone: "New in v0.4.0", "verified
identical on 0.2.0 and 0.3.1", "measured against the v0.3.1 tag", every
"post-0.4.0 tree" in LIMITATIONS. Bumping those falsifies the record instead of
updating it, which is why this cannot be a sed sweep — the surfaces have to be
sorted into current-state and provenance before a single edit.

Found because llm-ingestion-okf took our report of this defect class as a
hypothesis about their own repo, measured it, found a worse instance on their
public front page, and sent back the generalization: writing down a trap is not
the same as applying it.

VERIFIED BEFORE COMMITTING, NOT AFTER

  717 passed; coverage 128/128 recall, 6/6 documented gaps hold
  docs/LIMITATIONS.md: 33 items, README says 33
  fp-sweep reproduced all three published numbers exactly on the bumped tree —
    vendor-harvest 98/185 (53.0%), generated-notes 88/547 (16.1%),
    reference-corpus 133/389 (34.2%) — and self-docs runs clean, so the
    untested script survived the bump it imports names from
  forge description: 178 codepoints, under the 180 cap
  no tracked file carries a stale current-state version claim

Still to prove before the tag: a clean-venv install from this commit's sha, and
`git show <sha>` over the README. The install proves the package builds; only
the grep proves the text the tag will carry is right. That second check is the
one the old ordering could not perform, because by then the tag existed.
2026-08-11 06:42:56 +02:00
de097110d2 feat(disposition): separate the assessment axis from the action
`decide` returned a `Disposition` — WARN / QUARANTINE_REVIEW / FAIL_SECURE —
which names an ACTION. But BRIEF design principle 4 says the library reports
and the pipeline decides, and disposition.py admitted the gap in its own
docstring: "It imposes no blocking of its own." So we returned an action we
cannot enforce, having discarded the judgement that produced it. A consumer
wanting different behaviour had to reinterpret the action itself — which is
why a consumer ends up pinning our GRADING: the action was all they got.

`Risk` (NONE/LOW/ELEVATED/SEVERE) now carries that judgement, and
`Policy.action_map` lets a caller map it to their own action. Both overlays
move the assessment rather than the action, so a custom map cannot silently
drop the compound escalation or the quarantine floor. `guard`'s fail-closed
path pins both axes and deliberately bypasses the map: downgrading SEVERE
means "I accept this class of finding", never "I accept a crashed scanner".

`DispositionResult.assessment` is required with no default. `Risk.NONE` is the
natural-looking default and the wrong one — a site that forgot the field would
report clean, and the axis would fail open.

MEASURED ADDITIVE, not assumed:
  - 703 -> 715 tests, no existing test changed
  - coverage matrix 128/128 recall, 6/6 documented gaps still hold
  - the PRESET_USER_UPLOAD grading table locked in 0.3.1 re-measured row by
    row: ordinary link/image/autolink/refdef -> warn on BOTH doors, unchanged

Both locked consumer promises in docs/PLAN-v1.md were checked against that
measurement and neither fires: the grading is untouched (linkedin-studio), and
the relative-target asymmetry is untouched (llm-ingestion-okf).

Scope held to disposition, per PLAN-v1.md:380. Version stays 0.4.0; the 0.5.0
bump lands in its release commit with all five version surfaces at once —
that is the fix for the defect where the v0.4.0 tag carried a 0.3.4 README.

Records limitation 32: `Severity` still carries disposition intent on the
DETECTION side, which this change does not address and cannot without moving
the grading.
2026-08-10 20:55:46 +02:00
73ad5e1ed8 release(0.4.0): the raise is breaking, so the number says so
okf answered the exposure question with a measured no — zero call sites for the
three capped surfaces anywhere in their src/, and their screen_output path
reaches only scan_output, which truncates. So the blast radius is zero and 0.3.5
would have hurt nobody.

It is still 0.4.0. Adding a raise to a previously total public function is
breaking under SemVer, and this file declares SemVer; a survey that comes back
clean tells you the upgrade is easy, not that the contract held. The version
number describes the change, not the luck of who happens to call it.

Cost, stated: 0.4.0 was penciled in for the axis separation. That moves to 0.5.0.
Tags are never moved, so the choice is made once.

README still advertises v0.3.4 — it is updated after a clean-venv install proves
the tag resolves, so the install block never points at something unproven.
2026-08-10 14:49:47 +02:00
2d98d6809d feat(sanitize,fence,neutralize): reject oversize input instead of half-transforming it
The scanners cap by truncating: they return findings, so reading a prefix costs
detection in the tail and nothing else. The three transform surfaces return
*content*, where the same move is not available — a shortened document is silent
data loss, and a transformed prefix followed by an untransformed tail is a
bypass, since the attacker chooses where in the document the payload sits.

So they fail secure instead. Above MAX_INPUT_CHARS (1 000 000) sanitize, fence
and neutralize raise OversizeInputError. sanitize is step 1 of prepare_input and
only ever removes, so that one refusal bounds the whole input path.

OversizeInputError subclasses ContractViolation: a pipeline already bracketing
its quarantined stage keeps failing closed rather than meeting a type it has
never heard of. It inherits the alert-routable property too — sizes in the
message, refusing surface in details, no input in either.

Invariant now pinned across all three: returned text is always fully
transformed, or not returned at all.

Still uncapped and recorded in LIMITATIONS: scan_active_content called directly
(through scan_output it inherits that cap) and the okf link graph. Both are
detection-shaped, so truncate-and-flag transfers unchanged — mechanical, not
policy.

699 tests (+23), coverage 128/128 + 6/6, ReDoS sweep 0 candidates / 150.
2026-08-02 21:13:08 +02:00
adf93e47fb release(0.3.4): three quadratic patterns outside the lexicon, two on the input path
0.3.3 swept 83 lexicon patterns arm by arm and left the other ten regex-bearing
modules on 0.3.2's hand-written rows. Generalising the sweep found three more,
and the two on the input path matter more than the count suggests: `sanitize` is
step 1 of `prepare_input`, and `MAX_SCAN_CHARS` is applied in `scan_lexicon` and
`scan_output` only, so there was no cap to extrapolate to. That missing input
cap is now a documented residual of its own -- extending it changes the contract
for existing callers and is not something to smuggle into a ReDoS patch.

Version synced in all four places + CHANGELOG. LIMITATIONS 30 -> 31 items;
the 0.3.3 entry claiming the lexicon sweep's scope is corrected in place, since
"the output path" was never the whole surface either.

676 tests, coverage 128/128 + 6/6 gaps, sweep clean across 150 patterns.
2026-08-01 20:08:51 +02:00
701a4a47c7 release(0.3.3): the lexicon ReDoS fix, and a correction to 0.3.2's claim
Version synced across the four locked points (pyproject, __version__, README
badge, README install pin) + CHANGELOG.

Corrections this release carries, both measured rather than reasoned:

- docs/LIMITATIONS.md said the script-tag change removed "the last"
  quadratic-backtracking site on the output path. It did not. Corrected, with
  the 334.7s gate measurement that falsifies it.
- README's coverage line claimed 126/126 classes; the matrix reports 128/128.
  Stale since before v0.3.2. Test badge was 642, actual 666.

New residual recorded (LIMITATIONS, now 30 items, README synced): the sweep
flags on timing above a 1.5ms noise floor at N=8000, so an arm hiding under it
could still cost ~23s at the cap. What this supports is "no arm worse than ~23s",
not "no quadratic arm remains" -- and the blind spot is demonstrated, since a
generic-payload pass found only one of the two patterns.

666 tests green, coverage matrix 128/128 with 6/6 gaps holding, exit 0.
2026-07-31 21:51:00 +02:00
75ae48277b release(0.3.2): the ReDoS fix, with both residuals measured against the v0.3.1 tag
Version sync in all four places (pyproject, __version__, README badge, README
install pin) plus the changelog entry for cff0437.

Two claims that were about to ship in the release note did not survive being
measured against a v0.3.1 worktree, and are corrected in LIMITATIONS first:

- "Report-only, so the cost is a review, not a block" was wrong twice. HIGH under
  a low-trust preset is fail_secure, not a review -- report-only means the text is
  never mutated, not that a finding cannot block. And the new script-tag false
  positive costs no consumer a disposition at all: any text containing a literal
  `<script>` already produced active:raw-html at HIGH on 0.3.1, so the same prose
  disposed fail_secure under PRESET_USER_UPLOAD before this change and after it.
  The fail-open that was closed is narrower for the same reason -- it existed only
  in scan_lexicon called on its own; through either composed gate, raw-html
  already caught the unclosed tag. What changed is the label, not the outcome.

- "The realistic long value is still caught by egress:jwt-token" was true and
  hid the part that matters. Measured at the 257-char boundary: a generic long
  password still trips entropy:base64-blob at CRITICAL, disposition unchanged.
  A JWT used as a DB password is the case that moves -- its remaining detections
  top out below CRITICAL, so the any-tier block is lost and PRESET_TRUSTED_SOURCE
  drops from fail_secure to quarantine_review. PRESET_USER_UPLOAD still
  fail_secures. Recorded as a behaviour change in the changelog, not buried.

The first probe for that boundary used a 300-char run of "A" and found nothing on
either version: all-same-char values are suppressed as placeholders. The probe was
wrong, not the pattern.

662 passed, coverage matrix exit 0, LIMITATIONS still 29 items = README's 29.
2026-07-31 21:16:45 +02:00
6e9b8168e3 fix(calibration): grade active content on URL shape, not construct type
v0.3.0 made the untrusted upload path unusable: measured on both doors, an
ordinary remote image fail_secure'd and an ordinary link/autolink/refdef
quarantined, so only documents without external references persisted.

Two independent defects compounded; neither fix works alone:

1. `markdown-image: HIGH` fired on any external image. The exfil primitive is a
   URL that moves bytes outward, not an image. `is_ordinary_url` now grades on
   shape - http(s)/protocol-relative, no query, no userinfo, no percent-escape,
   no opaque host label or path segment -> LOW; anything data-carrying keeps the
   carrier's severity. raw-html and data: URIs stay HIGH unconditionally.
   Opacity reuses entropy's primitives; floors calibrated against real doc URLs
   (worst legit token H=4.08, exfil segments 4.36-4.54) and frozen in
   calibration.

2. The quarantine_default floor fired on ANY finding, a premise that broke when
   every ordinary link became a finding. It now fires at MEDIUM+ - a no-op for
   every detector that shipped before 0.3.0 (no LOW/INFO exists), which is what
   makes this a patch rather than a minor.

The corpus blind spot that let this pass 522 green tests is closed: the FP
corpus carries realistic markdown and is asserted on the OUTPUT gate under
PRESET_USER_UPLOAD, with a counter-corpus of exfil-shaped URLs that must still
block. Beaconing and short opaque segments are conceded in LIMITATIONS and
asserted by the coverage matrix rather than papered over.

No new public API; no new preset (0.4.0 work); allow_reserved default unchanged.
2026-07-25 15:36:02 +02:00
467b9e3e13 chore(release): 0.3.0 — a gate-loosening change earns a minor, not a patch
Version-sync across pyproject, __init__, README badge/status/install pin and
CHANGELOG, and cut [Unreleased] to [0.3.0].

Why minor: 0772daf made allow_reserved default to True on okf.import_bundle,
so a consumer who upgrades without touching their code gets a LOOSENED gate --
a reserved index.md/log.md in a received bundle is now scanned and may become
mergeable, where v0.2.0 rejected it unconditionally. Under 0.x a >=0.2,<0.3
pin absorbs a 0.2.1 silently but stops at 0.3.0, which is the signal such a
change should send. Found by llm-ingestion-okf, our first real downstream
consumer, against main.

Development Status stays 3 - Alpha and the README keeps "the public API may
still change": this ships the hardened surface (Sessions A/A2/B) to consumers
who are still pinned at v0.2.0 and therefore have none of it. It is not the
v1.0 freeze -- that stays gated on the first real integration coming back
green, which is exactly what 0.3.0 makes possible.

Verified: 522 passed; coverage matrix exit 0; version string present in all
four files; no dangling 0.2.0 outside CHANGELOG history.
2026-07-25 12:24:36 +02:00
4d53765c63 feat(guard): active-content detector wired into the output gate (review MAJOR #1)
Close the EchoLeak wiring hole (CVE-2025-32711 class): markdown images/
links, reference definitions, autolinks, raw active HTML and data: URIs
now surface as report-only findings (active:*, OWASP LLM05) in
scan_output step 6, so screen_output and okf.import_bundle dispose of
them instead of admitting them with findings=[].

- new active_content.py: canonical home of the shared pattern table +
  scan_active_content; neutralize refactored to import it (mutating API
  and behavior unchanged, all neutralize tests pass as-is)
- images/links flagged only for absolute/protocol-relative URLs:
  relative in-bundle links are legitimate wiki/OKF mechanism (principle 5)
- evidence carries defanged URLs only (hxxps://evil[.]example)
- EchoLeak vectors planted in both showcases; detach proofs cover them
- README export list + checklist step 6, CLAUDE.md context line updated

Suite: 321 -> 341 passed. Core invariant intact (dependencies=[]).
2026-07-15 06:11:33 +02:00
542ac92349 release: v0.2.0 — OKF adapter (stream 1); version sync pyproject/__version__/badge/CHANGELOG 2026-07-06 09:45:53 +02:00
07e0b2153a feat(okf): wire adapter into public API — import_bundle carries link graph, package exposes okf namespace (TDD, +2) 2026-07-06 09:44:41 +02:00
0af8f68cae feat(wiring): §6 bookends (prepare_input/screen_output) + full public API + README v0.1 (TDD)
Module 11 (final) — the top-level wiring. The library never makes the model call
(no SDK imported by the core), so the public surface is the toolkit plus two
library-side bookends around the caller's tool-less transform (Form 1, chosen
with the operator over an export-only toolkit and a full orchestrator — the
bookends fit existing pipelines with least friction, encode the two halves the
library can stand for, and impose no control flow):

- prepare_input(text, source=INPUT) -> PreparedInput(fenced, nonce, report):
  §6 steps 1-2, sanitize THEN fence (carrier can never smuggle a forged
  delimiter). Merged report carries both steps' findings; renders no disposition.
- screen_output(text, policy, *, provenance, transform_failed) -> DispositionResult:
  §6 steps 6-7, scan_output under guard() so a scanner error fails CLOSED
  (FAIL_SECURE, never a silent persist). transform_failed routes the compound
  forced-fallback rule.
- __all__ exports the full framework-agnostic surface: detectors, result types,
  disposition machinery + presets, contract asserters, the grounding seam.

Docs: README refreshed from the stale "brief / pre-implementation" line to a v0.1
alpha status with a Form-1 quickstart, the §6 adopt-this checklist, and an honest
-limitations section (structural unsolvability at the text layer; semantic
poisoning invisible to lexicon+entropy; text-only, no multimodal). CHANGELOG
seeded; CLAUDE.md remote/status lines corrected (remote IS set, no longer
brief-stage).

10 wiring tests (public surface, prepare_input compose, screen_output fail-closed
+ compound). 189 green (showcase + corpora follow).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HyRCQMocjZ6SmSQ6JidJ2k
2026-07-04 23:34:28 +02:00
a9c4ccd8c7 feat: scaffold package + report and sanitize modules (TDD)
Build order steps 1-2 of docs/PLAN.md:
- pyproject.toml (llm-ingestion-guard, stdlib-only core, extras [ml]/[judge]/[dev]), LICENSE (MIT)
- report: Finding/Report/Severity/Source shared type (pure data)
- sanitize: carrier stripping (zero-width, BIDI, Unicode-tag, HTML comment,
  data: URI) with the byte-identical / removes-only invariant
- docs/PLAN.md: v1 implementation plan (positioning A, gap-expanded scope,
  llm-security reuse map)

15 tests passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K8GmKRCdsPjWYAKWsNgeQS
2026-07-04 09:24:20 +02:00