llm-ingestion-okf/CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.4.0 — 2026-07-25

Phase 2. The bundle inbox (Door B) and external-bundle import (Door C) ship, and with them this library's first — and only ever — runtime dependency.

Added

• Door B — bundle inbox (process_inbox). Per dropped file: bytes → extraction → persist gate → render → collision gate → write → index link. Returns an InboxResult whose four buckets (persisted, quarantined, rejected, failed) are disjoint and complete, so a file that vanished mid-run surfaces as a missing entry rather than as nothing. One bad file never aborts the run: only an invalid ingested_at, a reserved okf_type, and a missing inbox directory fail the whole run, each being wrong for every file at once. Concept files are named inbox-{slug}.md, disjoint from index.md, Door A's ingest-*, and promoted-verdict-*; source_sha256 is taken over the original dropped bytes, so provenance stays re-verifiable against the operator's file.
• Door C — external bundle import (import_bundle). Reads an external OKF bundle and hands it whole to an injected gate over the guard's okf.import_bundle — a bundle-level call, because it resolves the cross-link graph across concepts — merging only concepts that clear the non-blocking floor. Two invariants, both load-bearing: a merged concept is written verbatim (stamping it would require round-tripping frontmatter through this library's line-oriented parser, which cannot represent the block lists the guard's parser accepts, and would persist bytes the guard never screened), and ownership is therefore proven by content identity — an occupied target name is re-used only when the bytes there are already identical, never overwritten. Re-importing an unchanged bundle is a no-op.
• extract_text — the Door B extraction registry. All file-type → text extraction lives in this library, because the guard is text-only. md/txt pass through, csv renders the markdown table, json is fenced verbatim, and html/htm reduce to text with html.parser — stdlib throughout. pdf, docx, and xlsx are gated behind the [extract] extra, which ships no parser yet: those types fail fast with a typed error naming the extra, never a silent skip.
• llm_ingestion_okf.guard_adapter — the shipped gate over the real guard (inbox_gate, import_gate), and the only module here that imports it. Door B screens the exact bytes it persists: the guard's prepare_input bookend prepares text for a model call this library never makes, so screen_output alone is used and the screened string is the written string. It follows that the gate refuses rather than repairs — a file carrying an invisible carrier is rejected, not stripped and persisted. The policy is PRESET_USER_UPLOAD, so any finding at all is held back.
• 15 new stable error codes, each registered in the errors.py docstrings (the stability contract) and pinned by the error-code conformance suite: extractor_decode_error, extractor_empty_csv, extractor_extra_missing, extractor_unknown, import_label_invalid, import_path_empty, import_path_too_long, import_provenance_invalid, import_slug_collision, inbox_slug_collision, inbox_slug_empty, inbox_slug_too_long, inbox_source_file_invalid, inbox_title_invalid, okf_type_reserved.

Changed

• llm-ingestion-guard>=0.2,<0.3 is now a mandatory runtime dependency. Installing this package installs the guard. Importing it does not: only guard_adapter imports the guard, so a Door A consumer keeps working whatever state the dependency is in, and the doors themselves still take an injected gate. Until the guard is published to a package index, install it from its tag — see the README. A packaging test enforces that this stays the only runtime dependency.
• An extraction title containing [ or ] is rejected at manifest load (ingest-spec §4, ratified D1). Previously only single-line was validated. A manifest that loaded before and carries a bracket in a title now fails fast with code manifest_schema: the title renders verbatim into the index link label - [title](target), where a bracket breaks index-link and navigation parsing downstream.

Fixed

• Materializing one manifest no longer deletes the files another manifest stamped into the same bundle. The §3 ownership scan classified every ingest-stamped file as replaceable, so a second manifest sharing a bundle removed the first one's concept files and their index links. Ownership is now narrowed to files whose stamp names the running manifest by stem — the stem is stable across content edits, so an edited manifest still reclaims what a prior run of itself wrote. The operator-copy restriction (a generated file copied into curated content) remains documented, not enforced.

Notes

• Phase 2's binary extraction is not in this release: the [extract] extra is declared but empty, and pdf/docx/xlsx therefore fail fast. That is the one outstanding item from the phase, and it ships separately.
• The ownership change above is an extension point, not a spec fix. Filed under "Fixed" because it stops silent data loss, but the spec owner (portfolio-optimiser-commons) has since pointed out that removing every ingest-stamped file is verbatim what ingest-spec v1 §5 mandates: v1 assumes one manifest per bundle and defers multiple manifests feeding one bundle as a named extension point. So this release implements that extension point and outruns the frozen text rather than conforming to it. The mechanic itself is not at risk — matching by manifest stem follows from the spec's own {stem}@{h} stamp definition, since {h} changes on every content edit — but the surrounding prose is queued for amendment in commons and is not ratified. Treat multi-manifest bundles as ahead of the spec until it is.
• Exception __cause__ preservation is now pinned by a conformance suite, one test per fail-fast wrap site, alongside the existing .code suite.

0.3.2 — 2026-07-23

Fixed

• Frontmatter and index-label values are emitted verbatim; only source_query is whitespace-collapsed. Earlier releases collapsed every whitespace run in every frontmatter value and index link label to a single space. §5 of ingest-spec.md mandates that collapse for source_query alone — where a multi-line SQL SELECT must render on one line — while every other value is validated single-line at manifest load and passed through unchanged: validation, not repair. A title carrying an internal whitespace run now survives byte-for-byte at both the title frontmatter and the index link label, instead of being silently altered. Output bytes change only for values that contained a collapsible whitespace run; the shipped golden fixtures and both consumers are unaffected.

0.3.1 — 2026-07-19

Fixed

• Documentation corrected a security claim that did not hold. The module docstring and README stated that this library "calls the guard at every persist gate". That described the intended end state in the present tense. Door A — the only door shipped — is ungated: the package has zero runtime dependencies and calls no guard function before writing to disk. Both places now say so plainly, and state that gating external or untrusted content is the caller's responsibility (okf.import_bundle, or prepare_input/screen_output) until the persist gates land with Doors B and C.

  No behavior changed in this release. The correction is published because a consumer read the earlier wording as safe-by-default and would have persisted ungated content on that basis.

0.3.0 — 2026-07-17

Added

• Stable machine-readable error codes. IngestError gained a keyword-only code attribute (default "unspecified"). Roughly 24 codes are documented in the errors.py docstrings, and those docstrings are the stability contract. Consumers should assert on exc.value.code, not on message text.

Changed

• Exception message text is explicitly declared unstable. It may change in any release. Tests matching on message strings (pytest.raises(match=...)) should migrate to code comparisons.

Notes

• Generic schema shape violations deliberately share the single code manifest_schema. One code per validation rule would have frozen an unnecessarily large surface. Finer resolution is a separate decision, not an assumed requirement.

0.2.0 — 2026-07-17

Added

• PEP 561 support. The py.typed marker ships with the package, so consumers get the inline type hints without a mypy override.
• IngestResult.stamp exposes the spec §5 provenance stamp on the result object.

0.1.0 — 2026-07-16

Phase 1 (Door A) implemented against the normative ingest-spec.md owned by portfolio-optimiser-commons. Never tagged; consumers pinned the commit dae0bd1a directly.

Added

• Fail-fast manifest validation (spec §3–§4).
• Spec §5 body renderers as pure functions.
• The file connector (CSV, fail-closed path boundary), the sql connector (read-only sqlite, env-resolved credentials), and the http connector behind an explicit per-run network opt-in.
• Spec §5 materialization with an in-memory staging collision gate.
• Index maintenance on re-materialization (spec §6).
• The spec §11 golden fixtures, compared byte-for-byte.
• The Door A public surface: materialize_bundle plus the typed error hierarchy rooted in IngestError.