chore(release): v0.4.0

Phase 2 ships: Doors B and C, the extraction registry, the guard adapter,
and with them the first and only runtime dependency this library takes.
The minor bump is the news consumers are waiting on -- installing the
package now installs llm-ingestion-guard>=0.2,<0.3, though importing it
still does not import the guard.

Two behaviour changes carried in the same release are called out in the
changelog rather than buried: an extraction title containing `[` or `]`
now fails at manifest load (ingest-spec section 4), and the section 3
ownership scan no longer deletes files another manifest stamped into the
same bundle.

uv.lock was stale -- it predated the guard dependency and could not be
regenerated, because the guard is not on a package index. A
[tool.uv.sources] entry pins the same tag the README documents, so
`uv lock`/`uv sync` resolve. It is uv-specific dev metadata: the built
wheel still carries `Requires-Dist: llm-ingestion-guard<0.3,>=0.2`,
verified against the wheel, so the range remains the pin and the git URL
remains a channel.

Binary extraction (pdf/docx/xlsx behind [extract]) is deliberately not in
this release and is stated as outstanding in both README and changelog.

425 tests green; ruff, ruff format and mypy --strict clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2aKJxLejT9S8jYwoZ9fut
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 12:08:48 +02:00
commit 524ade78a6
5 changed files with 119 additions and 10 deletions

View file

@ -5,6 +5,94 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.4.0] — 2026-07-25
Phase 2. The bundle inbox (Door B) and external-bundle import (Door C) ship,
and with them this library's first — and only ever — runtime dependency.
### Added
- **Door B — bundle inbox (`process_inbox`).** Per dropped file: bytes →
extraction → persist gate → render → collision gate → write → index link.
Returns an `InboxResult` whose four buckets (`persisted`, `quarantined`,
`rejected`, `failed`) are disjoint and complete, so a file that vanished
mid-run surfaces as a missing entry rather than as nothing. One bad file
never aborts the run: only an invalid `ingested_at`, a reserved `okf_type`,
and a missing inbox directory fail the whole run, each being wrong for every
file at once. Concept files are named `inbox-{slug}.md`, disjoint from
`index.md`, Door A's `ingest-*`, and `promoted-verdict-*`; `source_sha256`
is taken over the original dropped bytes, so provenance stays re-verifiable
against the operator's file.
- **Door C — external bundle import (`import_bundle`).** Reads an external OKF
bundle and hands it *whole* to an injected gate over the guard's
`okf.import_bundle` — a bundle-level call, because it resolves the
cross-link graph across concepts — merging only concepts that clear the
non-blocking floor. Two invariants, both load-bearing: a merged concept is
written **verbatim** (stamping it would require round-tripping frontmatter
through this library's line-oriented parser, which cannot represent the
block lists the guard's parser accepts, and would persist bytes the guard
never screened), and ownership is therefore proven by **content identity**
an occupied target name is re-used only when the bytes there are already
identical, never overwritten. Re-importing an unchanged bundle is a no-op.
- **`extract_text` — the Door B extraction registry.** All file-type → text
extraction lives in this library, because the guard is text-only. `md`/`txt`
pass through, `csv` renders the markdown table, `json` is fenced verbatim,
and `html`/`htm` reduce to text with `html.parser` — stdlib throughout.
`pdf`, `docx`, and `xlsx` are gated behind the `[extract]` extra, which
ships no parser yet: those types fail fast with a typed error naming the
extra, never a silent skip.
- **`llm_ingestion_okf.guard_adapter`** — the shipped gate over the real
guard (`inbox_gate`, `import_gate`), and the only module here that imports
it. Door B screens the exact bytes it persists: the guard's `prepare_input`
bookend prepares text for a model call this library never makes, so
`screen_output` alone is used and the screened string is the written string.
It follows that the gate **refuses rather than repairs** — a file carrying
an invisible carrier is rejected, not stripped and persisted. The policy is
`PRESET_USER_UPLOAD`, so any finding at all is held back.
- **15 new stable error codes**, each registered in the `errors.py` docstrings
(the stability contract) and pinned by the error-code conformance suite:
`extractor_decode_error`, `extractor_empty_csv`, `extractor_extra_missing`,
`extractor_unknown`, `import_label_invalid`, `import_path_empty`,
`import_path_too_long`, `import_provenance_invalid`,
`import_slug_collision`, `inbox_slug_collision`, `inbox_slug_empty`,
`inbox_slug_too_long`, `inbox_source_file_invalid`, `inbox_title_invalid`,
`okf_type_reserved`.
### Changed
- **`llm-ingestion-guard>=0.2,<0.3` is now a mandatory runtime dependency.**
Installing this package installs the guard. Importing it does not: only
`guard_adapter` imports the guard, so a Door A consumer keeps working
whatever state the dependency is in, and the doors themselves still take an
*injected* gate. Until the guard is published to a package index, install it
from its tag — see the README. A packaging test enforces that this stays the
only runtime dependency.
- **An extraction `title` containing `[` or `]` is rejected at manifest load**
(ingest-spec §4, ratified D1). Previously only single-line was validated. A
manifest that loaded before and carries a bracket in a title now fails fast
with code `manifest_schema`: the title renders verbatim into the index link
label `- [title](target)`, where a bracket breaks index-link and navigation
parsing downstream.
### Fixed
- **Materializing one manifest no longer deletes the files another manifest
stamped into the same bundle.** The §3 ownership scan classified every
ingest-stamped file as replaceable, so a second manifest sharing a bundle
removed the first one's concept files and their index links. Ownership is
now narrowed to files whose stamp names the running manifest by stem — the
stem is stable across content edits, so an edited manifest still reclaims
what a prior run of itself wrote. The operator-copy restriction (a generated
file copied into curated content) remains documented, not enforced.
### Notes
- Phase 2's binary extraction is **not** in this release: the `[extract]`
extra is declared but empty, and `pdf`/`docx`/`xlsx` therefore fail fast.
That is the one outstanding item from the phase, and it ships separately.
- Exception `__cause__` preservation is now pinned by a conformance suite, one
test per fail-fast wrap site, alongside the existing `.code` suite.
## [0.3.2] — 2026-07-23
### Fixed
@ -88,6 +176,8 @@ Phase 1 (Door A) implemented against the normative `ingest-spec.md` owned by
- The Door A public surface: `materialize_bundle` plus the typed error hierarchy
rooted in `IngestError`.
[0.4.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.2...v0.4.0
[0.3.2]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.1...v0.3.2
[0.3.1]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.0...v0.3.1
[0.3.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.2.0...v0.3.0
[0.2.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/dae0bd1a...v0.2.0

View file

@ -2,13 +2,16 @@
Shared ingestion library for OKF (Open Knowledge Format) bundles.
Status: phase 1 (spec-based ingestion) is implemented — manifest validation,
the `file`/`sql`/`http` connectors, deterministic materialization, index
generation, and the golden fixture suite under `examples/`. Phase 2 is in
progress: the bundle inbox (`process_inbox`) and external-bundle import
(`import_bundle`) are implemented against an **injected** persist gate, and
`llm_ingestion_okf.guard_adapter` wires that gate to the real guard (see
below). Phases 34 are planned (see `docs/plan/`).
Status: phases 1 and 2 are implemented. Phase 1 (spec-based ingestion) covers
manifest validation, the `file`/`sql`/`http` connectors, deterministic
materialization, index generation, and the golden fixture suite under
`examples/`. Phase 2 adds the bundle inbox (`process_inbox`) and
external-bundle import (`import_bundle`), both against an **injected** persist
gate, with `llm_ingestion_okf.guard_adapter` wiring that gate to the real
guard (see below). One phase-2 item is deliberately outstanding: binary
extraction (`pdf`/`docx`/`xlsx` behind the `[extract]` extra) is unimplemented,
so those types are rejected fail-fast. Phases 34 are planned (see
`docs/plan/`).
## Planned scope (v1)

View file

@ -4,7 +4,7 @@ build-backend = "hatchling.build"
[project]
name = "llm-ingestion-okf"
version = "0.3.2"
version = "0.4.0"
description = "Shared OKF (Open Knowledge Format) ingestion library: spec-based connectors, bundle inbox, and external-bundle import, with security delegated to llm-ingestion-guard."
readme = "README.md"
license = "MIT"
@ -49,3 +49,10 @@ python_version = "3.10"
[[tool.mypy.overrides]]
module = ["llm_ingestion_guard", "llm_ingestion_guard.*"]
ignore_missing_imports = true
# Development-time install CHANNEL for the guard, which is not on a package
# index yet. It resolves `uv lock`/`uv sync` against the tag; it is uv-specific
# and never reaches consumers — the built wheel carries the range from
# [project.dependencies] above, which is the pin.
[tool.uv.sources]
llm-ingestion-guard = { git = "https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git", tag = "v0.2.0" }

View file

@ -73,7 +73,7 @@ from .manifest import (
)
from .materialize import IngestResult, materialize_bundle
__version__ = "0.3.2"
__version__ = "0.4.0"
__all__ = [
"BlockedFile",

11
uv.lock generated
View file

@ -164,10 +164,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/5f/5d/3dcec2884ba1b0806d1408612555c38dd5d68e90156b59f75f6e36435c3a/librt-0.13.0-cp314-cp314t-win_arm64.whl", hash = "sha256:2f281549a4c52ac7bb97997f14353f8bd0e53a34ca0dad1c905cfd0b4a58ae99", size = 110771, upload-time = "2026-07-08T12:26:12.303Z" },
]
[[package]]
name = "llm-ingestion-guard"
version = "0.2.0"
source = { git = "https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git?tag=v0.2.0#542ac92349cd0015972105ba66017b36cce1e708" }
[[package]]
name = "llm-ingestion-okf"
version = "0.3.2"
version = "0.4.0"
source = { editable = "." }
dependencies = [
{ name = "llm-ingestion-guard" },
]
[package.dev-dependencies]
dev = [
@ -177,6 +185,7 @@ dev = [
]
[package.metadata]
requires-dist = [{ name = "llm-ingestion-guard", git = "https://git.fromaitochitta.com/open/llm-ingestion-pipeline-security.git?tag=v0.2.0" }]
provides-extras = ["extract"]
[package.metadata.requires-dev]