Compare commits
2 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 692f2df2ba | |||
| 7fe073ac61 |
5 changed files with 104 additions and 8 deletions
75
CHANGELOG.md
75
CHANGELOG.md
|
|
@ -1,3 +1,78 @@
|
|||
# Changelog
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [0.3.1] — 2026-07-19
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Documentation corrected a security claim that did not hold.** The module
|
||||
docstring and README stated that this library "calls the guard at every persist
|
||||
gate". That described the intended end state in the present tense. Door A — the
|
||||
only door shipped — is ungated: the package has zero runtime dependencies and
|
||||
calls no guard function before writing to disk. Both places now say so plainly,
|
||||
and state that gating external or untrusted content is the caller's
|
||||
responsibility (`okf.import_bundle`, or `prepare_input`/`screen_output`) until
|
||||
the persist gates land with Doors B and C.
|
||||
|
||||
No behavior changed in this release. The correction is published because a
|
||||
consumer read the earlier wording as safe-by-default and would have persisted
|
||||
ungated content on that basis.
|
||||
|
||||
## [0.3.0] — 2026-07-17
|
||||
|
||||
### Added
|
||||
|
||||
- **Stable machine-readable error codes.** `IngestError` gained a keyword-only
|
||||
`code` attribute (default `"unspecified"`). Roughly 24 codes are documented in
|
||||
the `errors.py` docstrings, and those docstrings are the stability contract.
|
||||
Consumers should assert on `exc.value.code`, not on message text.
|
||||
|
||||
### Changed
|
||||
|
||||
- **Exception message text is explicitly declared unstable.** It may change in any
|
||||
release. Tests matching on message strings (`pytest.raises(match=...)`) should
|
||||
migrate to code comparisons.
|
||||
|
||||
### Notes
|
||||
|
||||
- Generic schema shape violations deliberately share the single code
|
||||
`manifest_schema`. One code per validation rule would have frozen an
|
||||
unnecessarily large surface. Finer resolution is a separate decision, not an
|
||||
assumed requirement.
|
||||
|
||||
## [0.2.0] — 2026-07-17
|
||||
|
||||
### Added
|
||||
|
||||
- **PEP 561 support.** The `py.typed` marker ships with the package, so consumers
|
||||
get the inline type hints without a mypy override.
|
||||
- **`IngestResult.stamp`** exposes the spec §5 provenance stamp on the result
|
||||
object.
|
||||
|
||||
## [0.1.0] — 2026-07-16
|
||||
|
||||
Phase 1 (Door A) implemented against the normative `ingest-spec.md` owned by
|
||||
`portfolio-optimiser-commons`. Never tagged; consumers pinned the commit
|
||||
`dae0bd1a` directly.
|
||||
|
||||
### Added
|
||||
|
||||
- Fail-fast manifest validation (spec §3–§4).
|
||||
- Spec §5 body renderers as pure functions.
|
||||
- The `file` connector (CSV, fail-closed path boundary), the `sql` connector
|
||||
(read-only sqlite, env-resolved credentials), and the `http` connector behind an
|
||||
explicit per-run network opt-in.
|
||||
- Spec §5 materialization with an in-memory staging collision gate.
|
||||
- Index maintenance on re-materialization (spec §6).
|
||||
- The spec §11 golden fixtures, compared byte-for-byte.
|
||||
- The Door A public surface: `materialize_bundle` plus the typed error hierarchy
|
||||
rooted in `IngestError`.
|
||||
|
||||
[0.3.1]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.0...v0.3.1
|
||||
[0.3.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.2.0...v0.3.0
|
||||
[0.2.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/dae0bd1a...v0.2.0
|
||||
[0.1.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/src/commit/dae0bd1a
|
||||
|
|
|
|||
19
README.md
19
README.md
|
|
@ -36,12 +36,25 @@ Security is owned by the sibling package
|
|||
- **guard** answers "is this content safe to persist?" — scan, sanitize,
|
||||
quarantine, fail-secure, provenance stamping.
|
||||
- **this library** does the plumbing — connect a source, materialize a
|
||||
deterministic OKF bundle, generate the index — and calls the guard at every
|
||||
persist gate (`prepare_input`/`screen_output` for extracted text,
|
||||
`okf.import_bundle` for received bundles).
|
||||
deterministic OKF bundle, generate the index.
|
||||
|
||||
No security functionality is reimplemented here.
|
||||
|
||||
### What is gated today: nothing
|
||||
|
||||
Door A — the only door shipped so far — is **ungated**. The package has zero
|
||||
runtime dependencies and calls no guard function before writing to disk.
|
||||
`materialize_bundle` writes what it is given.
|
||||
|
||||
If you materialize external or otherwise untrusted content, gating is **your**
|
||||
responsibility at the call site: `okf.import_bundle` for received bundles, or
|
||||
`prepare_input`/`screen_output` around extracted text. The guard-calling
|
||||
persist gates are part of Doors B and C (phase 2), not of Door A.
|
||||
|
||||
This is stated plainly because the earlier wording ("calls the guard at every
|
||||
persist gate") described the intended end state in the present tense, and a
|
||||
consumer reasonably read it as safe-by-default.
|
||||
|
||||
## Roadmap
|
||||
|
||||
The library is built in four phases so that every known OKF surface in the
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|||
|
||||
[project]
|
||||
name = "llm-ingestion-okf"
|
||||
version = "0.3.0"
|
||||
version = "0.3.1"
|
||||
description = "Shared OKF (Open Knowledge Format) ingestion library: spec-based connectors, bundle inbox, and external-bundle import, with security delegated to llm-ingestion-guard."
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
|
|
|
|||
|
|
@ -2,8 +2,16 @@
|
|||
|
||||
Three entry doors: spec-based ingestion (manifest -> connector ->
|
||||
deterministic materialization -> index), a bundle inbox converting common
|
||||
file types to OKF concepts, and import of external OKF bundles. Security is
|
||||
delegated to llm-ingestion-guard at every persist gate.
|
||||
file types to OKF concepts, and import of external OKF bundles.
|
||||
|
||||
Security is owned by llm-ingestion-guard, never reimplemented here. Note
|
||||
what that does and does not mean today: Door A is UNGATED. It has zero
|
||||
runtime dependencies and calls no guard function on the way to disk. A
|
||||
caller that materializes external or otherwise untrusted content is
|
||||
responsible for gating it -- via guard's okf.import_bundle for received
|
||||
bundles, or prepare_input/screen_output around extracted text. Do not read
|
||||
"security is delegated" as "safe by default": materialize_bundle writes
|
||||
what it is given. The guard-calling persist gates arrive with Doors B and C.
|
||||
|
||||
Door A (spec-based ingestion) public surface: materialize_bundle plus the
|
||||
typed error hierarchy rooted in IngestError.
|
||||
|
|
@ -27,7 +35,7 @@ from .manifest import (
|
|||
)
|
||||
from .materialize import IngestResult, materialize_bundle
|
||||
|
||||
__version__ = "0.3.0"
|
||||
__version__ = "0.3.1"
|
||||
|
||||
__all__ = [
|
||||
"Extraction",
|
||||
|
|
|
|||
2
uv.lock
generated
2
uv.lock
generated
|
|
@ -166,7 +166,7 @@ wheels = [
|
|||
|
||||
[[package]]
|
||||
name = "llm-ingestion-okf"
|
||||
version = "0.3.0"
|
||||
version = "0.3.1"
|
||||
source = { editable = "." }
|
||||
|
||||
[package.dev-dependencies]
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue