Compare commits
2 commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 692f2df2ba | |||
| 7fe073ac61 |
5 changed files with 104 additions and 8 deletions
75
CHANGELOG.md
75
CHANGELOG.md
|
|
@ -1,3 +1,78 @@
|
||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
All notable changes to this project will be documented in this file.
|
All notable changes to this project will be documented in this file.
|
||||||
|
|
||||||
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||||
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
|
## [0.3.1] — 2026-07-19
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **Documentation corrected a security claim that did not hold.** The module
|
||||||
|
docstring and README stated that this library "calls the guard at every persist
|
||||||
|
gate". That described the intended end state in the present tense. Door A — the
|
||||||
|
only door shipped — is ungated: the package has zero runtime dependencies and
|
||||||
|
calls no guard function before writing to disk. Both places now say so plainly,
|
||||||
|
and state that gating external or untrusted content is the caller's
|
||||||
|
responsibility (`okf.import_bundle`, or `prepare_input`/`screen_output`) until
|
||||||
|
the persist gates land with Doors B and C.
|
||||||
|
|
||||||
|
No behavior changed in this release. The correction is published because a
|
||||||
|
consumer read the earlier wording as safe-by-default and would have persisted
|
||||||
|
ungated content on that basis.
|
||||||
|
|
||||||
|
## [0.3.0] — 2026-07-17
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Stable machine-readable error codes.** `IngestError` gained a keyword-only
|
||||||
|
`code` attribute (default `"unspecified"`). Roughly 24 codes are documented in
|
||||||
|
the `errors.py` docstrings, and those docstrings are the stability contract.
|
||||||
|
Consumers should assert on `exc.value.code`, not on message text.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
- **Exception message text is explicitly declared unstable.** It may change in any
|
||||||
|
release. Tests matching on message strings (`pytest.raises(match=...)`) should
|
||||||
|
migrate to code comparisons.
|
||||||
|
|
||||||
|
### Notes
|
||||||
|
|
||||||
|
- Generic schema shape violations deliberately share the single code
|
||||||
|
`manifest_schema`. One code per validation rule would have frozen an
|
||||||
|
unnecessarily large surface. Finer resolution is a separate decision, not an
|
||||||
|
assumed requirement.
|
||||||
|
|
||||||
|
## [0.2.0] — 2026-07-17
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **PEP 561 support.** The `py.typed` marker ships with the package, so consumers
|
||||||
|
get the inline type hints without a mypy override.
|
||||||
|
- **`IngestResult.stamp`** exposes the spec §5 provenance stamp on the result
|
||||||
|
object.
|
||||||
|
|
||||||
|
## [0.1.0] — 2026-07-16
|
||||||
|
|
||||||
|
Phase 1 (Door A) implemented against the normative `ingest-spec.md` owned by
|
||||||
|
`portfolio-optimiser-commons`. Never tagged; consumers pinned the commit
|
||||||
|
`dae0bd1a` directly.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- Fail-fast manifest validation (spec §3–§4).
|
||||||
|
- Spec §5 body renderers as pure functions.
|
||||||
|
- The `file` connector (CSV, fail-closed path boundary), the `sql` connector
|
||||||
|
(read-only sqlite, env-resolved credentials), and the `http` connector behind an
|
||||||
|
explicit per-run network opt-in.
|
||||||
|
- Spec §5 materialization with an in-memory staging collision gate.
|
||||||
|
- Index maintenance on re-materialization (spec §6).
|
||||||
|
- The spec §11 golden fixtures, compared byte-for-byte.
|
||||||
|
- The Door A public surface: `materialize_bundle` plus the typed error hierarchy
|
||||||
|
rooted in `IngestError`.
|
||||||
|
|
||||||
|
[0.3.1]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.3.0...v0.3.1
|
||||||
|
[0.3.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/v0.2.0...v0.3.0
|
||||||
|
[0.2.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/compare/dae0bd1a...v0.2.0
|
||||||
|
[0.1.0]: https://git.fromaitochitta.com/open/llm-ingestion-okf/src/commit/dae0bd1a
|
||||||
|
|
|
||||||
19
README.md
19
README.md
|
|
@ -36,12 +36,25 @@ Security is owned by the sibling package
|
||||||
- **guard** answers "is this content safe to persist?" — scan, sanitize,
|
- **guard** answers "is this content safe to persist?" — scan, sanitize,
|
||||||
quarantine, fail-secure, provenance stamping.
|
quarantine, fail-secure, provenance stamping.
|
||||||
- **this library** does the plumbing — connect a source, materialize a
|
- **this library** does the plumbing — connect a source, materialize a
|
||||||
deterministic OKF bundle, generate the index — and calls the guard at every
|
deterministic OKF bundle, generate the index.
|
||||||
persist gate (`prepare_input`/`screen_output` for extracted text,
|
|
||||||
`okf.import_bundle` for received bundles).
|
|
||||||
|
|
||||||
No security functionality is reimplemented here.
|
No security functionality is reimplemented here.
|
||||||
|
|
||||||
|
### What is gated today: nothing
|
||||||
|
|
||||||
|
Door A — the only door shipped so far — is **ungated**. The package has zero
|
||||||
|
runtime dependencies and calls no guard function before writing to disk.
|
||||||
|
`materialize_bundle` writes what it is given.
|
||||||
|
|
||||||
|
If you materialize external or otherwise untrusted content, gating is **your**
|
||||||
|
responsibility at the call site: `okf.import_bundle` for received bundles, or
|
||||||
|
`prepare_input`/`screen_output` around extracted text. The guard-calling
|
||||||
|
persist gates are part of Doors B and C (phase 2), not of Door A.
|
||||||
|
|
||||||
|
This is stated plainly because the earlier wording ("calls the guard at every
|
||||||
|
persist gate") described the intended end state in the present tense, and a
|
||||||
|
consumer reasonably read it as safe-by-default.
|
||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
The library is built in four phases so that every known OKF surface in the
|
The library is built in four phases so that every known OKF surface in the
|
||||||
|
|
|
||||||
|
|
@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
||||||
|
|
||||||
[project]
|
[project]
|
||||||
name = "llm-ingestion-okf"
|
name = "llm-ingestion-okf"
|
||||||
version = "0.3.0"
|
version = "0.3.1"
|
||||||
description = "Shared OKF (Open Knowledge Format) ingestion library: spec-based connectors, bundle inbox, and external-bundle import, with security delegated to llm-ingestion-guard."
|
description = "Shared OKF (Open Knowledge Format) ingestion library: spec-based connectors, bundle inbox, and external-bundle import, with security delegated to llm-ingestion-guard."
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
license = "MIT"
|
license = "MIT"
|
||||||
|
|
|
||||||
|
|
@ -2,8 +2,16 @@
|
||||||
|
|
||||||
Three entry doors: spec-based ingestion (manifest -> connector ->
|
Three entry doors: spec-based ingestion (manifest -> connector ->
|
||||||
deterministic materialization -> index), a bundle inbox converting common
|
deterministic materialization -> index), a bundle inbox converting common
|
||||||
file types to OKF concepts, and import of external OKF bundles. Security is
|
file types to OKF concepts, and import of external OKF bundles.
|
||||||
delegated to llm-ingestion-guard at every persist gate.
|
|
||||||
|
Security is owned by llm-ingestion-guard, never reimplemented here. Note
|
||||||
|
what that does and does not mean today: Door A is UNGATED. It has zero
|
||||||
|
runtime dependencies and calls no guard function on the way to disk. A
|
||||||
|
caller that materializes external or otherwise untrusted content is
|
||||||
|
responsible for gating it -- via guard's okf.import_bundle for received
|
||||||
|
bundles, or prepare_input/screen_output around extracted text. Do not read
|
||||||
|
"security is delegated" as "safe by default": materialize_bundle writes
|
||||||
|
what it is given. The guard-calling persist gates arrive with Doors B and C.
|
||||||
|
|
||||||
Door A (spec-based ingestion) public surface: materialize_bundle plus the
|
Door A (spec-based ingestion) public surface: materialize_bundle plus the
|
||||||
typed error hierarchy rooted in IngestError.
|
typed error hierarchy rooted in IngestError.
|
||||||
|
|
@ -27,7 +35,7 @@ from .manifest import (
|
||||||
)
|
)
|
||||||
from .materialize import IngestResult, materialize_bundle
|
from .materialize import IngestResult, materialize_bundle
|
||||||
|
|
||||||
__version__ = "0.3.0"
|
__version__ = "0.3.1"
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"Extraction",
|
"Extraction",
|
||||||
|
|
|
||||||
2
uv.lock
generated
2
uv.lock
generated
|
|
@ -166,7 +166,7 @@ wheels = [
|
||||||
|
|
||||||
[[package]]
|
[[package]]
|
||||||
name = "llm-ingestion-okf"
|
name = "llm-ingestion-okf"
|
||||||
version = "0.3.0"
|
version = "0.3.1"
|
||||||
source = { editable = "." }
|
source = { editable = "." }
|
||||||
|
|
||||||
[package.dev-dependencies]
|
[package.dev-dependencies]
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue