feat: initial commit — repo scaffold and v1 scope
Scope settled 2026-07-16: implements portfolio-optimiser-commons ingest-spec (commons keeps spec authorship), Python 3.10+ stdlib-only core, security delegated to llm-ingestion-guard at persist gates. Doors: spec-based ingestion, bundle inbox (md/txt/csv/json/html core, binary formats behind [extract]), external bundle import. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QeqhJpYQyghASjiJo5EhGg
This commit is contained in:
commit
47df0aeb4b
7 changed files with 268 additions and 0 deletions
113
CLAUDE.md
Normal file
113
CLAUDE.md
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
# llm-ingestion-okf
|
||||
|
||||
## Context
|
||||
|
||||
Shared OKF (Open Knowledge Format) ingestion library. Three entry doors,
|
||||
one boundary rule:
|
||||
|
||||
- **Door A — spec-based ingestion:** implements the normative
|
||||
`ingest-spec.md` owned by `portfolio-optimiser-commons` (manifest →
|
||||
`file`/`sql`/`http` connector → deterministic materialization of
|
||||
`ingest-{id}.md` → index generation; zero model calls). This repo
|
||||
IMPLEMENTS the spec; commons keeps authorship. Spec changes the library
|
||||
needs go via commons, never edited locally. The library ships the §11
|
||||
golden fixtures (byte-exact) that are missing today.
|
||||
- **Door B — bundle inbox:** converts dropped files to OKF concepts. All
|
||||
file-type→text extraction lives HERE (the guard is text-only). v1 core:
|
||||
`md`, `txt`, `csv`, `json`, `html` (stdlib). `pdf`/`docx`/`xlsx` only via
|
||||
the optional `[extract]` extra; without it those types are rejected
|
||||
fail-fast.
|
||||
- **Door C — external bundle import:** third-party OKF bundles are assessed
|
||||
per concept via the guard's `okf.import_bundle`; only non-rejected concepts
|
||||
are merged/materialized/indexed here.
|
||||
|
||||
**Boundary rule (non-negotiable, zero overlap):** `llm-ingestion-guard`
|
||||
(pinned `>=0.2,<0.3`) answers "is this content safe to persist?" —
|
||||
scan/sanitize/quarantine/fail-secure/provenance-stamp. This library is
|
||||
plumbing: connect source → materialize deterministic OKF bundle → generate
|
||||
index. Never reimplement security; call the guard at persist gates
|
||||
(`prepare_input`/`screen_output`, `okf.import_bundle`). When in doubt which
|
||||
side of the boundary something belongs on: ask the operator.
|
||||
|
||||
**Implementation baseline:** the stricter behaviors from
|
||||
`portfolio-optimiser` (streaming row caps, utf-8-sig, in-memory staging with
|
||||
pre-mutation collision gate, validated `ingested_at`, typed `IngestError`)
|
||||
are the library baseline. First consumer: `portfolio-optimiser-claude`.
|
||||
|
||||
### Open crossroads: the Node second-brain world (decision deferred)
|
||||
|
||||
The OKF second-brain plugins (okr, linkedin-studio, ms-ai-architect, and the
|
||||
marketplace catalog spec) are deliberately Node/ESM zero-dependency and
|
||||
cannot consume a Python library directly. okr has documented it does not
|
||||
want a Python bridge. v1 therefore aligns with the catalog's
|
||||
`okf-second-brain/spec.md` as a *contract* only and ships no Node code.
|
||||
Revisit when any of these become true:
|
||||
|
||||
1. A second Node consumer needs shared ingestion *code*, not just the
|
||||
catalog conventions (`okf-check.mjs`/`okf-index.mjs`).
|
||||
2. okr's doc-conversion layer (docx/pdf/eml/html → md) gets built and
|
||||
duplicates Door B's extraction logic.
|
||||
3. The catalog's convention-level alignment proves insufficient to keep
|
||||
bundle forms interoperable.
|
||||
|
||||
### Non-goals (v1)
|
||||
|
||||
- `strict-v1` wiki bundle profile (`claude-code-llm-wiki`) — v2 candidate;
|
||||
requires a configurable bundle contract (types/layers/frontmatter sets/
|
||||
reserved-file policy as config, not constants).
|
||||
- Verdict/feedback machinery (method-spec) — stays in consumer repos.
|
||||
- Embedding/RAG/retrieval layers.
|
||||
|
||||
## Stack
|
||||
|
||||
Python 3.10+. Package `llm_ingestion_okf` (src layout, hatchling).
|
||||
**Zero runtime dependencies in the core** (stdlib only, like the guard).
|
||||
The only permitted future runtime dependency is `llm-ingestion-guard`
|
||||
(itself zero-dep), added when the first persist gate is implemented.
|
||||
Binary extraction parsers live behind the `[extract]` extra only.
|
||||
|
||||
## Conventions
|
||||
|
||||
- Type hints everywhere; `mypy --strict` target.
|
||||
- Determinism is bit-exact: `ingested_at` is an explicit required argument
|
||||
(no wall-clock defaults); LF-only output; golden fixtures compared
|
||||
byte-for-byte.
|
||||
- No model calls anywhere in the run path.
|
||||
- Credentials only as env-var *references* resolved at runtime; never in
|
||||
manifests, logs, or frontmatter.
|
||||
- Network access requires an explicit per-run opt-in flag; refuse fail-fast
|
||||
otherwise.
|
||||
- Conventional Commits: `type(scope): description`.
|
||||
- English for all code, docs, and commit messages (public repo).
|
||||
|
||||
## Commands
|
||||
|
||||
- Test: `pytest`
|
||||
- Lint: `ruff check .` + `ruff format --check .`
|
||||
- Type check: `mypy --strict src/`
|
||||
|
||||
## Workflow
|
||||
|
||||
- TDD: no production code without a failing test first.
|
||||
- This repo is published PUBLICLY (`open/` namespace on Forgejo). `STATE.md`
|
||||
and `docs/oppstartsprompt.md` are LOCAL-ONLY (gitignored) — never commit
|
||||
session state or internal briefs. No secrets, sober English prose, no
|
||||
marketing language.
|
||||
- After `git commit`: push to Forgejo (`git push origin`) immediately.
|
||||
Never GitHub.
|
||||
|
||||
## Communication patterns
|
||||
|
||||
### Linking to local files
|
||||
|
||||
When pointing to local files in responses, always use markdown link syntax
|
||||
with a descriptive name:
|
||||
|
||||
- Use `[Human-friendly name](file:///absolute/path)` — never bare
|
||||
`file:///...` URLs or autolinks `<file://...>`.
|
||||
- Always use absolute paths. Never `~/` or relative paths.
|
||||
- For multiple files, render as a bullet list of named markdown links.
|
||||
|
||||
Why: bare `file://` URLs only render the first as clickable across multiple
|
||||
lines. Named markdown links make each entry independently clickable and look
|
||||
cleaner.
|
||||
Loading…
Add table
Add a link
Reference in a new issue