docs(ingest): I7 — programavslutning, D7-docs avgrenset til CSV + SQL

Docs-only program-avslutning for ingest-laget (D7-stacken). Bygget fra
commons-spec + eget repo alene — MAF-koden kun eksistens-bekreftet (I6-gate),
aldri lest/reverse-engineert.

- README: ingest-seksjon (file/CSV + sql/sqlite RO) som post-S10-tillegg;
  testtall 187 -> 265; HTTP/MCP kun peker (D7 har ingen HTTP-konnektor).
- docs/extending.md (ny): hvordan ingest-laget virker + hvordan legge til en
  kildetype; HTTP/MCP som ærlig extension-point-peker (spec §4, MAF I6-demo,
  create_sdk_mcp_server som ubygd vehikkel — brukt ingen steder i src/).
- docs/2026-07-04-I7-brief.md + -statusrapport.md: I7-brief + D7-lokal
  statusrapport m/ verifiseringslogg. Kryss-stack-kriteriet peker til
  MAF-programrapporten.

Ærlighetsregelen (method-spec §1): ingen artefakt påstår HTTP/MCP-støtte i D7
eller live-kilde-integrasjon. Grep-sjekk mot overpåstand ren.

Verifisering: 265 passed uten nøkkel/nettverk · ruff/mypy rene.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R7nuUMUyWB9viiBjz8Bo7R
This commit is contained in:
Kjell Tore Guttormsen 2026-07-04 20:08:06 +02:00
commit 0c42cebf90
4 changed files with 301 additions and 6 deletions

View file

@ -6,11 +6,12 @@ project in a portfolio of independent projects: agents generate candidate measur
mandatory deterministic validator gates the numbers, domain experts judge via
human-in-the-loop, and the system learns from the verdicts.
> **Status:** the D7 build (S5S10) is complete. The deterministic backbone, the agentic
> loop, and the learning loop are wired seam by seam, each proven by load-bearing tests
> (187 tests, all running offline without an API key). The programme's single budgeted
> **live model run has been executed and validated** — its artifacts are committed under
> [`runs/s10/`](runs/s10/) (see below).
> **Status:** the D7 build (S5S10) is complete, and the deterministic **ingest layer**
> (CSV and SQL source types) has since been added in front of the loop. The deterministic
> backbone, the agentic loop, the learning loop, and the ingest connectors are wired seam by
> seam, each proven by load-bearing tests (265 tests, all running offline without an API
> key). The programme's single budgeted **live model run has been executed and validated**
> its artifacts are committed under [`runs/s10/`](runs/s10/) (see below).
> **Disclaimer — technical framework only.** The deployer owns DPIA, risk assessment, and
> the legal basis for any processing. The framework ships only the technical
@ -87,6 +88,36 @@ empty-inbox control), `test_step8_promotion_loadbearing.py` (the gate refuses no
verdicts; the promoted signal stays out of the read-context), and
`test_sdk_isolation.py` (local config cannot capture the checker).
## The ingest layer — CSV and SQL, in front of the loop
The method spec forbids query-time retrieval against the bundle (§3 Step 1), so **data
reaches the model only via OKF bundles.** The ingest layer is the deterministic step that
satisfies that: a connector reads a real source, and the extract is materialized as an OKF
bundle the existing 8-step loop then consumes **unchanged**. It makes **zero model calls**,
touches no network, and `ingest.py` imports nothing from the SDK — it is pure standard
library. Built from the shared [`ingest-spec.md`](shared/ingest-spec.md) alone.
D7 implements the two conformance-required source types:
- **`file`** — a local CSV catalogue; extraction paths are boundary-checked fail-closed
against the source `root` (the OKF path rule).
- **`sql`** — a local SQLite database, opened **read-only** (`mode=ro`), one SELECT per
extraction; the connection location is resolved at run time from a named environment
variable (`connection_ref`), never stored in the manifest.
Both are frozen by byte-identical golden extractions
([`examples/ingest-golden-file/`](examples/ingest-golden-file/),
[`examples/ingest-golden-sql/`](examples/ingest-golden-sql/)) and by the load-bearing seam
tests (provenance stamping, navigability through the unchanged `okf.py`, the reserved
verdict layer, and re-ingest safety over a promoted verdict).
**Honesty rule (§1):** the `http` source type is an **optional extension point, not built in
D7** — a manifest naming it is rejected fail-fast at validation
(`test_malformed_manifest_is_rejected`), never silently accepted. The HTTP/MCP extension
point is demonstrated only in the MAF sibling (against a local mock, behind an opt-in
network flag); this repo ships no network connector and no live-source integration. How the
layer works and how one would extend it is documented in [`docs/extending.md`](docs/extending.md).
## The live run — S10, executed and validated
Where the MAF sibling proves its loop end-to-end with a scripted offline simulation, this
@ -115,7 +146,7 @@ Python ≥3.10 · [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk
```bash
uv sync # install dependencies
uv run pytest # 187 tests — run without any API key and without network
uv run pytest # 265 tests — run without any API key and without network
uv run ruff check . && uv run ruff format --check .
uv run mypy src # strict
```