From 696f19af1ee683576dc6afdf2c2f72fb427686b1 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Sat, 4 Jul 2026 17:17:12 +0200 Subject: [PATCH] docs(i6): http source extension-point + D7 create_sdk_mcp_server pointer --- docs/extending.md | 48 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/docs/extending.md b/docs/extending.md index 1f23f75..b63496d 100644 --- a/docs/extending.md +++ b/docs/extending.md @@ -52,6 +52,54 @@ Model choice is **config, not code** (B12): `src/portfolio_optimiser/data/model_ The role keys (`proposer`, `checker`, `default`) let you assign a distinct model per debate role; `default` is the fallback when a role is unmapped. +## Legg til en ingest-kilde (http-familien som worked example) + +The **ingest layer** (`ingest.py`, spec `shared/ingest-spec.md`) is a second, distinct source +seam from the retriever above: one JSON **manifest** per source coupling declares a `source.type` +and a list of extractions; `materialize(...)` runs the connector for that type and writes an OKF +bundle. The spec ships three source families — `file`/CSV (I2), `sql` (I4), and **`http`** (I6) — +and `http` is the framework's **worked extension-point example**: it shows exactly how a third, +network-transport family plugs into the same connector / materialization / gate contracts. + +The `http` manifest contract (spec §4): + +- `source.base_url` — the endpoint root; it **must not embed credentials** (userinfo is rejected + at schema validation). +- `source.credential_ref` — the **name of an environment variable** holding the secret, resolved + at run time (sent as `Authorization: Bearer `); the secret is never read from the + manifest, never logged, never stamped in the generated frontmatter. `null` → no auth. +- each extraction's `query` is joined onto `base_url` and its response body is rendered + **verbatim inside a fenced code block** (not a markdown table — a raw `|` or `\` survives + un-escaped); `max_rows` caps the response line count, fail-fast (never silent truncation). + +**Network is a per-run grant, never a manifest field (spec §8).** `materialize` refuses an `http` +source unless the caller passes `allow_network=True`: + +```python +# refused fail-fast — the manifest cannot grant itself network access: +materialize(manifest, bundle_dir, ingested_at="2026-07-04T12:00:00Z") +# opted in explicitly by the operator for this run: +materialize(manifest, bundle_dir, ingested_at="2026-07-04T12:00:00Z", allow_network=True) +``` + +This is the **local-only default, no silent egress** principle made mechanical: configuration is +data that cannot escalate its own authority; only the runtime `allow_network` grant can. The +transport itself is an **injectable seam** — `materialize(..., http_get=)` swaps the GET +implementation (the default `_urllib_get` is the only socket path). The golden case +(`examples/ingest-golden-http/`) and every test inject a canned `get` over committed fixture +payloads, so the suite runs offline against a **local mock** — no live source, no credentials. + +### D7 sibling hook — MCP as an extension of this family + +The spec (§4) documents an **MCP-based connector as an extension of the `http` family**, not a new +client wired into the optimiser run path — that stays a **Non-Goal** here: the in-process +`FunctionTool` seam is the default in the run path, and MCP is demonstrated (via `build_mcp_server` +in `datasource.py`), not wired in. On the Claude Agent SDK sibling (D7), the in-process server +hook is `create_sdk_mcp_server(name, version="1.0.0", tools=...) -> McpSdkServerConfig` (package +`claude-agent-sdk`) — verified 2026-07-04 against the official Claude Agent SDK Python docs. A +deployer who wants a network- or MCP-mediated source extends this family behind the same explicit, +per-run network grant; nothing here contacts a live endpoint. + ## Bevisst ikke bygget (90 %-kuttlista) Per the design philosophy (a ~90 % generic core with clear extension points — we do not chase