Commit graph

2 commits

Author SHA1 Message Date
b33ea00055 chore(deps): move the ingest library pin to v0.3.2 — as far as latest goes today
The pin had sat at v0.3.1 with STATE calling the hold "deliberate" and
recording no reason. Measured: no coord message ever announced v0.4.0 or
v0.5.0a* to this repo, so the hold was drift wearing a decision's clothes.

v0.3.2 is a pure fix (frontmatter and index labels emit verbatim; only
source_query is whitespace-collapsed, per ingest-spec §5), keeps
`dependencies = []`, and is green here: 668 passed.

WHY NOT FURTHER, both measured rather than assumed:

1. v0.4.0 introduces a REGRESSION that breaks our §6 removal path.
   Bisected v0.3.2 OK / v0.4.0 RED with a minimal repro: materialize a
   bundle, then re-materialize it with a CHANGED manifest, and the library
   no longer recognises its own stamp —

     MaterializationError: generated filename 'ingest-costs.md' collides
     with an existing file that does not carry the ingest stamp

   The stamp carries the manifest's name+hash (`ingest_manifest: m2@…`), so
   editing a manifest makes every file it previously wrote look curated.
   Re-ingesting the SAME manifest is fine, which is why fixtures miss it.
   It is `tests/test_ingest_loadbearing.py::test_reingest_with_active_
   removal_preserves_promoted_and_curated` that catches it. Reported
   upstream; not ours to fix.

2. Everything past v0.3.1 adds `llm-ingestion-guard>=0.2,<0.3` as a HARD
   runtime dependency (v0.3.1/v0.3.2: `dependencies = []`). That flips two
   documented invariants here — pyproject's "zero runtime deps" comment and
   the STATE marker line the guard repo reads machine-readably ("not a
   runtime dependency today"). An operator decision, not a version bump.

3. v0.5.0a2 is an alpha whose own CHANGELOG scopes it to a named pilot set
   — portfolio-optimiser-claude, the marketplace catalog, claude-code-llm-wiki
   — and says "do not pin this tag outside the pilot set", with the v0.2
   surface free to change without a deprecation cycle. This repo is not a
   pilot. Joining is llm-ingestion-okf's call, requested via coord.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GyAbxJoyypnLLUDcMvnKh8
2026-08-05 12:11:12 +02:00
ddd6338f02 feat(ingest): add the MCP connector as a transport inside the http family (S2.2)
Commons settled on 2026-08-01 that MCP is an extension of the `http` source
family, not a fourth family (`shared/ingest-spec.md` §4). This implements it
with ZERO schema change and zero spec amendment.

The transport discriminator lives in `base_url`, not in new manifest fields:
the shared library rejects unknown manifest keys fail-fast, and we consume it
pull-only at a pinned v0.3.1, so `server_ref`/`tool` as fields would have meant
a spec amendment plus a library release. It buys nothing — the library already
joins `base_url` + `/` + `query`, so `mcp+stdio://<server_ref>` + `<tool>`
reproduces exactly the two-part structure the (now stale) reference plan wanted.

Staying inside the family INHERITS what a fourth family would have had to write
and could have forgotten: the §8 network grant (measured to fire before any tool
call), the `max_rows` cap, §5 verbatim fenced rendering, and the §7 provenance
stamp. The discriminator gates rather than labels — `mcp_get` refuses a URL it
does not own, so an MCP transport can never quietly serve an `https://` manifest
and leave the bundle's provenance claiming a transport that was never used.

Parsing is string-based, not `urlsplit`-based: `urlsplit().hostname` lowercases
the host, which would silently break the case-sensitive env lookup `server_ref`
depends on.

`ingest.py` is untouched — it is AST-guarded mcp-free, so the transport lives in
its own module and is opt-in at the call site. `ingest_mcp.py` imports the open
`mcp` protocol client but never `agent_framework`, keeping the seam D7-portable.

Load-bearing, six mutations all measured RED: detach the scheme guard · make the
refusal unconditional · swap parsing to `urlsplit().hostname` · skip non-text
content instead of raising · force `allow_network=True` · smuggle in a MAF
import. Both source files restored byte-identical (`shasum -c`) after each.

Honesty: `stdio_call_tool` (the real stdio path) is written but never executed
end to end — every test injects a canned tool call, so the suite spawns no
subprocess and opens no socket. No golden fixture, and MCP stays unwired in the
optimiser run path. Stated in docs/extending.md rather than implied away.

555 -> 578 tests; ruff + mypy green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0112FPR5TX6pDLiNicBPzE8i
2026-08-02 21:14:57 +02:00