--- type: trekresearch-brief created: 2026-06-24 question: "What is the best way to expose a local document folder to a MAF 1.9.0 agent via MCPStdioTool such that retrieved content carries citation metadata (file + locator) — existing filesystem MCP server vs thin custom server, and the citation shape a context provider expects?" confidence: 0.82 dimensions: 6 mcp_servers_used: [microsoft-learn, tavily] local_agents_used: [installed-source-introspection (orchestrator, main context)] external_agents_used: [docs-researcher, community-researcher, security-researcher, contrarian-researcher, gemini-bridge (unavailable)] topic: 2 brief: .claude/projects/2026-06-24-fase2-mvp-vertical-slice/brief.md --- # Local-folder data access via MCP with citation provenance (MAF 1.9.0) > Generated by trekresearch (Voyage 5.6.0) on 2026-06-24. Topic 2 of 3 for the > Fase 2 MVP-vertical-slice brief. Scope: external swarm + installed-source > introspection (installed 1.9.0 API truth wins over Learn docs). Gemini > unavailable (MCP SDK broke on Google's May-2026 API change). ## Research Question What is the best way to expose a LOCAL document folder to a MAF 1.9.0 agent via `MCPStdioTool` so retrieved content carries citation metadata (file + locator) — reuse an existing filesystem MCP server, or build a thin custom server — and what citation shape does MAF's context/annotation model expect? Decision it feeds: the data-access step, the citation-aware context provider, provenance-stamping, and the build-vs-reuse decision for the local-folder server. ## Executive Summary **The brief's lean ("reuse the official `@modelcontextprotocol/server-filesystem` + attach citations via `parse_tool_results`") does not survive contact with the requirement and should be dropped.** The official filesystem server returns raw text and bare path strings only — **no line/char offsets, no chunk IDs; its `search_files` is filename-glob, not content search** — so it physically cannot yield a `(file + locator + snippet)` citation; you would have to re-read, re-chunk and re-locate inside the callback (i.e. build the whole citation engine anyway, in the worst place, behind a Node subprocess). MAF also does **not** auto-create citation annotations from MCP results. **Recommendation (honouring the CLAUDE.md "data access via MCP, JSON-Schema-validated, fail-fast" convention): build a THIN CUSTOM local-folder MCP server that returns citation-ready structured chunks (`{file, locator, snippet, score}`), wrapping a framework-agnostic in-process retriever core that is the D7-portable seam.** Carry provenance as first-class Pydantic data on the emitted proposal, independent of MAF's annotation propagation (which has an open Python streaming-drop bug, #4316). Confidence 0.82: the capability facts are HIGH (installed-source + official-README verified); the exact MVP retrieval depth (keyword vs local embeddings) is a deferrable design choice. ## Dimensions ### 1. Can the official filesystem server carry citations? — Confidence: high (NO) **External findings (docs + community + contrarian agree):** - `@modelcontextprotocol/server-filesystem` tools: `read_text_file` (whole file, or `head`/`tail` by line *count* only), `read_multiple_files`, `list_directory`, `directory_tree`, `get_file_info`, `search_files`, `write_file`, `edit_file`. - Read results = **raw UTF-8 text, no line numbers, no byte/char offsets, no chunk IDs.** `search_files` returns **matching file PATHS only** (filename glob — NOT content grep). `get_file_info` = size/timestamps, not a locator. - Net: the only locator it gives is the file path you already passed in. Enhanced community forks advertise `read_file_lines` / grep-with-line-numbers / chunking precisely because the official server lacks them. **Conclusion:** the official server cannot produce `(file + locator + snippet)` citations. It is fine only for "let the agent open named files," not citation-backed retrieval. ### 2. Does MAF auto-create citations from MCP results? — Confidence: high (NO) **Installed-source findings (1.9.0):** - Default parser `_parse_contents_from_mcp_tool_result` / `_parse_tool_result_from_mcp` maps MCP content → `Content.from_text/from_data/from_uri`; **attaches no annotations** (`agent_framework/_mcp.py:507-517`). - `structuredContent` IS now serialized to text Content (`_mcp.py:586-587`) — so the #3313 "structuredContent dropped → None" bug **is fixed in installed 1.9.0**. But it lands as JSON *text*, not as typed/annotated content — you still parse it yourself for provenance. - The seam to attach citations is `MCPTool.__init__(parse_tool_results: Callable[[CallToolResult], str | list[Content]])` — your callback fully replaces the default parse. **External findings:** - Confirmed: docs/RAG paths use **prompt-level** "cite your sources" (a `string_mapper`), not structured annotations. The one citation-aware RAG provider, `TextSearchProvider`, is **.NET-only — no Python equivalent**. - **Open bug #4316:** file-citation annotations are **silently dropped in the Python streaming path** (`annotations=None`); .NET fixed, Python lagging. → do NOT rely on MAF's `Annotation` propagation for load-bearing provenance. ### 3. Citation shape MAF expects — Confidence: high **Installed-source findings (1.9.0, ground truth):** - `Annotation` is a **TypedDict** (`agent_framework/_types.py:374`): `type: Literal["citation"]`, `title`, `url`, `file_id`, `tool_name`, `snippet`, `annotated_regions: Sequence[TextSpanRegion]`, `additional_properties`, `raw_representation`. - `TextSpanRegion` TypedDict (`_types.py:366`): `type: Literal["text_span"]`, `start_index`, `end_index`. - The class form `CitationAnnotation` was replaced by this TypedDict in 1.0.0b260123 (PR #3252) — installed 1.9.0 = **TypedDict** form. Build citations as dicts: `{"type":"citation","file_id":...,"title":...,"snippet":...,"annotated_regions":[{"start_index":..,"end_index":..}]}`. - Carrier: `FunctionResultContent.annotations: list[... ] | None` (accepts dict/TypedDict). **Use:** our provenance stamp's "≥1 citation" maps cleanly onto this `Annotation`. But because of #4316, treat `Annotation` as a *display/serialization* surface and keep the authoritative provenance as our own Pydantic object. ### 4. Context provider injection seam — Confidence: high (corrects an external error) **Installed-source findings (1.9.0) — and a flagged docs-vs-source conflict:** - `class ContextProvider` (`agent_framework/_sessions.py:351`), `__init__(self, source_id: str)` — **source_id REQUIRED** (confirms Fase 1). - Hooks: `async def before_run(...)` (`:370`) and `async def after_run(...)` (`:391`) — add messages/instructions/tools in `before_run`, process/store in `after_run`. - `Context.extend_instructions(self, source_id: str, instructions: str | Sequence[str])` (`:253`) and `extend_tools(source_id, tools)` (`:266`) — **the two-arg seam the brief and Fase 1 specified, confirmed verbatim.** - `MemoryContextProvider` / `InMemoryHistoryProvider` exist but are conversation memory, **not** retrievers. > **Premiss correction (important for /trekplan):** the docs-researcher reported, with high confidence from the Learn API-ref page, that Python uses `invoking`/`invoked` hooks and that `source_id` / `extend_instructions` are "C#-only, not in Python." **That is wrong for installed 1.9.0** — the installed source has `before_run`/`after_run`, `ContextProvider(source_id)`, and `extend_instructions(source_id, …)`. Per CLAUDE.md, installed source wins. Do NOT let the planning phase adopt the `invoking`/`invoked` / no-source_id surface. **Design fit:** a custom `ExpeLContextProvider(ContextProvider)` (Fase 1 seam, promote to core) injects retrieved+cited content + prior verdicts in `before_run` via `extend_instructions(source_id, …)`, and the same provider family is where the ExpeL learning loop reads VerdictStore. No built-in local-folder retrieval provider exists in Python — retrieval is ours to build. ### 5. Build vs reuse (the decision) — Confidence: high (recommendation) **Triangulated picture:** - **Reuse official server:** cannot cite (Dim 1); adds a Node runtime + `npx` cold-start + version-pin burden; npx PATH/nvm handshake failure is THE most-reported stdio blocker on macOS (B5); supply-chain surface (chalk/debug + Shai-Hulud hit MCP-SDK transitives). Security: usable only if pinned ≥ `2025.7.1` (EscapeRoute CVE-2025-53109/53110, fixed there; current `2026.1.14` clean), vendored (never `npx @latest`), allowlist scoped to the doc folder (never `$HOME`). - **Thin custom MCP server:** removes the npm supply-chain surface, emits provenance natively (`{file, line_range/char_span, snippet, score}` via `structuredContent`), honours the CLAUDE.md "data access via MCP, JSON-Schema-validated, fail-fast" convention, and is portable to the D7 Claude-SDK sibling's in-process-MCP idiom. Cost: **we own** path-canonicalisation + symlink-realpath validation (the exact bug class the official server had fixed twice — TDD against the EscapeRoute scenarios) and input validation (no shelling out; use native file APIs). - **In-process retriever, no MCP** (contrarian's first choice): simplest + lowest latency + best D7 fit, but conflicts with the project's "data access via MCP" convention. Keep as the fallback if that convention is relaxed. - Reference design: `shinpr/mcp-local-rag` (LanceDB file-based + local embeddings; returns `path + chunk index + title + score + chunk text` + `read_chunk_neighbors` context expansion) — the closest local-only, citation-grade pattern. ### 6. Retrieval depth for the MVP — Confidence: medium - The MVP uses a **tiny synthetic "anleggskostnad" domain** (Fase 0). A full vector stack is not required for D5 (90%) / D6 (cost). A simple in-process retriever (keyword/substring + chunk-at-ingest with exact locators) likely suffices and keeps deps minimal. - The Python "real RAG" path (SK `VectorStore.create_search_function().as_agent_framework_tool()`) **requires pulling in `semantic-kernel ≥ 1.38`** — a different package with its own RCE CVE (CVE-2026-26030, eval in in-memory vector-store filter, < 1.39.4). Adding it conflicts with the "GA-pakker pinnet eksplisitt, ikke metaen" dependency discipline and adds CVE surface. **Defer / avoid for MVP** unless semantic retrieval proves necessary; if adopted, pin ≥ 1.39.4. ## External Knowledge ### Best Practice - Canonical `MCPStdioTool` wiring (official docstring + Learn): `MCPStdioTool(name="filesystem", command="npx", args=["-y","@modelcontextprotocol/server-filesystem", ])`, used as `async with`; requires `pip install mcp --pre`. - MAF can also expose an *agent* as an MCP server (`agent.as_mcp_server()`) — not relevant to local-folder retrieval. Generic server-building is the MCP Python SDK / FastMCP domain. ### Security (D3 no-silent-egress + local-only) - Stdio MCP server makes **no network calls**; only inherent egress in the path is the `npx` fetch at launch (removed by vendoring/pinning). OTel `_meta` trace-context injection into `tools/call` is inert with instrumentation off and carries only random W3C IDs (no project data). - EscapeRoute CVEs (CVE-2025-53110 prefix-collision, CVE-2025-53109 symlink-escape) — both **fixed in 2025.7.1**; exploit trigger is prompt injection of the agent (relevant: our maker-checker + blocking validator are compensating controls). Never run the server with elevated privileges. - npx supply chain: pin exact version + lockfile, verify tarball for `postinstall`, quarantine fresh versions, consider containerization. A thin custom dependency-light server removes this surface entirely. ### Known Issues - MAF MCP bugs — **verified against installed 1.9.0**: #3313 structuredContent parsing **FIXED** (`_mcp.py:586`); #2884 stdio session-invalidation handled (`is_connected` reset + `ClosedResourceError` catch, `_mcp.py:935/1355/1544`). Still: #2284 `_meta` dropped (don't rely on `_meta` for provenance — use `structuredContent`/`content`); #4316 Python streaming citation-annotation drop (open). - npx/nvm PATH handshake failure on macOS (servers#64) — use absolute `command` path + explicit `env` PATH/NODE_PATH, prefer pre-installed/`uvx`-pinned over cold `npx -y`. Filesystem server fails to start if any allowed dir is unavailable (servers#3232). - Don't split a citation across `content` vs `structuredContent` vs `_meta` — clients (MAF included) forward them inconsistently; put it in one reliably-forwarded channel. ## Gemini Second Opinion Unavailable (same MCP-SDK/Interactions-API failure as Topic 1). No independent Gemini triangulation for this topic; treat as absent, not negative. ## Synthesis Three sources converge on a conclusion the brief did not anticipate: **the citation requirement, not the data-access requirement, drives the design — and it rules out the off-the-shelf filesystem server.** Citations of the form `(file + locator + snippet)` must be *exact by construction*, which means chunk-and-locate **at ingest**, in code we own. Bolting them on afterward — over a third-party server's lossy whole-file text, through a `parse_tool_results` callback, relying on a MAF annotation path with an open streaming-drop bug — stacks three fragilities to recreate something we could have produced cleanly up front. The right shape is a **two-layer seam**: (1) a framework-agnostic in-process retriever core — `retrieve(query) -> list[RetrievedChunk]`, where `RetrievedChunk` is a Pydantic type carrying `file`, `locator` (line range / char span), `snippet`, `score` — which is the **D7-portable contract** (MAF and Claude Agent SDK both satisfy it); and (2) a **thin custom MCP server** wrapping that core, returning the chunks as structured `structuredContent`, which honours the project's "data access via MCP, JSON-Schema-validated, fail-fast" convention and ports to the Claude SDK's in-process-MCP idiom. Provenance lives as first-class Pydantic data on the emitted proposal; the MAF `Annotation` TypedDict is only a display/serialisation view of it, so #4316 never sits on the load-bearing path. The official filesystem server is demoted to a possible *future* "browse named files" convenience, pinned and scoped. This also threads the CLAUDE.md needle: the contrarian's "skip MCP, in-process only" is the simplest and the best D7 fit, but it breaks the MCP-data-access convention; the thin-custom-MCP-server-over-an-in-process-core keeps the convention *and* the D7 portability *and* citation fidelity — at the cost of owning sandbox correctness (testable, bounded). ## Open Questions - **MVP retrieval depth:** keyword/substring + ingest-time chunking (no extra deps, D5/D6-aligned) vs local embeddings (LanceDB/SK VectorStore, heavier + CVE surface). Default to keyword for the tiny synthetic domain; revisit only if recall is poor. Resolve in /trekplan. - **Locator format:** line range vs char span. It is the shared D7 contract — define and test it in the golden-suite up front. Default: char span (`start_index`/ `end_index`) to map straight onto `TextSpanRegion`; carry line numbers in `additional_properties` for human display. - **Does the MVP even need MCP, or is the in-process retriever enough?** The CLAUDE.md convention says data access via MCP. The thin-custom-server honours it; confirm the operator wants the MCP boundary in Fase 2 vs deferring it (the in-process core is needed either way). Surface in /trekplan. - **Chunk-at-ingest vs at-query:** ingest-time chunking gives exact, cacheable locators; decide whether the MVP ingests once or reads live. Tie to determinism NFR. ## Recommendation **Build a thin custom local-folder MCP server that returns citation-ready structured chunks, wrapping a framework-agnostic in-process retriever core. Do NOT reuse `@modelcontextprotocol/server-filesystem` for citation-backed retrieval.** 1. **Retriever core (D7 seam):** `retrieve(query, top_k) -> list[RetrievedChunk]`; `RetrievedChunk(file: str, locator: TextSpan, snippet: str, score: float)`. Chunk + assign locators at ingest so citations are exact by construction. Start with keyword/substring retrieval (no heavy deps; D5/D6). Pure Python, no MAF/MCP imports — portable to the Claude SDK sibling. 2. **Thin custom MCP server (MAF-facing, honours the convention):** a minimal `mcp`/FastMCP stdio server exposing a `search(query)` tool that returns the chunks via `structuredContent` (JSON-Schema-validated, fail-fast on bad config). Path handling is security-critical: canonicalise + boundary-check (no `startsWith`), resolve symlink realpath and fail closed, native file APIs (no shelling out) — TDD against the EscapeRoute bypass patterns. 3. **Wire into MAF:** `MCPStdioTool(name="docfolder", command=, args=[...])` with a `parse_tool_results` callback that maps `structuredContent` chunks into `FunctionResultContent` + `Annotation(type="citation", file_id, snippet, annotated_regions=[TextSpanRegion])`. On Intel mac, pass absolute `command` + explicit `env` PATH; pre-install the server (no cold `npx`/uvx fetch on the path). 4. **Provenance = first-class Pydantic data** on the emitted `ValidatedProposal` (≥1 citation: file + locator + snippet, plus model/role + validator decision + token usage). The MAF `Annotation` TypedDict is only its display view — route around #4316 by never depending on MAF annotation propagation for the assertion. 5. **Citation-aware context injection:** a custom `ExpeLContextProvider(ContextProvider)` (`source_id` required) uses `before_run` + `extend_instructions(source_id, …)` to inject cited content (and prior verdicts) — the confirmed two-arg seam. 6. **If the official filesystem server is ever used** (future "browse named files"): pin ≥ `2025.7.1` vendored (never `npx @latest`), allowlist the doc folder only (never `$HOME`), absolute node path + explicit env. Fallback (only if the MCP-data-access convention is relaxed for the MVP): skip the MCP server and call the in-process retriever directly via a `@tool`/`ContextProvider` — simplest, lowest-latency, best D7 fit. Risks to carry into the plan: (a) we own MCP-server sandbox correctness — treat path/symlink validation as security-critical, TDD it; (b) MAF Python citation annotation propagation is buggy in streaming (#4316) — provenance must be our own data; (c) adding `semantic-kernel` for vector RAG pulls a CVE-bearing package — defer. ## Sources | # | Source | Type | Quality | Used in | |---|--------|------|---------|---------| | 1 | `.venv/.../agent_framework/_mcp.py:507-517,586-587,655` (default parser, structuredContent) | codebase | high | Dim 2, Known Issues | | 2 | `.venv/.../agent_framework/_mcp.py:911,935,1330,1355,1544` (session lifecycle, ClosedResourceError) | codebase | high | Known Issues | | 3 | `.venv/.../agent_framework/_types.py:366,374` (`TextSpanRegion`, `Annotation` TypedDict) | codebase | high | Dim 3 | | 4 | `.venv/.../agent_framework/_sessions.py:351,370,391,253,266` (`ContextProvider`, `before_run`/`after_run`, `extend_instructions`) | codebase | high | Dim 4 | | 5 | (tool surface, no locators) | official | high | Dim 1 | | 6 | | official | high | Dim 1,5 | | 7 | | official | high | Dim 5, Best Practice | | 8 | | official | high | Best Practice | | 9 | (TextSearchProvider .NET-only; SK VectorStore path) | official | high | Dim 2, 6 | | 10 | (Annotation TypedDict PR #3252) | official | high | Dim 3 | | 11 | (Python streaming citation drop, OPEN) | community | high | Dim 2, Synthesis | | 12 | (structuredContent dropped, fixed) | community | high | Dim 2, Known Issues | | 13 | (stdio session not invalidated, fixed #3154) | community | high | Known Issues | | 14 | (`_meta` discarded) | community | medium | Known Issues | | 15 | (npx/nvm PATH handshake) | community | high | Dim 5, Known Issues | | 16 | (startup fails on unavailable dir) | community | medium | Known Issues | | 17 | (EscapeRoute CVEs) | community | high | Dim 5, Security | | 18 | (CVE-2025-53109) | official | high | Security | | 19 | (CVE-2025-53110) | official | high | Security | | 20 | (no advisories > 2025.7.1) | community | high | Security | | 21 | (chalk/debug, Shai-Hulud) | community | high | Dim 5, Security | | 22 | (local-only citation-grade RAG reference) | community | medium | Dim 5 | | 23 | (Claude SDK in-process MCP idiom) | official | high | Dim 5, Synthesis | | 24 | (hosted FileSearch cites service-side) | official | medium | Dim 2 | | 25 | (MCP token bloat) | community | medium | Known Issues |