portfolio-optimiser/.claude/projects/2026-06-24-fase2-mvp-vertical-slice/research/02-local-folder-mcp-citation-provenance.md

---
type: trekresearch-brief
created: 2026-06-24
question: "What is the best way to expose a local document folder to a MAF 1.9.0 agent via MCPStdioTool such that retrieved content carries citation metadata (file + locator) — existing filesystem MCP server vs thin custom server, and the citation shape a context provider expects?"
confidence: 0.82
dimensions: 6
mcp_servers_used: [microsoft-learn, tavily]
local_agents_used: [installed-source-introspection (orchestrator, main context)]
external_agents_used: [docs-researcher, community-researcher, security-researcher, contrarian-researcher, gemini-bridge (unavailable)]
topic: 2
brief: .claude/projects/2026-06-24-fase2-mvp-vertical-slice/brief.md
---

# Local-folder data access via MCP with citation provenance (MAF 1.9.0)

> Generated by trekresearch (Voyage 5.6.0) on 2026-06-24. Topic 2 of 3 for the
> Fase 2 MVP-vertical-slice brief. Scope: external swarm + installed-source
> introspection (installed 1.9.0 API truth wins over Learn docs). Gemini
> unavailable (MCP SDK broke on Google's May-2026 API change).

## Research Question

What is the best way to expose a LOCAL document folder to a MAF 1.9.0 agent via
`MCPStdioTool` so retrieved content carries citation metadata (file + locator) —
reuse an existing filesystem MCP server, or build a thin custom server — and what
citation shape does MAF's context/annotation model expect? Decision it feeds: the
data-access step, the citation-aware context provider, provenance-stamping, and the
build-vs-reuse decision for the local-folder server.

## Executive Summary

**The brief's lean ("reuse the official `@modelcontextprotocol/server-filesystem`
+ attach citations via `parse_tool_results`") does not survive contact with the
requirement and should be dropped.** The official filesystem server returns raw
text and bare path strings only — **no line/char offsets, no chunk IDs; its
`search_files` is filename-glob, not content search** — so it physically cannot
yield a `(file + locator + snippet)` citation; you would have to re-read, re-chunk
and re-locate inside the callback (i.e. build the whole citation engine anyway, in
the worst place, behind a Node subprocess). MAF also does **not** auto-create
citation annotations from MCP results. **Recommendation (honouring the CLAUDE.md
"data access via MCP, JSON-Schema-validated, fail-fast" convention): build a THIN
CUSTOM local-folder MCP server that returns citation-ready structured chunks
(`{file, locator, snippet, score}`), wrapping a framework-agnostic in-process
retriever core that is the D7-portable seam.** Carry provenance as first-class
Pydantic data on the emitted proposal, independent of MAF's annotation propagation
(which has an open Python streaming-drop bug, #4316). Confidence 0.82: the
capability facts are HIGH (installed-source + official-README verified); the
exact MVP retrieval depth (keyword vs local embeddings) is a deferrable design
choice.

## Dimensions

### 1. Can the official filesystem server carry citations? — Confidence: high (NO)

**External findings (docs + community + contrarian agree):**
- `@modelcontextprotocol/server-filesystem` tools: `read_text_file` (whole file, or `head`/`tail` by line *count* only), `read_multiple_files`, `list_directory`, `directory_tree`, `get_file_info`, `search_files`, `write_file`, `edit_file`. <https://github.com/modelcontextprotocol/servers/blob/main/src/filesystem/README.md>
- Read results = **raw UTF-8 text, no line numbers, no byte/char offsets, no chunk IDs.** `search_files` returns **matching file PATHS only** (filename glob — NOT content grep). `get_file_info` = size/timestamps, not a locator.
- Net: the only locator it gives is the file path you already passed in. Enhanced community forks advertise `read_file_lines` / grep-with-line-numbers / chunking precisely because the official server lacks them.

**Conclusion:** the official server cannot produce `(file + locator + snippet)` citations. It is fine only for "let the agent open named files," not citation-backed retrieval.

### 2. Does MAF auto-create citations from MCP results? — Confidence: high (NO)

**Installed-source findings (1.9.0):**
- Default parser `_parse_contents_from_mcp_tool_result` / `_parse_tool_result_from_mcp` maps MCP content → `Content.from_text/from_data/from_uri`; **attaches no annotations** (`agent_framework/_mcp.py:507-517`).
- `structuredContent` IS now serialized to text Content (`_mcp.py:586-587`) — so the #3313 "structuredContent dropped → None" bug **is fixed in installed 1.9.0**. But it lands as JSON *text*, not as typed/annotated content — you still parse it yourself for provenance.
- The seam to attach citations is `MCPTool.__init__(parse_tool_results: Callable[[CallToolResult], str | list[Content]])` — your callback fully replaces the default parse.

**External findings:**
- Confirmed: docs/RAG paths use **prompt-level** "cite your sources" (a `string_mapper`), not structured annotations. The one citation-aware RAG provider, `TextSearchProvider`, is **.NET-only — no Python equivalent**. <https://learn.microsoft.com/agent-framework/agents/rag>
- **Open bug #4316:** file-citation annotations are **silently dropped in the Python streaming path** (`annotations=None`); .NET fixed, Python lagging. → do NOT rely on MAF's `Annotation` propagation for load-bearing provenance. <https://github.com/microsoft/agent-framework/issues/4316>

### 3. Citation shape MAF expects — Confidence: high

**Installed-source findings (1.9.0, ground truth):**
- `Annotation` is a **TypedDict** (`agent_framework/_types.py:374`): `type: Literal["citation"]`, `title`, `url`, `file_id`, `tool_name`, `snippet`, `annotated_regions: Sequence[TextSpanRegion]`, `additional_properties`, `raw_representation`.
- `TextSpanRegion` TypedDict (`_types.py:366`): `type: Literal["text_span"]`, `start_index`, `end_index`.
- The class form `CitationAnnotation` was replaced by this TypedDict in 1.0.0b260123 (PR #3252) — installed 1.9.0 = **TypedDict** form. Build citations as dicts: `{"type":"citation","file_id":...,"title":...,"snippet":...,"annotated_regions":[{"start_index":..,"end_index":..}]}`.
- Carrier: `FunctionResultContent.annotations: list[... ] | None` (accepts dict/TypedDict).

**Use:** our provenance stamp's "≥1 citation" maps cleanly onto this `Annotation`. But because of #4316, treat `Annotation` as a *display/serialization* surface and keep the authoritative provenance as our own Pydantic object.

### 4. Context provider injection seam — Confidence: high (corrects an external error)

**Installed-source findings (1.9.0) — and a flagged docs-vs-source conflict:**
- `class ContextProvider` (`agent_framework/_sessions.py:351`), `__init__(self, source_id: str)` — **source_id REQUIRED** (confirms Fase 1).
- Hooks: `async def before_run(...)` (`:370`) and `async def after_run(...)` (`:391`) — add messages/instructions/tools in `before_run`, process/store in `after_run`.
- `Context.extend_instructions(self, source_id: str, instructions: str | Sequence[str])` (`:253`) and `extend_tools(source_id, tools)` (`:266`) — **the two-arg seam the brief and Fase 1 specified, confirmed verbatim.**
- `MemoryContextProvider` / `InMemoryHistoryProvider` exist but are conversation memory, **not** retrievers.

> **Premiss correction (important for /trekplan):** the docs-researcher reported, with high confidence from the Learn API-ref page, that Python uses `invoking`/`invoked` hooks and that `source_id` / `extend_instructions` are "C#-only, not in Python." **That is wrong for installed 1.9.0** — the installed source has `before_run`/`after_run`, `ContextProvider(source_id)`, and `extend_instructions(source_id, …)`. Per CLAUDE.md, installed source wins. Do NOT let the planning phase adopt the `invoking`/`invoked` / no-source_id surface.

**Design fit:** a custom `ExpeLContextProvider(ContextProvider)` (Fase 1 seam, promote to core) injects retrieved+cited content + prior verdicts in `before_run` via `extend_instructions(source_id, …)`, and the same provider family is where the ExpeL learning loop reads VerdictStore. No built-in local-folder retrieval provider exists in Python — retrieval is ours to build.

### 5. Build vs reuse (the decision) — Confidence: high (recommendation)

**Triangulated picture:**
- **Reuse official server:** cannot cite (Dim 1); adds a Node runtime + `npx` cold-start + version-pin burden; npx PATH/nvm handshake failure is THE most-reported stdio blocker on macOS (B5); supply-chain surface (chalk/debug + Shai-Hulud hit MCP-SDK transitives). Security: usable only if pinned ≥ `2025.7.1` (EscapeRoute CVE-2025-53109/53110, fixed there; current `2026.1.14` clean), vendored (never `npx @latest`), allowlist scoped to the doc folder (never `$HOME`).
- **Thin custom MCP server:** removes the npm supply-chain surface, emits provenance natively (`{file, line_range/char_span, snippet, score}` via `structuredContent`), honours the CLAUDE.md "data access via MCP, JSON-Schema-validated, fail-fast" convention, and is portable to the D7 Claude-SDK sibling's in-process-MCP idiom. Cost: **we own** path-canonicalisation + symlink-realpath validation (the exact bug class the official server had fixed twice — TDD against the EscapeRoute scenarios) and input validation (no shelling out; use native file APIs).
- **In-process retriever, no MCP** (contrarian's first choice): simplest + lowest latency + best D7 fit, but conflicts with the project's "data access via MCP" convention. Keep as the fallback if that convention is relaxed.
- Reference design: `shinpr/mcp-local-rag` (LanceDB file-based + local embeddings; returns `path + chunk index + title + score + chunk text` + `read_chunk_neighbors` context expansion) — the closest local-only, citation-grade pattern. <https://github.com/shinpr/mcp-local-rag>

### 6. Retrieval depth for the MVP — Confidence: medium

- The MVP uses a **tiny synthetic "anleggskostnad" domain** (Fase 0). A full vector stack is not required for D5 (90%) / D6 (cost). A simple in-process retriever (keyword/substring + chunk-at-ingest with exact locators) likely suffices and keeps deps minimal.
- The Python "real RAG" path (SK `VectorStore.create_search_function().as_agent_framework_tool()`) **requires pulling in `semantic-kernel ≥ 1.38`** — a different package with its own RCE CVE (CVE-2026-26030, eval in in-memory vector-store filter, < 1.39.4). Adding it conflicts with the "GA-pakker pinnet eksplisitt, ikke metaen" dependency discipline and adds CVE surface. **Defer / avoid for MVP** unless semantic retrieval proves necessary; if adopted, pin ≥ 1.39.4.

## External Knowledge

### Best Practice
- Canonical `MCPStdioTool` wiring (official docstring + Learn): `MCPStdioTool(name="filesystem", command="npx", args=["-y","@modelcontextprotocol/server-filesystem", <dir>])`, used as `async with`; requires `pip install mcp --pre`. <https://learn.microsoft.com/agent-framework/agents/tools/local-mcp-tools>
- MAF can also expose an *agent* as an MCP server (`agent.as_mcp_server()`) — not relevant to local-folder retrieval. Generic server-building is the MCP Python SDK / FastMCP domain. <https://github.com/modelcontextprotocol/python-sdk>

### Security (D3 no-silent-egress + local-only)
- Stdio MCP server makes **no network calls**; only inherent egress in the path is the `npx` fetch at launch (removed by vendoring/pinning). OTel `_meta` trace-context injection into `tools/call` is inert with instrumentation off and carries only random W3C IDs (no project data).
- EscapeRoute CVEs (CVE-2025-53110 prefix-collision, CVE-2025-53109 symlink-escape) — both **fixed in 2025.7.1**; exploit trigger is prompt injection of the agent (relevant: our maker-checker + blocking validator are compensating controls). Never run the server with elevated privileges. <https://cymulate.com/blog/cve-2025-53109-53110-escaperoute-anthropic/>
- npx supply chain: pin exact version + lockfile, verify tarball for `postinstall`, quarantine fresh versions, consider containerization. A thin custom dependency-light server removes this surface entirely.

### Known Issues
- MAF MCP bugs — **verified against installed 1.9.0**: #3313 structuredContent parsing **FIXED** (`_mcp.py:586`); #2884 stdio session-invalidation handled (`is_connected` reset + `ClosedResourceError` catch, `_mcp.py:935/1355/1544`). Still: #2284 `_meta` dropped (don't rely on `_meta` for provenance — use `structuredContent`/`content`); #4316 Python streaming citation-annotation drop (open).
- npx/nvm PATH handshake failure on macOS (servers#64) — use absolute `command` path + explicit `env` PATH/NODE_PATH, prefer pre-installed/`uvx`-pinned over cold `npx -y`. Filesystem server fails to start if any allowed dir is unavailable (servers#3232).
- Don't split a citation across `content` vs `structuredContent` vs `_meta` — clients (MAF included) forward them inconsistently; put it in one reliably-forwarded channel.

## Gemini Second Opinion

Unavailable (same MCP-SDK/Interactions-API failure as Topic 1). No independent
Gemini triangulation for this topic; treat as absent, not negative.

## Synthesis

Three sources converge on a conclusion the brief did not anticipate: **the
citation requirement, not the data-access requirement, drives the design — and it
rules out the off-the-shelf filesystem server.** Citations of the form
`(file + locator + snippet)` must be *exact by construction*, which means
chunk-and-locate **at ingest**, in code we own. Bolting them on afterward — over a
third-party server's lossy whole-file text, through a `parse_tool_results` callback,
relying on a MAF annotation path with an open streaming-drop bug — stacks three
fragilities to recreate something we could have produced cleanly up front.

The right shape is a **two-layer seam**: (1) a framework-agnostic in-process
retriever core — `retrieve(query) -> list[RetrievedChunk]`, where `RetrievedChunk`
is a Pydantic type carrying `file`, `locator` (line range / char span), `snippet`,
`score` — which is the **D7-portable contract** (MAF and Claude Agent SDK both
satisfy it); and (2) a **thin custom MCP server** wrapping that core, returning the
chunks as structured `structuredContent`, which honours the project's "data access
via MCP, JSON-Schema-validated, fail-fast" convention and ports to the Claude SDK's
in-process-MCP idiom. Provenance lives as first-class Pydantic data on the emitted
proposal; the MAF `Annotation` TypedDict is only a display/serialisation view of it,
so #4316 never sits on the load-bearing path. The official filesystem server is
demoted to a possible *future* "browse named files" convenience, pinned and scoped.

This also threads the CLAUDE.md needle: the contrarian's "skip MCP, in-process
only" is the simplest and the best D7 fit, but it breaks the MCP-data-access
convention; the thin-custom-MCP-server-over-an-in-process-core keeps the convention
*and* the D7 portability *and* citation fidelity — at the cost of owning sandbox
correctness (testable, bounded).

## Open Questions

- **MVP retrieval depth:** keyword/substring + ingest-time chunking (no extra deps,
  D5/D6-aligned) vs local embeddings (LanceDB/SK VectorStore, heavier + CVE surface).
  Default to keyword for the tiny synthetic domain; revisit only if recall is poor.
  Resolve in /trekplan.
- **Locator format:** line range vs char span. It is the shared D7 contract — define
  and test it in the golden-suite up front. Default: char span (`start_index`/
  `end_index`) to map straight onto `TextSpanRegion`; carry line numbers in
  `additional_properties` for human display.
- **Does the MVP even need MCP, or is the in-process retriever enough?** The CLAUDE.md
  convention says data access via MCP. The thin-custom-server honours it; confirm the
  operator wants the MCP boundary in Fase 2 vs deferring it (the in-process core is
  needed either way). Surface in /trekplan.
- **Chunk-at-ingest vs at-query:** ingest-time chunking gives exact, cacheable
  locators; decide whether the MVP ingests once or reads live. Tie to determinism NFR.

## Recommendation

**Build a thin custom local-folder MCP server that returns citation-ready
structured chunks, wrapping a framework-agnostic in-process retriever core. Do NOT
reuse `@modelcontextprotocol/server-filesystem` for citation-backed retrieval.**

1. **Retriever core (D7 seam):** `retrieve(query, top_k) -> list[RetrievedChunk]`;
   `RetrievedChunk(file: str, locator: TextSpan, snippet: str, score: float)`.
   Chunk + assign locators at ingest so citations are exact by construction.
   Start with keyword/substring retrieval (no heavy deps; D5/D6). Pure Python,
   no MAF/MCP imports — portable to the Claude SDK sibling.
2. **Thin custom MCP server (MAF-facing, honours the convention):** a minimal
   `mcp`/FastMCP stdio server exposing a `search(query)` tool that returns the
   chunks via `structuredContent` (JSON-Schema-validated, fail-fast on bad config).
   Path handling is security-critical: canonicalise + boundary-check (no
   `startsWith`), resolve symlink realpath and fail closed, native file APIs (no
   shelling out) — TDD against the EscapeRoute bypass patterns.
3. **Wire into MAF:** `MCPStdioTool(name="docfolder", command=<abs python>, args=[...])`
   with a `parse_tool_results` callback that maps `structuredContent` chunks into
   `FunctionResultContent` + `Annotation(type="citation", file_id, snippet,
   annotated_regions=[TextSpanRegion])`. On Intel mac, pass absolute `command` +
   explicit `env` PATH; pre-install the server (no cold `npx`/uvx fetch on the path).
4. **Provenance = first-class Pydantic data** on the emitted `ValidatedProposal`
   (≥1 citation: file + locator + snippet, plus model/role + validator decision +
   token usage). The MAF `Annotation` TypedDict is only its display view — route
   around #4316 by never depending on MAF annotation propagation for the assertion.
5. **Citation-aware context injection:** a custom `ExpeLContextProvider(ContextProvider)`
   (`source_id` required) uses `before_run` + `extend_instructions(source_id, …)` to
   inject cited content (and prior verdicts) — the confirmed two-arg seam.
6. **If the official filesystem server is ever used** (future "browse named files"):
   pin ≥ `2025.7.1` vendored (never `npx @latest`), allowlist the doc folder only
   (never `$HOME`), absolute node path + explicit env.

Fallback (only if the MCP-data-access convention is relaxed for the MVP): skip the
MCP server and call the in-process retriever directly via a `@tool`/`ContextProvider`
— simplest, lowest-latency, best D7 fit.

Risks to carry into the plan: (a) we own MCP-server sandbox correctness — treat
path/symlink validation as security-critical, TDD it; (b) MAF Python citation
annotation propagation is buggy in streaming (#4316) — provenance must be our own
data; (c) adding `semantic-kernel` for vector RAG pulls a CVE-bearing package — defer.

## Sources

| # | Source | Type | Quality | Used in |
|---|--------|------|---------|---------|
| 1 | `.venv/.../agent_framework/_mcp.py:507-517,586-587,655` (default parser, structuredContent) | codebase | high | Dim 2, Known Issues |
| 2 | `.venv/.../agent_framework/_mcp.py:911,935,1330,1355,1544` (session lifecycle, ClosedResourceError) | codebase | high | Known Issues |
| 3 | `.venv/.../agent_framework/_types.py:366,374` (`TextSpanRegion`, `Annotation` TypedDict) | codebase | high | Dim 3 |
| 4 | `.venv/.../agent_framework/_sessions.py:351,370,391,253,266` (`ContextProvider`, `before_run`/`after_run`, `extend_instructions`) | codebase | high | Dim 4 |
| 5 | <https://github.com/modelcontextprotocol/servers/blob/main/src/filesystem/README.md> (tool surface, no locators) | official | high | Dim 1 |
| 6 | <https://www.npmjs.com/package/@modelcontextprotocol/server-filesystem> | official | high | Dim 1,5 |
| 7 | <https://learn.microsoft.com/python/api/agent-framework-core/agent_framework.mcpstdiotool?view=agent-framework-python-latest> | official | high | Dim 5, Best Practice |
| 8 | <https://learn.microsoft.com/agent-framework/agents/tools/local-mcp-tools> | official | high | Best Practice |
| 9 | <https://learn.microsoft.com/agent-framework/agents/rag> (TextSearchProvider .NET-only; SK VectorStore path) | official | high | Dim 2, 6 |
| 10 | <https://learn.microsoft.com/agent-framework/support/upgrade/python-2026-significant-changes> (Annotation TypedDict PR #3252) | official | high | Dim 3 |
| 11 | <https://github.com/microsoft/agent-framework/issues/4316> (Python streaming citation drop, OPEN) | community | high | Dim 2, Synthesis |
| 12 | <https://github.com/microsoft/agent-framework/issues/3313> (structuredContent dropped, fixed) | community | high | Dim 2, Known Issues |
| 13 | <https://github.com/microsoft/agent-framework/issues/2884> (stdio session not invalidated, fixed #3154) | community | high | Known Issues |
| 14 | <https://github.com/microsoft/agent-framework/issues/2284> (`_meta` discarded) | community | medium | Known Issues |
| 15 | <https://github.com/modelcontextprotocol/servers/issues/64> (npx/nvm PATH handshake) | community | high | Dim 5, Known Issues |
| 16 | <https://github.com/modelcontextprotocol/servers/issues/3232> (startup fails on unavailable dir) | community | medium | Known Issues |
| 17 | <https://cymulate.com/blog/cve-2025-53109-53110-escaperoute-anthropic/> (EscapeRoute CVEs) | community | high | Dim 5, Security |
| 18 | <https://github.com/advisories/GHSA-q66q-fx2p-7w4m> (CVE-2025-53109) | official | high | Security |
| 19 | <https://github.com/advisories/GHSA-hc55-p739-j48w> (CVE-2025-53110) | official | high | Security |
| 20 | <https://security.snyk.io/package/npm/@modelcontextprotocol%2Fserver-filesystem> (no advisories > 2025.7.1) | community | high | Security |
| 21 | <https://stacklok.com/blog/examining-the-impact-of-npm-supply-chain-attacks-on-mcp> (chalk/debug, Shai-Hulud) | community | high | Dim 5, Security |
| 22 | <https://github.com/shinpr/mcp-local-rag> (local-only citation-grade RAG reference) | community | medium | Dim 5 |
| 23 | <https://docs.claude.com/en/api/agent-sdk/custom-tools> (Claude SDK in-process MCP idiom) | official | high | Dim 5, Synthesis |
| 24 | <https://learn.microsoft.com/azure/foundry/agents/how-to/tools/file-search> (hosted FileSearch cites service-side) | official | medium | Dim 2 |
| 25 | <https://www.anthropic.com/engineering/code-execution-with-mcp> (MCP token bloat) | community | medium | Known Issues |