portfolio-optimiser/.claude/projects/2026-06-24-fase2-mvp-vertical-slice/research/02-local-folder-mcp-citation-provenance.md

type	created	question	confidence	dimensions	mcp_servers_used	local_agents_used	external_agents_used	topic	brief
trekresearch-brief	2026-06-24	What is the best way to expose a local document folder to a MAF 1.9.0 agent via MCPStdioTool such that retrieved content carries citation metadata (file + locator) — existing filesystem MCP server vs thin custom server, and the citation shape a context provider expects?	0.82	6	microsoft-learn tavily	installed-source-introspection (orchestrator main context)	docs-researcher community-researcher security-researcher contrarian-researcher gemini-bridge (unavailable)	2	.claude/projects/2026-06-24-fase2-mvp-vertical-slice/brief.md

Local-folder data access via MCP with citation provenance (MAF 1.9.0)

Generated by trekresearch (Voyage 5.6.0) on 2026-06-24. Topic 2 of 3 for the Fase 2 MVP-vertical-slice brief. Scope: external swarm + installed-source introspection (installed 1.9.0 API truth wins over Learn docs). Gemini unavailable (MCP SDK broke on Google's May-2026 API change).

Research Question

What is the best way to expose a LOCAL document folder to a MAF 1.9.0 agent via MCPStdioTool so retrieved content carries citation metadata (file + locator) — reuse an existing filesystem MCP server, or build a thin custom server — and what citation shape does MAF's context/annotation model expect? Decision it feeds: the data-access step, the citation-aware context provider, provenance-stamping, and the build-vs-reuse decision for the local-folder server.

Executive Summary

**The brief's lean ("reuse the official @modelcontextprotocol/server-filesystem

• attach citations via parse_tool_results") does not survive contact with the requirement and should be dropped.** The official filesystem server returns raw text and bare path strings only — no line/char offsets, no chunk IDs; its search_files is filename-glob, not content search — so it physically cannot yield a (file + locator + snippet) citation; you would have to re-read, re-chunk and re-locate inside the callback (i.e. build the whole citation engine anyway, in the worst place, behind a Node subprocess). MAF also does not auto-create citation annotations from MCP results. Recommendation (honouring the CLAUDE.md "data access via MCP, JSON-Schema-validated, fail-fast" convention): build a THIN CUSTOM local-folder MCP server that returns citation-ready structured chunks ({file, locator, snippet, score}), wrapping a framework-agnostic in-process retriever core that is the D7-portable seam. Carry provenance as first-class Pydantic data on the emitted proposal, independent of MAF's annotation propagation (which has an open Python streaming-drop bug, #4316). Confidence 0.82: the capability facts are HIGH (installed-source + official-README verified); the exact MVP retrieval depth (keyword vs local embeddings) is a deferrable design choice.

Dimensions

1. Can the official filesystem server carry citations? — Confidence: high (NO)

External findings (docs + community + contrarian agree):

• @modelcontextprotocol/server-filesystem tools: read_text_file (whole file, or head/tail by line count only), read_multiple_files, list_directory, directory_tree, get_file_info, search_files, write_file, edit_file. https://github.com/modelcontextprotocol/servers/blob/main/src/filesystem/README.md
• Read results = raw UTF-8 text, no line numbers, no byte/char offsets, no chunk IDs. search_files returns matching file PATHS only (filename glob — NOT content grep). get_file_info = size/timestamps, not a locator.
• Net: the only locator it gives is the file path you already passed in. Enhanced community forks advertise read_file_lines / grep-with-line-numbers / chunking precisely because the official server lacks them.

Conclusion: the official server cannot produce (file + locator + snippet) citations. It is fine only for "let the agent open named files," not citation-backed retrieval.

2. Does MAF auto-create citations from MCP results? — Confidence: high (NO)

Installed-source findings (1.9.0):

• Default parser _parse_contents_from_mcp_tool_result / _parse_tool_result_from_mcp maps MCP content → Content.from_text/from_data/from_uri; attaches no annotations (agent_framework/_mcp.py:507-517).
• structuredContent IS now serialized to text Content (_mcp.py:586-587) — so the #3313 "structuredContent dropped → None" bug is fixed in installed 1.9.0. But it lands as JSON text, not as typed/annotated content — you still parse it yourself for provenance.
• The seam to attach citations is MCPTool.__init__(parse_tool_results: Callable[[CallToolResult], str | list[Content]]) — your callback fully replaces the default parse.

External findings:

• Confirmed: docs/RAG paths use prompt-level "cite your sources" (a string_mapper), not structured annotations. The one citation-aware RAG provider, TextSearchProvider, is .NET-only — no Python equivalent. https://learn.microsoft.com/agent-framework/agents/rag
• Open bug #4316: file-citation annotations are silently dropped in the Python streaming path (annotations=None); .NET fixed, Python lagging. → do NOT rely on MAF's Annotation propagation for load-bearing provenance. https://github.com/microsoft/agent-framework/issues/4316

3. Citation shape MAF expects — Confidence: high

Installed-source findings (1.9.0, ground truth):

• Annotation is a TypedDict (agent_framework/_types.py:374): type: Literal["citation"], title, url, file_id, tool_name, snippet, annotated_regions: Sequence[TextSpanRegion], additional_properties, raw_representation.
• TextSpanRegion TypedDict (_types.py:366): type: Literal["text_span"], start_index, end_index.
• The class form CitationAnnotation was replaced by this TypedDict in 1.0.0b260123 (PR #3252) — installed 1.9.0 = TypedDict form. Build citations as dicts: {"type":"citation","file_id":...,"title":...,"snippet":...,"annotated_regions":[{"start_index":..,"end_index":..}]}.
• Carrier: FunctionResultContent.annotations: list[... ] | None (accepts dict/TypedDict).

Use: our provenance stamp's "≥1 citation" maps cleanly onto this Annotation. But because of #4316, treat Annotation as a display/serialization surface and keep the authoritative provenance as our own Pydantic object.

4. Context provider injection seam — Confidence: high (corrects an external error)

Installed-source findings (1.9.0) — and a flagged docs-vs-source conflict:

• class ContextProvider (agent_framework/_sessions.py:351), __init__(self, source_id: str) — source_id REQUIRED (confirms Fase 1).
• Hooks: async def before_run(...) (:370) and async def after_run(...) (:391) — add messages/instructions/tools in before_run, process/store in after_run.
• Context.extend_instructions(self, source_id: str, instructions: str | Sequence[str]) (:253) and extend_tools(source_id, tools) (:266) — the two-arg seam the brief and Fase 1 specified, confirmed verbatim.
• MemoryContextProvider / InMemoryHistoryProvider exist but are conversation memory, not retrievers.

Premiss correction (important for /trekplan): the docs-researcher reported, with high confidence from the Learn API-ref page, that Python uses invoking/invoked hooks and that source_id / extend_instructions are "C#-only, not in Python." That is wrong for installed 1.9.0 — the installed source has before_run/after_run, ContextProvider(source_id), and extend_instructions(source_id, …). Per CLAUDE.md, installed source wins. Do NOT let the planning phase adopt the invoking/invoked / no-source_id surface.

Design fit: a custom ExpeLContextProvider(ContextProvider) (Fase 1 seam, promote to core) injects retrieved+cited content + prior verdicts in before_run via extend_instructions(source_id, …), and the same provider family is where the ExpeL learning loop reads VerdictStore. No built-in local-folder retrieval provider exists in Python — retrieval is ours to build.

5. Build vs reuse (the decision) — Confidence: high (recommendation)

Triangulated picture:

• Reuse official server: cannot cite (Dim 1); adds a Node runtime + npx cold-start + version-pin burden; npx PATH/nvm handshake failure is THE most-reported stdio blocker on macOS (B5); supply-chain surface (chalk/debug + Shai-Hulud hit MCP-SDK transitives). Security: usable only if pinned ≥ 2025.7.1 (EscapeRoute CVE-2025-53109/53110, fixed there; current 2026.1.14 clean), vendored (never npx @latest), allowlist scoped to the doc folder (never $HOME).
• Thin custom MCP server: removes the npm supply-chain surface, emits provenance natively ({file, line_range/char_span, snippet, score} via structuredContent), honours the CLAUDE.md "data access via MCP, JSON-Schema-validated, fail-fast" convention, and is portable to the D7 Claude-SDK sibling's in-process-MCP idiom. Cost: we own path-canonicalisation + symlink-realpath validation (the exact bug class the official server had fixed twice — TDD against the EscapeRoute scenarios) and input validation (no shelling out; use native file APIs).
• In-process retriever, no MCP (contrarian's first choice): simplest + lowest latency + best D7 fit, but conflicts with the project's "data access via MCP" convention. Keep as the fallback if that convention is relaxed.
• Reference design: shinpr/mcp-local-rag (LanceDB file-based + local embeddings; returns path + chunk index + title + score + chunk text + read_chunk_neighbors context expansion) — the closest local-only, citation-grade pattern. https://github.com/shinpr/mcp-local-rag

6. Retrieval depth for the MVP — Confidence: medium

• The MVP uses a tiny synthetic "anleggskostnad" domain (Fase 0). A full vector stack is not required for D5 (90%) / D6 (cost). A simple in-process retriever (keyword/substring + chunk-at-ingest with exact locators) likely suffices and keeps deps minimal.
• The Python "real RAG" path (SK VectorStore.create_search_function().as_agent_framework_tool()) requires pulling in semantic-kernel ≥ 1.38 — a different package with its own RCE CVE (CVE-2026-26030, eval in in-memory vector-store filter, < 1.39.4). Adding it conflicts with the "GA-pakker pinnet eksplisitt, ikke metaen" dependency discipline and adds CVE surface. Defer / avoid for MVP unless semantic retrieval proves necessary; if adopted, pin ≥ 1.39.4.

External Knowledge

Best Practice

• Canonical MCPStdioTool wiring (official docstring + Learn): MCPStdioTool(name="filesystem", command="npx", args=["-y","@modelcontextprotocol/server-filesystem", <dir>]), used as async with; requires pip install mcp --pre. https://learn.microsoft.com/agent-framework/agents/tools/local-mcp-tools
• MAF can also expose an agent as an MCP server (agent.as_mcp_server()) — not relevant to local-folder retrieval. Generic server-building is the MCP Python SDK / FastMCP domain. https://github.com/modelcontextprotocol/python-sdk

Security (D3 no-silent-egress + local-only)

• Stdio MCP server makes no network calls; only inherent egress in the path is the npx fetch at launch (removed by vendoring/pinning). OTel _meta trace-context injection into tools/call is inert with instrumentation off and carries only random W3C IDs (no project data).
• EscapeRoute CVEs (CVE-2025-53110 prefix-collision, CVE-2025-53109 symlink-escape) — both fixed in 2025.7.1; exploit trigger is prompt injection of the agent (relevant: our maker-checker + blocking validator are compensating controls). Never run the server with elevated privileges. https://cymulate.com/blog/cve-2025-53109-53110-escaperoute-anthropic/
• npx supply chain: pin exact version + lockfile, verify tarball for postinstall, quarantine fresh versions, consider containerization. A thin custom dependency-light server removes this surface entirely.

Known Issues

• MAF MCP bugs — verified against installed 1.9.0: #3313 structuredContent parsing FIXED (_mcp.py:586); #2884 stdio session-invalidation handled (is_connected reset + ClosedResourceError catch, _mcp.py:935/1355/1544). Still: #2284 _meta dropped (don't rely on _meta for provenance — use structuredContent/content); #4316 Python streaming citation-annotation drop (open).
• npx/nvm PATH handshake failure on macOS (servers#64) — use absolute command path + explicit env PATH/NODE_PATH, prefer pre-installed/uvx-pinned over cold npx -y. Filesystem server fails to start if any allowed dir is unavailable (servers#3232).
• Don't split a citation across content vs structuredContent vs _meta — clients (MAF included) forward them inconsistently; put it in one reliably-forwarded channel.

Gemini Second Opinion

Unavailable (same MCP-SDK/Interactions-API failure as Topic 1). No independent Gemini triangulation for this topic; treat as absent, not negative.

Synthesis

Three sources converge on a conclusion the brief did not anticipate: the citation requirement, not the data-access requirement, drives the design — and it rules out the off-the-shelf filesystem server. Citations of the form (file + locator + snippet) must be exact by construction, which means chunk-and-locate at ingest, in code we own. Bolting them on afterward — over a third-party server's lossy whole-file text, through a parse_tool_results callback, relying on a MAF annotation path with an open streaming-drop bug — stacks three fragilities to recreate something we could have produced cleanly up front.

The right shape is a two-layer seam: (1) a framework-agnostic in-process retriever core — retrieve(query) -> list[RetrievedChunk], where RetrievedChunk is a Pydantic type carrying file, locator (line range / char span), snippet, score — which is the D7-portable contract (MAF and Claude Agent SDK both satisfy it); and (2) a thin custom MCP server wrapping that core, returning the chunks as structured structuredContent, which honours the project's "data access via MCP, JSON-Schema-validated, fail-fast" convention and ports to the Claude SDK's in-process-MCP idiom. Provenance lives as first-class Pydantic data on the emitted proposal; the MAF Annotation TypedDict is only a display/serialisation view of it, so #4316 never sits on the load-bearing path. The official filesystem server is demoted to a possible future "browse named files" convenience, pinned and scoped.

This also threads the CLAUDE.md needle: the contrarian's "skip MCP, in-process only" is the simplest and the best D7 fit, but it breaks the MCP-data-access convention; the thin-custom-MCP-server-over-an-in-process-core keeps the convention and the D7 portability and citation fidelity — at the cost of owning sandbox correctness (testable, bounded).

Open Questions

• MVP retrieval depth: keyword/substring + ingest-time chunking (no extra deps, D5/D6-aligned) vs local embeddings (LanceDB/SK VectorStore, heavier + CVE surface). Default to keyword for the tiny synthetic domain; revisit only if recall is poor. Resolve in /trekplan.
• Locator format: line range vs char span. It is the shared D7 contract — define and test it in the golden-suite up front. Default: char span (start_index/ end_index) to map straight onto TextSpanRegion; carry line numbers in additional_properties for human display.
• Does the MVP even need MCP, or is the in-process retriever enough? The CLAUDE.md convention says data access via MCP. The thin-custom-server honours it; confirm the operator wants the MCP boundary in Fase 2 vs deferring it (the in-process core is needed either way). Surface in /trekplan.
• Chunk-at-ingest vs at-query: ingest-time chunking gives exact, cacheable locators; decide whether the MVP ingests once or reads live. Tie to determinism NFR.

Recommendation

Build a thin custom local-folder MCP server that returns citation-ready structured chunks, wrapping a framework-agnostic in-process retriever core. Do NOT reuse @modelcontextprotocol/server-filesystem for citation-backed retrieval.

1. Retriever core (D7 seam): retrieve(query, top_k) -> list[RetrievedChunk]; RetrievedChunk(file: str, locator: TextSpan, snippet: str, score: float). Chunk + assign locators at ingest so citations are exact by construction. Start with keyword/substring retrieval (no heavy deps; D5/D6). Pure Python, no MAF/MCP imports — portable to the Claude SDK sibling.
2. Thin custom MCP server (MAF-facing, honours the convention): a minimal mcp/FastMCP stdio server exposing a search(query) tool that returns the chunks via structuredContent (JSON-Schema-validated, fail-fast on bad config). Path handling is security-critical: canonicalise + boundary-check (no startsWith), resolve symlink realpath and fail closed, native file APIs (no shelling out) — TDD against the EscapeRoute bypass patterns.
3. Wire into MAF: MCPStdioTool(name="docfolder", command=<abs python>, args=[...]) with a parse_tool_results callback that maps structuredContent chunks into FunctionResultContent + Annotation(type="citation", file_id, snippet, annotated_regions=[TextSpanRegion]). On Intel mac, pass absolute command + explicit env PATH; pre-install the server (no cold npx/uvx fetch on the path).
4. Provenance = first-class Pydantic data on the emitted ValidatedProposal (≥1 citation: file + locator + snippet, plus model/role + validator decision + token usage). The MAF Annotation TypedDict is only its display view — route around #4316 by never depending on MAF annotation propagation for the assertion.
5. Citation-aware context injection: a custom ExpeLContextProvider(ContextProvider) (source_id required) uses before_run + extend_instructions(source_id, …) to inject cited content (and prior verdicts) — the confirmed two-arg seam.
6. If the official filesystem server is ever used (future "browse named files"): pin ≥ 2025.7.1 vendored (never npx @latest), allowlist the doc folder only (never $HOME), absolute node path + explicit env.

Fallback (only if the MCP-data-access convention is relaxed for the MVP): skip the MCP server and call the in-process retriever directly via a @tool/ContextProvider — simplest, lowest-latency, best D7 fit.

Risks to carry into the plan: (a) we own MCP-server sandbox correctness — treat path/symlink validation as security-critical, TDD it; (b) MAF Python citation annotation propagation is buggy in streaming (#4316) — provenance must be our own data; (c) adding semantic-kernel for vector RAG pulls a CVE-bearing package — defer.

Sources

#	Source	Type	Quality	Used in
1	.venv/.../agent_framework/_mcp.py:507-517,586-587,655 (default parser, structuredContent)	codebase	high	Dim 2, Known Issues
2	.venv/.../agent_framework/_mcp.py:911,935,1330,1355,1544 (session lifecycle, ClosedResourceError)	codebase	high	Known Issues
3	.venv/.../agent_framework/_types.py:366,374 (TextSpanRegion, Annotation TypedDict)	codebase	high	Dim 3
4	.venv/.../agent_framework/_sessions.py:351,370,391,253,266 (ContextProvider, before_run/after_run, extend_instructions)	codebase	high	Dim 4
5	https://github.com/modelcontextprotocol/servers/blob/main/src/filesystem/README.md (tool surface, no locators)	official	high	Dim 1
6	https://www.npmjs.com/package/@modelcontextprotocol/server-filesystem	official	high	Dim 1,5
7	https://learn.microsoft.com/python/api/agent-framework-core/agent_framework.mcpstdiotool?view=agent-framework-python-latest	official	high	Dim 5, Best Practice
8	https://learn.microsoft.com/agent-framework/agents/tools/local-mcp-tools	official	high	Best Practice
9	https://learn.microsoft.com/agent-framework/agents/rag (TextSearchProvider .NET-only; SK VectorStore path)	official	high	Dim 2, 6
10	https://learn.microsoft.com/agent-framework/support/upgrade/python-2026-significant-changes (Annotation TypedDict PR #3252)	official	high	Dim 3
11	https://github.com/microsoft/agent-framework/issues/4316 (Python streaming citation drop, OPEN)	community	high	Dim 2, Synthesis
12	https://github.com/microsoft/agent-framework/issues/3313 (structuredContent dropped, fixed)	community	high	Dim 2, Known Issues
13	https://github.com/microsoft/agent-framework/issues/2884 (stdio session not invalidated, fixed #3154)	community	high	Known Issues
14	https://github.com/microsoft/agent-framework/issues/2284 (_meta discarded)	community	medium	Known Issues
15	https://github.com/modelcontextprotocol/servers/issues/64 (npx/nvm PATH handshake)	community	high	Dim 5, Known Issues
16	https://github.com/modelcontextprotocol/servers/issues/3232 (startup fails on unavailable dir)	community	medium	Known Issues
17	https://cymulate.com/blog/cve-2025-53109-53110-escaperoute-anthropic/ (EscapeRoute CVEs)	community	high	Dim 5, Security
18	https://github.com/advisories/GHSA-q66q-fx2p-7w4m (CVE-2025-53109)	official	high	Security
19	https://github.com/advisories/GHSA-hc55-p739-j48w (CVE-2025-53110)	official	high	Security
20	https://security.snyk.io/package/npm/@modelcontextprotocol%2Fserver-filesystem (no advisories > 2025.7.1)	community	high	Security
21	https://stacklok.com/blog/examining-the-impact-of-npm-supply-chain-attacks-on-mcp (chalk/debug, Shai-Hulud)	community	high	Dim 5, Security
22	https://github.com/shinpr/mcp-local-rag (local-only citation-grade RAG reference)	community	medium	Dim 5
23	https://docs.claude.com/en/api/agent-sdk/custom-tools (Claude SDK in-process MCP idiom)	official	high	Dim 5, Synthesis
24	https://learn.microsoft.com/azure/foundry/agents/how-to/tools/file-search (hosted FileSearch cites service-side)	official	medium	Dim 2
25	https://www.anthropic.com/engineering/code-execution-with-mcp (MCP token bloat)	community	medium	Known Issues