diff --git a/docs/research/2026-06-24-maf-vs-claude-agent-sdk.md b/docs/research/2026-06-24-maf-vs-claude-agent-sdk.md new file mode 100644 index 0000000..9440c99 --- /dev/null +++ b/docs/research/2026-06-24-maf-vs-claude-agent-sdk.md @@ -0,0 +1,192 @@ +# Microsoft Agent Framework (MAF) vs. Claude Agent SDK — a practitioner's comparison + +> **Date:** 2026-06-24 · **Audience:** people who know agent frameworks but not these two specifically. +> **Why this exists:** portfolio-optimiser is built on MAF; the same method will be rebuilt on Claude +> Agent SDK as a separate sibling repo (decision D7). This doc grounds that work and is a standalone +> reference. +> +> **Verification status:** MAF facts were ground-truth-introspected against the installed +> `agent-framework-core==1.9.0` in this repo + Microsoft Learn. Claude Agent SDK facts were verified +> against Anthropic docs (code.claude.com) + npm/PyPI. Items the research could not confirm are marked +> **(unverified)** and must not be repeated as fact. + +--- + +## TL;DR — they are two different *kinds* of thing + +- **MAF is a multi-agent *orchestration framework*.** You declare the topology. It ships named, + first-class collaboration patterns (Sequential, Concurrent, GroupChat, Handoff, Magentic) and an + explicit, deterministic graph workflow engine. The control plane is **explicit and yours**. It is + **model-agnostic** (provider abstraction), Azure/enterprise heritage (the merger of Semantic Kernel + + AutoGen), with OpenTelemetry observability built in. + +- **Claude Agent SDK is a single autonomous *agent harness*.** It exposes the same agentic loop that + powers Claude Code (gather context → act → verify → repeat). You steer **one** capable agent with + tools, hooks, permissions, and the ability to delegate to subagents. The control plane is + **implicit** — Claude decides the sequence; you constrain it with hooks. It is **Claude-only** (but + multi-cloud), coding/ops heritage, observability is DIY. + +**Mental model:** MAF = you are the *conductor writing the score*; the agents are players executing +your composition. Claude Agent SDK = you hire one very capable *generalist*, give it tools and +guardrails, and let it decide how to get the job done — delegating to *specialists* (subagents) when +it judges it useful. + +The single sharpest axis: **MAF makes orchestration explicit and declarative; Claude Agent SDK makes +it implicit and emergent, bounded by hooks.** + +--- + +## Side-by-side + +| Dimension | Microsoft Agent Framework (MAF) | Claude Agent SDK | +|---|---|---| +| **What it is** | Orchestration framework for agents + workflows | Programmatic harness over the Claude Code agent loop | +| **Lineage** | Successor to & merger of Semantic Kernel + AutoGen | Renamed from "Claude Code SDK"; evolved beyond coding | +| **Maturity** | **GA 1.0.0** (Python, 2026-04-02); installed here: core 1.9.0 | Pre-1.0, very fast cadence (TS at 0.3.x, 225+ npm releases) | +| **Languages** | Python ≥3.10 and .NET (C#) — not full parity | Python ≥3.10 and TypeScript — TS slightly richer | +| **Models** | **Model-agnostic** via provider packages (Azure OpenAI, Foundry, OpenAI, Ollama, Anthropic*, Gemini*, Bedrock*) | **Claude only**, but multi-cloud: Anthropic API / Bedrock / Vertex / Azure Foundry | +| **Core primitive** | `Agent` (LLM loop) + `WorkflowBuilder` (graph of `Executor` nodes) | `query()` async generator + `ClaudeSDKClient` (persistent) | +| **Multi-agent** | **Named patterns**: Sequential, Concurrent, GroupChat, Handoff, Magentic | **No named patterns** — compose yourself via subagents / N calls | +| **Determinism** | Native: `FunctionExecutor` node + conditional edges in the graph | Via `PreToolUse` hook that denies until a step has run | +| **Tools / MCP** | MCP **client** (stdio/HTTP/WS) + agent can be exposed **as** MCP server | Built-in tools + custom tools via in-process MCP server; MCP **client** | +| **HITL** | `ToolApprovalMiddleware`, `RunContext.request_info()`, Magentic plan-review | `permission_mode`, `allowed_tools`, `PreToolUse`/`PermissionRequest` hooks | +| **Observability** | **Built-in OpenTelemetry** (GenAI semantic conventions, metrics, evals) | **DIY** via `PostToolUse`/`SubagentStop` hooks; no built-in OTel (unverified) | +| **State / memory** | Sessions, file/Redis history, workflow checkpointing, compaction strategies | Session resume (JSONL on your FS), auto-compaction, `PreCompact` hook | +| **License** | MIT | "SEE LICENSE IN README" + Anthropic Commercial ToS (FOSS license **unverified**) | +| **Runs where** | Your process; Azure/Foundry-first for hosted tools | Your process / filesystem; bundles a Claude Code binary | + +\* beta provider packages — require `--pre`, not production-ready. + +--- + +## The dimensions that matter most + +### 1. Multi-agent orchestration — the biggest divide + +**MAF** gives you the topology as a first-class, named object. All five live in a separate package, +`agent-framework-orchestrations` (GA 1.0.0; **not** bundled with core): + +| Pattern | Builder | Behaviour | +|---|---|---| +| Sequential | `SequentialBuilder` | A → B → C, output chained | +| Concurrent | `ConcurrentBuilder` | Same input fan-out, results aggregated | +| GroupChat | `GroupChatBuilder` | Star topology; an orchestrator picks the next speaker (round-robin / prompt / custom selection fn) | +| Handoff | `HandoffBuilder` | Point-to-point control transfer between agents | +| Magentic | `MagenticBuilder` | Magentic-One manager that plans, tracks a progress ledger, and coordinates specialists dynamically | + +> Note: Microsoft itself warns Magentic is "untested … outside of the original Magentic-One design." +> Treat as beta for novel use cases. + +**Claude Agent SDK** has **no** equivalent. The multi-agent mechanism is **subagents**: your main +agent spawns separate agent instances (via the `Agent` tool) defined programmatically through +`AgentDefinition` (or `.claude/agents/*.md`). Key properties: + +- The main agent decides *when* to delegate, from each subagent's `description`. You can also ask + explicitly ("use the X agent"). +- A subagent does **not** inherit the parent's history; the only channel in is the spawn prompt, and + it returns **only its final answer** (not intermediate steps). +- Subagents can run in **parallel** and **nest** (up to 5 levels, since Claude Code 2.1.172). +- There is **no built-in coordination protocol** between peer agents — no debate loop, no consensus, + no voting. To coordinate N peers you run N `query()` calls and aggregate yourself, or have one + orchestrator agent fan out to subagents. (A `Workflow` tool for 10–100-agent scale exists in the + **TypeScript** SDK; availability in Python is **unverified**.) + +**So:** if your design *is* a named multi-agent topology (a maker-checker debate, a planner-manager), +MAF hands it to you. On Claude Agent SDK you hand-roll it from subagents + your own glue. + +### 2. Determinism / forcing a blocking step + +This is MAF's headline differentiator and directly relevant to a "mandatory deterministic validator." + +- **MAF:** add a `FunctionExecutor` — a pure Python function, no LLM — as a node in `WorkflowBuilder`, + with conditional edges (`EdgeCondition`, `SwitchCaseEdgeGroup`). It blocks the graph until it + returns. The sequence is **declared in code**; the validator is a graph node by construction. +- **Claude Agent SDK:** there is no declarative graph. You force a sequence with a `PreToolUse` hook + that returns `permissionDecision: "deny"` for any tool until your validator tool has run. You can + also rewrite tool input (`updatedInput`), inject context, or replace tool output before the model + sees it. Powerful, but the determinism is *enforced by interception*, not *expressed as structure*. + +**Net:** stronger, more legible deterministic control → MAF. More flexible autonomy with guardrails → +Claude Agent SDK. + +### 3. Model coupling + +- **MAF** abstracts the provider behind `BaseChatClient`; you swap Azure OpenAI ↔ OpenAI ↔ Foundry ↔ + local OpenAI-compatible endpoints ↔ (beta) Anthropic/Gemini/Bedrock. Instructions, tools, and + middleware are portable; provider-specific run-options (`OpenAIChatOptions`, `FoundryChatOptions`) + are not. +- **Claude Agent SDK** runs **Claude only** — but across Anthropic API, Amazon Bedrock, Google Vertex, + and Azure AI Foundry via env vars (`CLAUDE_CODE_USE_BEDROCK/VERTEX/FOUNDRY`), with no code change. + Model *aliases* (`opus`/`sonnet`/`haiku`/`fable`/`inherit`) or full IDs in `ClaudeAgentOptions` / + `AgentDefinition`. + +### 4. Tools, MCP, and data access + +Both are MCP **clients**. The difference is symmetry: + +- **MAF:** MCP client over three transports (`MCPStdioTool`, `MCPStreamableHTTPTool`, + `MCPWebsocketTool`); and crucially `agent.as_mcp_server()` exposes an agent **as** an MCP server for + other clients. Function tools via `@tool`/`FunctionTool`. +- **Claude Agent SDK:** built-in tools (`Read`/`Write`/`Edit`/`Bash`/`Grep`/`Glob`/`WebSearch`/ + `WebFetch`/`Agent`/`AskUserQuestion`); custom tools via `@tool` + `create_sdk_mcp_server` running + **in-process** (exposed as `mcp__{server}__{tool}`); external MCP servers via `mcp_servers`. + +### 5. Observability + +- **MAF:** OpenTelemetry is built in (`agent_framework.observability`, GenAI semantic conventions, + token-usage histograms, workflow/edge/MCP spans) plus an experimental eval harness + (`evaluate_agent`, `evaluate_workflow`). +- **Claude Agent SDK:** you build it from hooks (`PostToolUse`, `SubagentStop`, `parent_tool_use_id`), + asynchronously if you want non-blocking logging. No first-class OTel integration found **(unverified)**. + +--- + +## What each is best at + +**MAF wins when** you need explicit, auditable orchestration; a *blocking deterministic step* between +LLM turns expressed as structure; named multi-agent debate/handoff patterns out of the box; model +portability; built-in telemetry from day one; or you are migrating from Semantic Kernel / AutoGen. + +**Claude Agent SDK wins when** you want one strong agent that lives on *your* infrastructure (your +filesystem, your secrets, no vendor sandbox); coding/DevOps automation (its native turf); fast +prototype → production with `query()`; rich MCP-client integration; and you are happy to be on Claude +models across multiple clouds. Its orchestration is emergent, not declared — great for open-ended +autonomy, weaker for "prove the exact sequence ran." + +--- + +## Implication for portfolio-optimiser (decision D7) + +The product is a **maker-checker debate** that generates candidate measures, gated by a **mandatory, +blocking deterministic validator**, with a HITL learning loop. + +- On **MAF** (this repo): debate = `GroupChatBuilder`; validator = a `FunctionExecutor` graph node — + blocking by construction. Direct fit; this is *why* MAF was chosen. +- On **Claude Agent SDK** (sibling repo, D7): debate = an orchestrator agent fanning out to maker / + checker subagents (or N `query()` calls + your aggregation); the validator is enforced by a + `PreToolUse` hook that denies any action until the validator tool has run. Fully possible, but you + hand-roll the orchestration and the "blocking" is interception, not a declared edge. + +**Because the two paradigms differ this much, a single shared runtime abstraction is the wrong move.** +The portable, framework-neutral asset is the **method**, not the orchestration code: the Agent Skill, +the IR contract, the validator rules, the synthetic reference domain, and — most valuable for a +fair comparison — a **shared golden/conformance suite** (synthetic domain inputs → expected validator +verdicts) that both implementations must pass. That is what makes "the same method on two frameworks" +a measurable claim rather than two vaguely similar projects. + +--- + +## Open / unverified items (do not repeat as fact) + +- Exact date of the Claude Code SDK → Claude Agent SDK rename (secondary sources say ~Sept 2025). +- The code's FOSS license (npm says "SEE LICENSE IN README"). +- `canUseTool` callback details; `Workflow` tool availability in the **Python** Claude Agent SDK. +- Claude Agent SDK built-in OpenTelemetry support (none found, not definitively absent). +- MAF beta Anthropic provider package name (`agent-framework-claude` assumed, not confirmed). + +## Sources + +- Microsoft Agent Framework: `learn.microsoft.com/agent-framework`, `github.com/microsoft/agent-framework`, + + ground-truth introspection of installed `agent-framework-core` 1.9.0. +- Claude Agent SDK: `code.claude.com/docs/en/agent-sdk` (overview, migration-guide, subagents, hooks, + custom-tools), `github.com/anthropics/claude-agent-sdk-{python,typescript}`, npm/PyPI version checks.