feat(provenance): a run records which external service it actually called
The egress declaration (Trekk B3) says what a run MAY contact. It cannot say what it DID: after the run, nothing distinguished "the agents queried the price register" from "the agents ignored it", and a proposal resting on an external service should be traceable to it. ToolCallRecorder(FunctionMiddleware) mirrors BudgetMiddleware(ChatMiddleware) one layer down — that one observes the debate's chat calls, this one its tool calls. It observes only: call_next is always awaited, so a trace can never alter the run it traces. The record lands on ProvenanceStamp.external_calls, read AFTER the debate so it is a record rather than an intention. MEASURED, not assumed, before any of it was written: FunctionMiddleware fires for a tool served over a REAL MCP stdio subprocess, and context.function.name carries the BARE tool name with no server prefix. That measurement decided the design — MAF cannot tell us which server a tool came from, so attribution comes from our own config, and a name allowed by two servers is recorded UNATTRIBUTED (server="") rather than credited to the first match. Naming a service that may never have been contacted is the one place a guess must not go. Only CONFIGURED tools are recorded. The middleware fires for every function the agents invoke, including the in-process retrieve_cost_docs on the road path; logging those would turn the record into a false egress claim. An empty list is a positive statement — nothing outside this process was contacted — which is why it is always serialized rather than omitted. Honesty limit, written on ExternalCall itself: this is the call and its source. It is NOT evidence that the service's answer reached the proposal, nor a verified rendering of that answer. One finding, and it is the reason for measuring rather than trusting green: the road-path negative test was VACUOUS. Its scripted tool call named an argument the tool does not declare (code vs query), MAF rejected the call before invocation, and the test asserted an empty record against a run where no tool ran at all — green under the exact mutation it existed to catch. It now spies on the recorder and asserts the invocation genuinely reached it before asserting it was not recorded. This is last session's lesson again: a scenario that cannot distinguish two implementations proves nothing. The tool-call double is registered in test_scripted_client_consolidation.py's _DELEGATING_OVERRIDES — it cannot live in the reply_selector seam, which returns a reply STRING, and a response that is not text is its whole subject. Load-bearing MEASURED (tests/test_b4_mcp_call_trace_loadbearing.py) against the whole 755-test suite, four mutations all red: detach the recorder from the debate middleware · record every function invocation · attribute an ambiguous name to the first server · stop reading the recorder into provenance. Control: a run with no configured servers records nothing, so the empty record is a real answer and not the only one the seam can produce. Ran it, not just tested it: the real recorder against a real MCP server subprocess returns ExternalCall(server='prisregister', tool='lookup_unit_price'), and a scripted CLI run's outbox artefact carries the empty list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VtRd8y1PDPGwkrRXFhubqr
This commit is contained in:
parent
455d93d33e
commit
991131be3f
6 changed files with 402 additions and 2 deletions
|
|
@ -176,3 +176,24 @@ Tre ting er verdt å vite som bestiller:
|
|||
og få svar med feil tilgang.
|
||||
|
||||
Uten `--mcp-config` gjøres ingen nettverkskall i det hele tatt.
|
||||
|
||||
### Etterpå: hva ble faktisk kalt
|
||||
|
||||
Kunngjøringen er en **tillatelse** — den sier hva kjøringen *kan* kontakte. Den sier ikke om
|
||||
agentene brukte tjenesten eller lot den ligge. Derfor fører kjøringen også et **kall-spor**, som
|
||||
følger med artefaktene under `provenance.external_calls`:
|
||||
|
||||
```json
|
||||
"external_calls": [{"server": "prisregister", "tool": "lookup_unit_price"}]
|
||||
```
|
||||
|
||||
En tom liste er en påstand, ikke et hull: ingenting utenfor prosessen ble kontaktet. Verktøy som
|
||||
kjører lokalt (som dokumentsøket) telles ikke med — de er ikke kontakt med noen.
|
||||
|
||||
**Hva sporet ikke sier.** Det viser at verktøyet ble kalt, og hvilken server det tilhører. Det er
|
||||
ikke bevis for at tjenestens svar er det tallet som havnet i forslaget, og det er ingen kontrollert
|
||||
gjengivelse av hva tjenesten svarte. Skal svaret etterprøves, må det gjøres mot tjenesten selv.
|
||||
|
||||
En sjelden detalj, tatt med fordi den ellers ville sett ut som en feil: hvis to servere tilbyr
|
||||
verktøy med **samme navn**, står `server` tomt. Rammeverket oppgir bare verktøynavnet, så de to lar
|
||||
seg ikke skille — og da er «vet ikke» riktigere enn å gjette på den ene.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue