open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

fix(agents): an agent cannot promise what its tools forbid

Browse source 
R4 — `verifier-agent.md` carried two contracts at once: §Output Format said
"Append to: implementation-log.md", §Read-Only Guarantee said "never modifies
any files", and `tools:` granted only Read/Glob/Grep. Which one wins is
nondeterministic, and the loss is not a blocked write but a full-file Write on
the log the agent SHARES with the parallel implementer agents — the defect
`implement-log-append.test.mjs` exists to prevent, entering through the one file
that test does not read. The orchestrator half was already right
(`implement.md` Step 5 appends with Bash `>>` and tells the agent not to write),
so the fix is one-way: the agent file now returns its report inline and names
who appends it, and why a Write there would clobber.

The guard is the blanket invariant over the catalogue, not a fact about one
file: any agent whose tools grant no write capability must instruct no write AND
say positively that it returns findings inline. Tools and body are both read, so
stripping `Write` from any agent whose body still writes turns it red. Measured
1 of 7 agents carried the defect; the sweep asserts a write-tool-less agent
exists so the invariant cannot pass vacuously.

R6 — both "Required Frontmatter" rules were enforced by nothing, and the only
test reading agent frontmatter checked `name:` against a hand-written 3-of-7
list. The new guard takes nothing by hand: required keys are parsed from each
rule's own yaml block, the swept files from each rule's own `paths:`, the plugin
name from plugin.json — add a key to a rule and it is enforced next run. The
repo was already 7/7 and 21/21 compliant, so a green first run proves nothing:
all seven arms were seen red against a temporarily introduced defect one at a
time, including emptying a rule's yaml block to show the derivation is not
vacuous.

The color enum is deliberately NOT guarded: the official subagent docs list
red/blue/green/yellow/purple/orange/pink/cyan (no magenta) while issue 19292
lists magenta but neither purple nor orange, and this plugin ships both. Pinning
an unsettled set would encode an unverified premise rather than measure one.

Suite 1777 -> 1785, 0 fail. Frozen tests/snapshots/v5.0.0/ untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MEMKCAyVzYTMzLaqPcULVr
This commit is contained in:
Kjell Tore Guttormsen 2026-08-20 22:59:51 +02:00
commit d5714261d1
4 changed files with 289 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

14
agents/verifier-agent.md
View file

@ -140,7 +140,15 @@ Checking for secrets...
## Output Format
Append to: `~/.claude/config-audit/sessions/{session-id}/implementation-log.md`
Return the report below as your final message. Do NOT write it to a file: this
agent is read-only by design (`tools: Read, Glob, Grep`) and has no write tool,
so a write instruction here would be a contract it cannot keep.
The orchestrator appends what you return to
`~/.claude/config-audit/sessions/{session-id}/implementation-log.md` itself,
with Bash `>>` (`commands/implement.md` Step 5) — never the Write tool. That log
is shared with the implementer agents running in parallel, and a full-file Write
on it silently clobbers their entries.
```markdown
## Verification Report
@ -243,8 +251,8 @@ Optional: Generate before/after comparison:
This agent:
- Only uses Read, Glob, Grep tools
- Never modifies any files
- Reports findings without taking action
- Never modifies any files, including the shared implementation log
- Reports findings without taking action — every result is returned inline
- Safe to run multiple times
## Model policy