config-audit/commands/campaign.md
Kjell Tore Guttormsen 09f817977c fix(commands): stop assuming shell state survives between blocks
Dogfooding `plan` + `implement` against a throwaway config surfaced one root
defect with many arms: the command templates treat consecutive fenced blocks as
one shell. They are not. Every ```bash fence runs as its own Bash call in its own
process, so a variable set in one block is empty in the next, and `$$` is a
different PID (measured: 21710 vs 22109).

The planner agent confirmed the sharpest arm at runtime, reporting that
`Mode: $RAW_FLAG` "arrived literally unsubstituted" — `--raw` was documented in
three command files while being functionally dead. A machine sweep found the same
root in 20 places across 9 files, well past the two the written fasit predicted:

  - `$RAW_FLAG` read from non-shell agent prompts (analyze, plan, implement)
  - `$TMPFILE` read across blocks (tokens, manifest, whats-active,
    plugin-health) — each command could not read the file it had just written
  - `$GLOBAL_FLAG` across blocks (fix)
  - `$TODAY` never assigned in any block (campaign), passing
    `--reference-date ""` to a write CLI in six places
  - three `$$` temp paths handed to the Read tool (fix), which expands neither

All now follow the hardened drift.md pattern: a fixed literal path, or a
re-derivation inside each block that needs it.

Also fixed, all confirmed against ground truth rather than inferred:

  - `implement` printed a rollback ID it never captured (the timestamp lived only
    inside a command substitution) — the one message a user reads after a bad run
  - `plan` reported "No analysis results found" for valid sessions, because Read
    was pointed at a glob it cannot expand; now uses Glob and verifies the
    analysis report exists before spawning the agent
  - five phase commands wrote state.yaml with two of four required fields; since
    the agent writes all four, a follow-up write silently deleted the rest
  - `implement` promised rollback deletes created files; rollback deliberately
    leaves them (M-BUG-26 still open) — the doc, not the engine, was wrong
  - `implement` claimed a score delta with no pre-change measurement
  - `verifier-agent` was told to write a report it has no tool to write
  - dead `Task` tool name in always-loaded rule context; planner-agent template
    demonstrated the inline file content its own line 110 forbids

The sweeps land as tests/commands/command-shell-state-shape.test.mjs, verified
red before the fix and proven able to fail by reintroducing the defect. Two
existing tests asserted the old bash-block mechanism rather than the intent and
were updated. Suite 1449/0; frozen v5.0.0 snapshots and all scanner code
untouched.

Not fixed, deliberately: neither command scope-gates its actions to the audit
target. The generated plan included an edit to a real file under ~/.claude,
outside the throwaway target, because the skill/agent scanners are machine-wide.
That is a design change, not a side fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0195udHgCcFegzm7ecKku2Yc
2026-08-01 20:12:17 +02:00

17 KiB

name description argument-hint allowed-tools model
config-audit:campaign Machine-wide audit campaign — track which repos are pending/audited/planned/implemented across sessions, with a machine-wide roll-up. Human-approved writes only. [init | add <path>... | set-status <path> <status> | refresh-tokens | export <path>] Read, Write, Edit, Bash, Glob opus

Config-Audit: Campaign

A single config-audit session audits one scope. A campaign sits above sessions: a durable ledger of every repo you mean to bring up to standard, each repo's lifecycle status (pending → audited → planned → implemented), and a machine-wide roll-up of findings by severity. It persists to ~/.claude/config-audit/campaign-ledger.jsonoutside the plugin dir, next to sessions/ — so it survives plugin uninstall/reinstall/upgrade and resumes across sessions.

The Iron rule (Verifiseringsplikt): nothing is ever auto-written. Reporting is read-only. Every mutation — creating the ledger, adding a repo, changing a status — is proposed first and applied only on explicit approval, by invoking one deterministic write-CLI subcommand. The command never hand-edits the ledger JSON.

This is the THIN campaign surface (ledger + roll-up + status + a cross-repo prioritized backlog to pick from + plan export). It does not run audits or apply fixes itself: it tracks where each repo stands, shows what to tackle next, exports a planned repo's plan into that repo's own docs/, and points at the existing per-repo /config-audit implement (backup + apply + verify) and /config-audit rollback for execution — Block 4c reuses that machinery, it does not reinvent it.

Three CLIs back this command

  • Read (report): scanners/campaign-cli.mjs — loads + validates the ledger, emits the repo list + roll-up + backlog. Never writes.
  • Write (mutate): scanners/campaign-write-cli.mjsinit / add / set-status / refresh-tokens, each a thin wrapper over the invariant-enforcing lib transforms + save. Invoked only after the user approves a specific action. refresh-tokens is the live cross-repo token sweep: it runs the manifest's always-loaded accounting across every tracked repo and folds the result into the machine-wide token bill (shared global layer counted once
    • per-repo deltas).
  • Export: scanners/campaign-export-cli.mjs--repo <path> resolves the repo's linked session, reads its action-plan.md, and assembles a docs/config-audit-plan-<sessionId>.md. Read-only (a preview) by default; it writes the file only under --write, which is invoked only after the user approves. The CLI writes the file byte-faithfully — the plan is never re-typed.

All take --ledger-file <path> (defaults to the durable path) and --output-file <path>; the write-CLI + export-CLI also take --reference-date <YYYY-MM-DD> (the date stamp), and the export-CLI takes --sessions-dir <path> (defaults to ~/.claude/config-audit/sessions).

Implementation

Step 1: Parse arguments

From $ARGUMENTS, pick the mode:

  • (empty) or reportreport (read-only). Default.
  • init → initialize the ledger.
  • add <path>... → add one or more repo paths.
  • add --discover <root> → find git repos under <root> and let the user pick which to add.
  • set-status <path> <status> → transition a tracked repo (status ∈ pending/audited/planned/implemented).
  • refresh-tokens → live cross-repo token sweep: compute the machine-wide always-loaded bill.
  • export <path> → export a planned repo's action plan into that repo's own docs/.
  • help → show this surface and stop.

Set a shared date stamp for any write: TODAY=$(date +%F).

Step 2: Always report current state first

Whatever the mode, start by showing where the campaign stands (read-only):

node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-cli.mjs \
  --output-file ~/.claude/config-audit/sessions/campaign-report.json 2>/dev/null; echo $?

Exit 0 = a campaign exists, 1 = not initialized yet (advisory — normal first run), 3 = real error → "The campaign ledger couldn't be read — it may be corrupt." (Stop; do not attempt a write over a corrupt ledger.)

Read ~/.claude/config-audit/sessions/campaign-report.json with the Read tool (per the UX rules — never show the raw JSON). It has initialized, repos[] (each: path, name, status, sessionId, findingsBySeverity, tokens, updatedDate), rollUp {totalRepos, byStatus, bySeverity, reposWithFindings, tokens}, and backlog[] — the single cross-repo prioritized work list (each: path, name, status, findingsBySeverity, totalFindings, weightedScore, rank), already sorted DESC by severity (most critical work first).

Present it as two short tables:

Campaign roll-up

Status Repos
pending / audited / planned / implemented

…plus a one-line severity total across audited repos (e.g. "Findings so far: 3 critical, 8 high, 5 medium, 12 low across 4 audited repos").

Repos

Repo Status Findings (C/H/M/L) Last updated

Prioritized backlog — the one cross-repo list to pick from, highest-severity work first. Render backlog[] in rank order (it is already sorted); omit this table entirely when the backlog is empty (nothing outstanding — say "Backlog clear — no outstanding findings across tracked repos."). Implemented repos and repos with no known findings are deliberately absent.

# Repo Status Findings (C/H/M/L) Total

After it, point the user at the top item: "Highest priority: <name> (<status>) — pick it with /config-audit (audit), /config-audit plan, or /config-audit implement in that repo, then record progress here with set-status." The backlog is a pick-list, not an executor — this command does not run audits or fixes (that is the later execution block).

Machine-wide token bill — render from rollUp.tokens (the whole-machine always-loaded accounting). Note the shape: sharedGlobal, perRepoDelta, and machineWide are flat {always, onDemand, external, unknown} number maps; byRepo[] is {name, path, always, onDemand, external} already sorted DESC by always-loaded cost; reposWithTokens is the count.

If reposWithTokens is 0, no sweep has run yet — say: "No token bill yet — run /config-audit campaign refresh-tokens to compute the machine-wide always-loaded cost." and omit the table. Otherwise lead with the headline and the once-vs-delta split:

Always-loaded every turn, machine-wide: ~machineWide.always tokenssharedGlobal.always paid once (global config + installed plugins, in every repo) + perRepoDelta.always across reposWithTokens repos' own project config.

Then the most expensive repos (their per-repo delta — what each adds beyond the shared layer):

# Repo Always-loaded delta
1 <byRepo[0].name> <byRepo[0].always>

Add one plain-language line so the number is actionable, e.g. "The shared global layer is the biggest lever — trim ~/.claude/CLAUDE.md, the global agent listing, or rarely-used plugins to cut cost in every repo at once." The bill reflects the last sweep; re-run refresh-tokens after config changes.

If initialized is false, say so plainly: "No campaign yet. Run /config-audit campaign init to start one." Then — if the mode was init or add — continue to that step (those bootstrap a campaign); for report/set-status on an uninitialized ledger, stop after this message.

Step 3 (mode init): Initialize

If already initialized, say so and stop (no clobber). Otherwise tell the user what will happen, then create it:

# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs init \
  --reference-date "$TODAY" \
  --output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?

Exit 0 = created, 1 = already initialized (advisory). Confirm: "Campaign ledger created at ~/.claude/config-audit/campaign-ledger.json." Then suggest add.

Step 4 (mode add): Add repos — propose, approve, write

Gather candidates.

  • Explicit paths: use the paths given after add.
  • --discover <root>: find git repos (depth-limited), e.g.
    find "<root>" -maxdepth 3 -type d -name .git 2>/dev/null | sed 's:/\.git$::'
    
    Present the discovered repos as a numbered list and ask which to add (and confirm any that are already tracked will be skipped). Use Glob as a fallback if find is unavailable.

Confirm, then write. Show the final list and ask for explicit approval. On approval, add them in one call (idempotent — already-tracked repos are skipped, not reset):

# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs add <path1> <path2> ... \
  --reference-date "$TODAY" \
  --output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?

(For a single repo with a custom display name, add --name "<name>".) Read the result file and report what was added vs skipped, then re-show the repo table.

Step 5 (mode set-status): Transition a repo — propose, approve, write

Confirm the repo is tracked (from Step 2's report) and that status is one of pending/audited/planned/implemented. State the transition ("<name>: pending → audited") and ask for approval.

When marking a repo audited, optionally attach its findings-by-severity so the machine-wide roll-up stays meaningful. Two honest sources, in order of preference:

  1. If the repo was audited in a config-audit session, read that session's finding counts and build {"critical":C,"high":H,"medium":M,"low":L} — pass --session <id> too.
  2. Otherwise, use counts the user provides. Never invent counts (Verifiseringsplikt) — if none are available, transition the status without --findings.

On approval:

# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs set-status <path> <status> \
  --reference-date "$TODAY" \
  [--findings '{"critical":0,"high":0,"medium":0,"low":0}'] [--session <id>] \
  --output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?

Exit 3 = invalid status, untracked repo, or no ledger → report the message plainly and do not retry blindly. On success, read the result and re-show the updated roll-up + repo row.

Step 6 (mode refresh-tokens): Sweep tokens machine-wide — propose, approve, write

The token bill in Step 2 reflects the last sweep. refresh-tokens recomputes it: for every tracked repo it runs the manifest's always-loaded accounting and refreshes the machine-wide bill — the shared global layer (global CLAUDE.md + installed plugins + global agents/MCP) counted once, plus each repo's own project-config delta.

It writes only the ledger's token fields (never status or findings), is idempotent (a re-sweep replaces, never accumulates), and skips — never aborts on — any repo that can't be read. Tell the user it will read each tracked repo's live config (a few seconds per repo), then on approval:

# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs refresh-tokens \
  --reference-date "$TODAY" \
  --output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?

Exit 0 = swept (or nothing to sweep — a benign no-op on an empty campaign), 3 = no/corrupt ledger. Read the result: swept[] (repos accounted for), skipped[] (each {path, reason}), and the refreshed rollUp.tokens. Re-render the Machine-wide token bill (Step 2) with the new numbers. If anything was skipped, name those repos plainly so the user knows the bill omits them (honest coverage — Verifiseringsplikt).

Step 7 (mode export): Export a repo's plan to its own docs/ — preview, approve, write

"Planer følger arbeidsstedet": a planned repo's action plan belongs in that repo's docs/, not buried in a session dir. This step copies it there, byte-faithfully.

Preview first (read-only — never writes). The repo must be tracked and have a linked session that carries an action-plan.md (i.e. /config-audit plan has run there). Run without --write:

# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-export-cli.mjs --repo "<path>" \
  --reference-date "$TODAY" \
  --output-file ~/.claude/config-audit/sessions/campaign-export.json 2>/dev/null; echo $?

Exit 0 = previewable, 1 = tracked but not exportable yet, 3 = error (untracked repo, no/corrupt ledger). Read ~/.claude/config-audit/sessions/campaign-export.json with the Read tool.

  • Exit 1 — read problems and guide, then stop (nothing to export):
    • no-session-linked → "<name> has no linked audit session. Link one with /config-audit campaign set-status <path> <status> --session <id>, or audit + plan it first."
    • no-action-plan → "<name>'s session has no plan yet. Run /config-audit plan in that repo first, mark it planned, then export."
  • Exit 0 — show, then ask. Tell the user the destination (targetPath) and a short preview — the first ~12 lines of document only, never the whole file, never the raw JSON (UX rules). Ask for explicit approval to write it.

On approval, write it (the CLI does the faithful copy — do NOT hand-write the file):

# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-export-cli.mjs --repo "<path>" --write \
  --reference-date "$TODAY" \
  --output-file ~/.claude/config-audit/sessions/campaign-export.json 2>/dev/null; echo $?

Confirm: "Plan exported to <targetPath>." Then hand off to the existing execution machinery (Block 4c reuses it — this command does not run it for you):

To execute: run /config-audit implement in <path> — it backs up every changed file, applies the plan, and verifies. To undo: /config-audit rollback. When done, record it: /config-audit campaign set-status <path> implemented.

Step 8: Next steps

Tailor to where the campaign stands:

  • Just initialized / few repos: "/config-audit campaign add --discover ~/repos to enroll your repos."
  • Pending repos exist: "Run /config-audit in a pending repo to audit it, then /config-audit campaign set-status <path> audited to record the result here."
  • Audited but not planned: "/config-audit plan in that repo, then mark it planned."
  • Planned repos exist: "/config-audit campaign export <path> to drop the plan into that repo's own docs/, then /config-audit implement there to execute it (backup + verify)."
  • Backlog has items: point at the top backlog repo and the natural next verb for its status (audit → plan → export → implement).
  • No token bill yet (or config changed): "/config-audit campaign refresh-tokens to compute the machine-wide always-loaded cost — the shared global layer is the biggest lever."
  • Always: the campaign survives this session — re-run /config-audit campaign anytime to see the machine-wide picture.

Notes

  • Read-only report, human-approved writes. campaign-cli never writes; every mutation — ledger changes via campaign-write-cli, plan exports via campaign-export-cli --write — happens only after explicit approval, exactly mirroring how /config-audit knowledge-refresh gates register writes. The export-CLI's default (no --write) is a read-only preview.
  • Deterministic core, not byte-stable command. The lib transforms + all three CLIs are unit-tested and deterministic (--reference-date injected); this command's orchestration is judgment-driven and deliberately not in the snapshot suite (like /config-audit optimize and knowledge-refresh).
  • The -cli suffix keeps all three CLIs out of the scan-orchestrator, so the scanner count and the byte-stable snapshot suite are unaffected.
  • THIN scope: ledger + roll-up (findings + machine-wide token bill) + status + a cross-repo prioritized backlog + plan export. The token sweep reuses the manifest's existing always-loaded accounting per repo — it does not reinvent measurement, only aggregates it machine-wide. Execution is not reinvented here — export drops a planned repo's plan into its own docs/ (a durable record), and the user runs the existing /config-audit implement (backup + apply + verify) + /config-audit rollback to execute and undo. This command tracks state and routes the work; it does not run audits or apply fixes itself.