open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity
config-audit/commands/campaign.md
Kjell Tore Guttormsen 0b763f25c1 fix(commands): router dogfood — five seam defects, plus the placeholder class 
Dogfooding `/config-audit` (the router) against the repo, fasit written before
any run (docs/router-fasit.local.md, untouched). Every claim below is measured
behaviour, not a reading of the source.

1. Bare `<target-path>` inside the step-3 fence is a shell REDIRECTION, not an
   argument. Measured in zsh: both CLIs failed before starting, no output file
   was written, and the echoed status was 1 — inside the band the router's own
   gate calls "continue normally". Quoting makes an unsubstituted placeholder
   reach argv, so it fails in the CLI where the exit code means something.
   Swept the whole class: 30 sites across 12 further command files, since a
   defect in one file is a class until the opposite is measured. New guard:
   command-placeholder-shell-safety.test.mjs.

2. The orchestrator's exit code was discarded. Two commands on one line share a
   single trailing `echo $?`, which reports only the last: measured, an
   orchestrator exit 3 echoed as posture's 0, so the "3 -> stop" gate could
   never fire. Both statuses are now captured and echoed.

3. "Running 12 configuration scanners" — the orchestrator registers 16. The new
   test binds the narrated count to the registry so the next scanner added
   cannot re-stale it silently.

4. The Area Breakdown table hardcoded 7 rows; posture emits 9 quality areas.
   Token Efficiency (a B on this repo) and Plugin Hygiene never reached the
   user. Rows added, and the row set is now asserted against lib/scoring.mjs.
   Label aligned: "MCP Servers" -> "MCP", as posture emits it.

5. Step 6 rendered "the headline line from the humanized stderr scorecard" and
   forbade deriving a replacement — while step 3 sent posture's stderr to
   /dev/null, as UX rule 2 requires, and the prose is absent from the JSON
   payload (measured). The slot could only be improvised. posture's stderr now
   goes to a file in the session dir, as commands/posture.md already did; the
   user still never sees raw scanner output.

Also: `grep -q -- "--raw"` matched any argument CONTAINING --raw (measured on
`--rawdog` and on a path with --raw in it) — anchored to whole arguments.
SCOPE_FLAGS renamed SCOPE_FLAG, since zsh does not word-split and the plural
invited the M-BUG-45 shape.

command-shell-state-shape.test.mjs only recognised line-initial assignments, so
it reported the idiomatic `node …; STATUS=$?` capture as never assigned. Widened
to assignments after a separator; verified it still fails on a real cross-block
reference before trusting it.

Suite 1477 -> 1483, frozen v5.0.0 snapshots untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDAwy1ZXRpZxht1wyCeSbF
2026-08-09 21:18:05 +02:00

332 lines
18 KiB
Markdown
Raw Blame History

---
name: config-audit:campaign
description: Machine-wide audit campaign — track which repos are pending/audited/planned/implemented across sessions, with a machine-wide roll-up. Human-approved writes only.
argument-hint: "[init | add <path>... | set-status <path> <status> | refresh-tokens | export <path>]"
allowed-tools: Read, Write, Edit, Bash, Glob
model: opus
---
# Config-Audit: Campaign
A single config-audit session audits **one** scope. A **campaign** sits above sessions: a
durable ledger of every repo you mean to bring up to standard, each repo's lifecycle status
(**pending → audited → planned → implemented**), and a machine-wide roll-up of findings by
severity. It persists to `~/.claude/config-audit/campaign-ledger.json` — **outside** the
plugin dir, next to `sessions/` — so it survives plugin uninstall/reinstall/upgrade and
resumes across sessions.
**The Iron rule (Verifiseringsplikt): nothing is ever auto-written.** Reporting is read-only.
Every mutation — creating the ledger, adding a repo, changing a status — is proposed first
and applied **only on explicit approval**, by invoking one deterministic write-CLI subcommand.
The command never hand-edits the ledger JSON.
This is the **THIN** campaign surface (ledger + roll-up + status + a cross-repo prioritized
backlog to pick from + plan **export**). It does not run audits or apply fixes itself: it tracks
where each repo stands, shows what to tackle next, exports a planned repo's plan into that repo's
own `docs/`, and points at the **existing** per-repo `/config-audit implement` (backup + apply +
verify) and `/config-audit rollback` for execution — Block 4c reuses that machinery, it does not
reinvent it.
## Three CLIs back this command
- **Read (report):** `scanners/campaign-cli.mjs` — loads + validates the ledger, emits the
repo list + roll-up + backlog. Never writes.
- **Write (mutate):** `scanners/campaign-write-cli.mjs` — `init` / `add` / `set-status` /
`refresh-tokens`, each a thin wrapper over the invariant-enforcing lib transforms + save.
Invoked **only** after the user approves a specific action. `refresh-tokens` is the live
cross-repo token sweep: it runs the manifest's always-loaded accounting across every tracked
repo and folds the result into the machine-wide token bill (shared global layer counted once
+ per-repo deltas).
- **Export:** `scanners/campaign-export-cli.mjs` — `--repo <path>` resolves the repo's linked
session, reads its `action-plan.md`, and assembles a `docs/config-audit-plan-<sessionId>.md`.
Read-only (a preview) by default; it writes the file **only** under `--write`, which is
invoked **only** after the user approves. The CLI writes the file byte-faithfully — the plan is
never re-typed.
All take `--ledger-file <path>` (defaults to the durable path) and `--output-file <path>`; the
write-CLI + export-CLI also take `--reference-date <YYYY-MM-DD>` (the date stamp), and the
export-CLI takes `--sessions-dir <path>` (defaults to `~/.claude/config-audit/sessions`).
## Implementation
### Step 1: Parse arguments
From `$ARGUMENTS`, pick the mode:
- *(empty)* or `report` → **report** (read-only). Default.
- `init` → initialize the ledger.
- `add <path>...` → add one or more repo paths.
- `add --discover <root>` → find git repos under `<root>` and let the user pick which to add.
- `set-status <path> <status>` → transition a tracked repo (`status` ∈ pending/audited/planned/implemented).
- `refresh-tokens` → live cross-repo token sweep: compute the machine-wide always-loaded bill.
- `export <path>` → export a planned repo's action plan into that repo's own `docs/`.
- `help` → show this surface and stop.
Every write step derives its own date stamp inside its own block — there is no shared one to
set here, because each fenced block runs as a separate process.
### Step 2: Always report current state first
Whatever the mode, start by showing where the campaign stands (read-only):
```bash
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-cli.mjs \
--output-file ~/.claude/config-audit/sessions/campaign-report.json 2>/dev/null; echo $?
```
Exit **0** = a campaign exists, **1** = not initialized yet (advisory — normal first run),
**3** = real error → "The campaign ledger couldn't be read — it may be corrupt." (Stop; do not
attempt a write over a corrupt ledger.)
Read `~/.claude/config-audit/sessions/campaign-report.json` with the Read tool (per the UX
rules — never show the raw JSON). It has `initialized`, `repos[]` (each: `path, name, status,
sessionId, findingsBySeverity, tokens, updatedDate`), `rollUp {totalRepos, byStatus, bySeverity,
reposWithFindings, tokens}`, and `backlog[]` — the single cross-repo prioritized work list (each:
`path, name, status, findingsBySeverity, totalFindings, weightedScore, rank`), already sorted
DESC by severity (most critical work first).
Present it as two short tables:
**Campaign roll-up**
| Status | Repos |
|--------|-------|
| pending / audited / planned / implemented | … |
…plus a one-line severity total across audited repos (e.g. "Findings so far: 3 critical,
8 high, 5 medium, 12 low across 4 audited repos").
**Repos**
| Repo | Status | Findings (C/H/M/L) | Last updated |
|------|--------|--------------------|--------------|
**Prioritized backlog** — the one cross-repo list to pick from, highest-severity work first.
Render `backlog[]` in `rank` order (it is already sorted); omit this table entirely when the
backlog is empty (nothing outstanding — say "Backlog clear — no outstanding findings across
tracked repos."). Implemented repos and repos with no known findings are deliberately absent.
| # | Repo | Status | Findings (C/H/M/L) | Total |
|---|------|--------|--------------------|-------|
After it, point the user at the top item: "Highest priority: **`<name>`** (`<status>`) — pick it
with `/config-audit` (audit), `/config-audit plan`, or `/config-audit implement` in that repo,
then record progress here with `set-status`." The backlog is a **pick-list**, not an executor —
this command does not run audits or fixes (that is the later execution block).
**Machine-wide token bill** — render from `rollUp.tokens` (the whole-machine always-loaded
accounting). Note the shape: `sharedGlobal`, `perRepoDelta`, and `machineWide` are flat
`{always, onDemand, external, unknown}` number maps; `byRepo[]` is `{name, path, always,
onDemand, external}` already sorted DESC by always-loaded cost; `reposWithTokens` is the count.
If `reposWithTokens` is `0`, no sweep has run yet — say: "No token bill yet — run
`/config-audit campaign refresh-tokens` to compute the machine-wide always-loaded cost." and
omit the table. Otherwise lead with the headline and the once-vs-delta split:
> **Always-loaded every turn, machine-wide: ~`machineWide.always` tokens** — `sharedGlobal.always`
> paid once (global config + installed plugins, in *every* repo) + `perRepoDelta.always` across
> `reposWithTokens` repos' own project config.
Then the **most expensive repos** (their per-repo delta — what each adds beyond the shared layer):
| # | Repo | Always-loaded delta |
|---|------|---------------------|
| 1 | `<byRepo[0].name>` | `<byRepo[0].always>` |
Add one plain-language line so the number is actionable, e.g. "The shared global layer is the
biggest lever — trim `~/.claude/CLAUDE.md`, the global agent listing, or rarely-used plugins
to cut cost in every repo at once." The bill reflects the **last** sweep; re-run
`refresh-tokens` after config changes.
If `initialized` is false, say so plainly: "No campaign yet. Run `/config-audit campaign init`
to start one." Then — if the mode was `init` or `add` — continue to that step (those bootstrap
a campaign); for `report`/`set-status` on an uninitialized ledger, stop after this message.
### Step 3 (mode `init`): Initialize
If already initialized, say so and stop (no clobber). Otherwise tell the user what will happen,
then create it:
```bash
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs init \
--reference-date "$TODAY" \
--output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?
```
Exit **0** = created, **1** = already initialized (advisory). Confirm: "Campaign ledger created
at `~/.claude/config-audit/campaign-ledger.json`." Then suggest `add`.
### Step 4 (mode `add`): Add repos — propose, approve, write
**Gather candidates.**
- Explicit paths: use the paths given after `add`.
- `--discover <root>`: find git repos (depth-limited), e.g.
```bash
find "<root>" -maxdepth 3 -type d -name .git 2>/dev/null | sed 's:/\.git$::'
```
Present the discovered repos as a numbered list and ask **which** to add (and confirm any
that are already tracked will be skipped). Use Glob as a fallback if `find` is unavailable.
**Confirm, then write.** Show the final list and ask for explicit approval. On approval, add
them in one call (idempotent — already-tracked repos are skipped, not reset):
```bash
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs add "<path1>" "<path2>" ... \
--reference-date "$TODAY" \
--output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?
```
(For a single repo with a custom display name, add `--name "<name>"`.) Read the result file and
report what was `added`, `addedUnverified`, and `skipped`, then re-show the repo table.
`addedUnverified[]` holds paths that were tracked but could **not** be read right now (they do
not exist, or are not directories). They are tracked deliberately — an unmounted volume is a
legitimate reason for a repo to be missing today — but they must be named, not glossed over:
"Tracked, but I couldn't read `<path>` — check for a typo, or mount it before the next token
sweep." An unreported phantom row stays in the backlog forever and quietly widens every
machine-wide total.
### Step 5 (mode `set-status`): Transition a repo — propose, approve, write
Confirm the repo is tracked (from Step 2's report) and that `status` is one of
pending/audited/planned/implemented. State the transition ("`<name>`: pending → audited") and
ask for approval.
When marking a repo **audited**, optionally attach its findings-by-severity so the machine-wide
roll-up stays meaningful. Two honest sources, in order of preference:
1. If the repo was audited in a config-audit session, read that session's finding counts and
build `{"critical":C,"high":H,"medium":M,"low":L}` — pass `--session <id>` too.
2. Otherwise, use counts the user provides. **Never invent counts** (Verifiseringsplikt) — if
none are available, transition the status without `--findings`.
On approval:
```bash
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs set-status "<path>" "<status>" \
--reference-date "$TODAY" \
[--findings '{"critical":0,"high":0,"medium":0,"low":0}'] [--session "<id>"] \
--output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?
```
Exit **3** = invalid status, untracked repo, or no ledger → report the message plainly and do
not retry blindly. On success, read the result and re-show the updated roll-up + repo row.
### Step 6 (mode `refresh-tokens`): Sweep tokens machine-wide — propose, approve, write
The token bill in Step 2 reflects the **last** sweep. `refresh-tokens` recomputes it: for every
tracked repo it runs the manifest's always-loaded accounting and refreshes the machine-wide bill —
the shared global layer (global CLAUDE.md + installed plugins + global agents/MCP) counted **once**,
plus each repo's own project-config delta.
It writes only the ledger's token fields (never status or findings), is idempotent (a re-sweep
replaces, never accumulates), and **skips — never aborts on** — any repo that can't be read. Tell
the user it will read each tracked repo's live config (a few seconds per repo), then on approval:
```bash
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-write-cli.mjs refresh-tokens \
--reference-date "$TODAY" \
--output-file ~/.claude/config-audit/sessions/campaign-write.json 2>/dev/null; echo $?
```
Exit **0** = swept (or nothing to sweep — a benign no-op on an empty campaign), **3** = no/corrupt
ledger. Read the result: `swept[]` (repos accounted for), `skipped[]` (each `{path, reason}`), and
the refreshed `rollUp.tokens`. Re-render the **Machine-wide token bill** (Step 2) with the new
numbers. If anything was skipped, name those repos plainly so the user knows the bill omits them
(honest coverage — Verifiseringsplikt).
### Step 7 (mode `export`): Export a repo's plan to its own `docs/` — preview, approve, write
"Planer følger arbeidsstedet": a planned repo's action plan belongs in **that repo's** `docs/`,
not buried in a session dir. This step copies it there, byte-faithfully.
**Preview first (read-only — never writes).** The repo must be tracked and have a linked session
that carries an `action-plan.md` (i.e. `/config-audit plan` has run there). Run without `--write`:
```bash
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-export-cli.mjs --repo "<path>" \
--reference-date "$TODAY" \
--output-file ~/.claude/config-audit/sessions/campaign-export.json 2>/dev/null; echo $?
```
Exit **0** = previewable, **1** = tracked but not exportable yet, **3** = error (untracked repo,
no/corrupt ledger). Read `~/.claude/config-audit/sessions/campaign-export.json` with the Read tool.
- **Exit 1 — read `problems`** and guide, then stop (nothing to export):
- `no-session-linked` → "`<name>` has no linked audit session. Link one with
`/config-audit campaign set-status <path> <status> --session <id>`, or audit + plan it first."
- `no-action-plan` → "`<name>`'s session has no plan yet. Run `/config-audit plan` in that repo
first, mark it `planned`, then export."
- **Exit 0 — show, then ask.** Tell the user the destination (`targetPath`) and a **short** preview
— the first ~12 lines of `document` only, never the whole file, never the raw JSON (UX rules).
Ask for explicit approval to write it.
**On approval, write it** (the CLI does the faithful copy — do NOT hand-write the file):
```bash
# Re-derive here: each fenced block is its own Bash call, so a TODAY set
# in an earlier block is empty by the time this one runs.
TODAY=$(date +%F)
node ${CLAUDE_PLUGIN_ROOT}/scanners/campaign-export-cli.mjs --repo "<path>" --write \
--reference-date "$TODAY" \
--output-file ~/.claude/config-audit/sessions/campaign-export.json 2>/dev/null; echo $?
```
Confirm: "Plan exported to `<targetPath>`." Then hand off to the **existing** execution machinery
(Block 4c reuses it — this command does not run it for you):
> To **execute**: run `/config-audit implement` in `<path>` — it backs up every changed file,
> applies the plan, and verifies. To **undo**: `/config-audit rollback`. When done, record it:
> `/config-audit campaign set-status <path> implemented`.
### Step 8: Next steps
Tailor to where the campaign stands:
- **Just initialized / few repos:** "`/config-audit campaign add --discover ~/repos` to enroll
your repos."
- **Pending repos exist:** "Run `/config-audit` in a pending repo to audit it, then
`/config-audit campaign set-status <path> audited` to record the result here."
- **Audited but not planned:** "`/config-audit plan` in that repo, then mark it `planned`."
- **Planned repos exist:** "`/config-audit campaign export <path>` to drop the plan into that
repo's own `docs/`, then `/config-audit implement` there to execute it (backup + verify)."
- **Backlog has items:** point at the top backlog repo and the natural next verb for its status
(audit → plan → export → implement).
- **No token bill yet (or config changed):** "`/config-audit campaign refresh-tokens` to compute
the machine-wide always-loaded cost — the shared global layer is the biggest lever."
- Always: the campaign survives this session — re-run `/config-audit campaign` anytime to see
the machine-wide picture.
## Notes
- **Read-only report, human-approved writes.** `campaign-cli` never writes; every mutation —
ledger changes via `campaign-write-cli`, plan exports via `campaign-export-cli --write` — happens
only after explicit approval, exactly mirroring how `/config-audit knowledge-refresh` gates
register writes. The export-CLI's default (no `--write`) is a read-only preview.
- **Deterministic core, not byte-stable command.** The lib transforms + all three CLIs are
unit-tested and deterministic (`--reference-date` injected); this command's orchestration is
judgment-driven and deliberately **not** in the snapshot suite (like `/config-audit optimize`
and `knowledge-refresh`).
- The `-cli` suffix keeps all three CLIs out of the scan-orchestrator, so the scanner count and
the byte-stable snapshot suite are unaffected.
- **THIN scope:** ledger + roll-up (findings + machine-wide token bill) + status + a cross-repo
prioritized backlog + plan export. The token sweep reuses the manifest's existing always-loaded
accounting per repo — it does not reinvent measurement, only aggregates it machine-wide.
Execution is **not** reinvented here — `export` drops a planned repo's plan into its own `docs/`
(a durable record), and the user runs the existing `/config-audit implement` (backup + apply +
verify) + `/config-audit rollback` to execute and undo. This command tracks state and routes the
work; it does not run audits or apply fixes itself.
Reference in a new issue View git blame Copy permalink