open/config-audit
1
0
Fork 0
Code Issues Releases 1 Activity

feat(scanners): the recovery path is code you can run, not prose you can read

Browse source 
R1+R2 as one chunk — both KRITISK rows of the Q3 severity table sit on the
restore path, and neither closes alone.

R1: rollback-engine.mjs verified every checksum before AND after each write,
resolved the legacy backup root and reported createdNotRemoved — and none of it
was reachable. Measured: 16 files under scanners/ carry a process.argv entry;
the engine was not one of them. commands/rollback.md drove the restore as model
prose: an ESM import block a template cannot execute, ad-hoc `cp` offered
underneath as the runnable path, and "(checksum verified)" pre-rendered three
times in the success output. `cp` establishes no checksum, so the verification
was a property of the template rather than of the run — on the one surface that
runs when the user is already in trouble.

R2: implement.md Step 3 hand-built its backup (mkdir, cp, a date-derived id, a
manifest typed out in the template) while parseManifest knew one frozen sample
of that format, pinned by a HAND-WRITTEN fixture instead of by the template's
own text. Rename a key and parseManifest returns zero files while rollback
reports success.

Fixing only R1 leaves the new CLI parsing a prose format; fixing only R2 leaves
a clean format with no runnable entry.

- scanners/rollback-cli.mjs — --list / --create / --restore / --delete over the
  existing engine, on the shared requireValidArgs gate. Exit 0 done, 1
  outstanding (gate refusal with nothing written, or a backup that covered fewer
  targets than given), 2 a file failed, 3 could not do the job. A gated restore
  is 1, not 3: "this write leaves your project" is a verdict about a write that
  WAS examined, and it rides in the payload where a command under 2>/dev/null
  can act on it.
- createBackup gains `created` (recorded, never copied — no backup can hold a
  file that does not exist) and `skipped`, so a backup covering fewer files than
  asked is no longer indistinguishable from a clean one.
- implement.md Step 3 and rollback.md now call the CLI. parseManifest's
  implement-format branch stays: nothing writes that shape now, but every backup
  made before this chunk is on disk in it.
- backup-restore-contract.test.mjs checks every field rollback.md renders
  against a payload produced by RUNNING the CLI. That is what replaced
  "(checksum verified)".

20 guards seen red against the original state before any production code, then
each against its own defect. Two holes that surfaced there were mine: the
implement assertion matched `--create` as a substring of `--created` and stayed
green when the call was removed; and mutating the argv gate showed
requireValidArgs sets exit 3 by itself, so a CLI can report that it could not
parse its arguments and still run the restore underneath — that case is now
asserted on the bytes.

Suite 1752 -> 1777, 0 fail. Frozen tests/snapshots/v5.0.0 untouched. Dogfooded
through the templates' own command lines against a sandboxed HOME, including the
machine-wide arm: refused with the file unchanged, then restored under
--approve-scope.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Logq8GGWKhtyDem63FTEnG
This commit is contained in:
Kjell Tore Guttormsen 2026-08-18 21:28:15 +02:00
commit 44b222859e
13 changed files with 1083 additions and 86 deletions
Download patch file Download diff file Expand all files Collapse all files

51
commands/implement.md
View file

@ -87,37 +87,32 @@ AskUserQuestion:
### Step 3: Create backup
Create backup silently, and **print the backup ID** — Step 6 has to tell the user
how to roll back, and a timestamp that only ever existed inside a command
substitution cannot be quoted later. Shell state does not survive to the next
block, so capture the printed value and substitute it literally from here on:
Create the backup through the code that owns the format. Pass one `--target` per
pre-existing file the plan will MODIFY, and one `--created` per file the plan
will CREATE — a backup cannot hold a file that does not exist yet, so those are
recorded rather than copied, and `/config-audit rollback` reads them back to tell
the user which files it is leaving behind.
```bash
BACKUP_ID=$(date +%Y%m%d_%H%M%S)
mkdir -p ~/.claude/config-audit/backups/"$BACKUP_ID"/files/ 2>/dev/null
echo "$BACKUP_ID"
node ${CLAUDE_PLUGIN_ROOT}/scanners/rollback-cli.mjs --create --target "<file-1>" --target "<file-2>" --created "<new-file-1>" --repo "$PWD" --output-file /tmp/config-audit-implement-backup.json 2>/dev/null; echo $?
```
Use the printed ID wherever `{backup-id}` appears below. Never invent or re-derive
it with a second `date` call — a run that straddles a second boundary would hand
the user a rollback ID that does not exist.
| Exit | Meaning |
|------|---------|
| 0 | every target was backed up |
| 1 | at least one target was not there — read `skipped[]` before continuing |
| 3 | the CLI could not run (show the stderr message); do not edit anything |
Copy each file to be modified. Generate `manifest.yaml` with checksums.
Read `/tmp/config-audit-implement-backup.json`. The payload's `backupId` is the
ID to quote from here on — **never re-derive it**. Shell state does not survive
to the next block, and a second `date` call that straddles a second boundary
would hand the user a rollback ID that does not exist. Use it wherever
`{backup-id}` appears below.
The manifest is what `/config-audit rollback` reads, so it MUST carry both lists:
```yaml
files: # pre-existing files this run will MODIFY
- backup: files/root/CLAUDE.md
original: /abs/path/CLAUDE.md
sha256: <sha256 of the pre-change content>
created: # files this run will CREATE (no backup can exist)
- /abs/path/.claude/rules/post-quality.md
```
Record every `create`-type action under `created:`. Rollback cannot restore a
file that never existed, but it must be able to tell the user which files it is
leaving behind — a half-restored target is only dangerous when it is silent.
On exit 1, name the `skipped[]` paths before doing anything else: the plan is
about to change files that have no backup behind them. If any skipped path is one
the plan MODIFIES (rather than creates), stop and report — that action cannot be
rolled back.
Tell the user: **"Backup created. Implementing actions..."**
@ -222,8 +217,10 @@ A full-file Write that names only two of the four silently deletes the other two
## Rollback
If the user requests rollback at any point:
1. Read `manifest.yaml` from backup
2. Restore each file and verify checksums
1. Run `/config-audit rollback {backup-id}` — it drives `scanners/rollback-cli.mjs`,
which verifies each checksum before and after writing. Do not restore by hand:
a copy performs neither check, and the result cannot be reported as verified.
2. Read the restore payload and report the per-file `status` it returns
3. **Report — do not delete — the files this run created.** Rollback restores from
backup, and no backup can exist for a file that did not exist before. Those
paths stay on disk; `/config-audit rollback` lists them under "Left in place"

119
commands/rollback.md
View file

@ -10,6 +10,12 @@ model: sonnet
Restore configuration files from a previous backup. Without arguments, lists available backups. With a backup ID, restores files from that backup.
Every step below runs `scanners/rollback-cli.mjs`, which drives the rollback
engine: it verifies each file's checksum before AND after writing it, resolves
backups made under the pre-v2.2.0 root, and reports the files a restore cannot
undo. Never restore by copying files back by hand — a copy performs none of
those checks, and the result cannot honestly be reported as verified.
## Arguments
- `$ARGUMENTS` may contain a backup ID (format: `YYYYMMDD_HHMMSS`)
@ -19,14 +25,18 @@ Restore configuration files from a previous backup. Without arguments, lists ava
### List mode (no argument)
Parse flags and list available backups from `~/.claude/config-audit/backups/`:
```bash
RAW_FLAG=""
if echo "$ARGUMENTS" | grep -q -- "--raw"; then RAW_FLAG="--raw"; fi
ls -1 ~/.claude/config-audit/backups/
node ${CLAUDE_PLUGIN_ROOT}/scanners/rollback-cli.mjs --list --output-file /tmp/config-audit-rollback-list.json 2>/dev/null; echo $?
```
Exit 0 = listed; 3 = the CLI could not do its job (show the stderr message).
Read `/tmp/config-audit-rollback-list.json` and render one row per entry in
`backups[]` — `{id}`, how many `{files}` it holds, and `{createdAt}`. An entry
whose `{legacy}` is true was made under the pre-v2.2.0 backup root; say so, since
its path differs from the one printed below.
```
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Available Backups
@ -40,16 +50,16 @@ ls -1 ~/.claude/config-audit/backups/
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
```
Use the Read tool on each backup's `manifest.yaml` (the list of changes captured at backup time) to extract the file list and timestamps.
If `{count}` is 0, say there are no backups yet and stop — do not offer a restore.
### Restore mode (with backup ID)
1. Read the list of changes from `~/.claude/config-audit/backups/{backup-id}/manifest.yaml` using the Read tool
2. Classify the `original:` paths before showing them. A restore writes to the
absolute path recorded at backup time, which may be machine-wide even when the
backup was taken from a project — so the file list must be rendered as the
absolute originals, never shortened to a repo-relative-looking form that
implies the write stays local:
1. The `backups[]` entry for that ID (from the list payload above) carries the
`files[]` this restore would write. Classify those `{originalPath}` values
before showing them. A restore writes to the absolute path recorded at backup
time, which may be machine-wide even when the backup was taken from a project
— so the file list must be rendered as the absolute originals, never shortened
to a repo-relative-looking form that implies the write stays local:
```bash
node ${CLAUDE_PLUGIN_ROOT}/scanners/write-scope-cli.mjs --target "<original-1>" --target "<original-2>" --repo "$PWD" --output-file /tmp/config-audit-rollback-scope.json 2>/dev/null; echo $?
@ -76,21 +86,45 @@ Use the Read tool on each backup's `manifest.yaml` (the list of changes captured
- "Cancel"
- "Yes — restore, including outside this project"
```
3. For each file in the list of changes:
a. Read the backup file from `~/.claude/config-audit/backups/{backup-id}/files/{safeName}`
b. Write to the original path
c. Verify the checksum matches the recorded value in the list of changes
4. Show result:
2. Run the restore. Classifying is not approving: add `--approve-scope` only
after the user has answered yes to the question above, and only then.
```bash
node ${CLAUDE_PLUGIN_ROOT}/scanners/rollback-cli.mjs --restore "<backup-id>" --repo "$PWD" --output-file /tmp/config-audit-rollback-restore.json 2>/dev/null; echo $?
```
Restored 3 files from backup 20260403_163045
- /abs/path/.claude/settings.json (checksum verified)
- /abs/path/hooks/hooks.json (checksum verified)
- .claude/rules/typescript.md (checksum verified)
Append ` --approve-scope` to that command when the user approved a restore
that leaves this project. To preview without writing anything, append
` --dry-run` instead — a dry run reports what would happen and touches no file.
| Exit | Meaning |
|------|---------|
| 0 | every file restored |
| 1 | nothing was written — the restore needs approval it was not given |
| 2 | at least one file failed; read `failed[]` before saying anything else |
| 3 | the CLI could not run (bad ID, unreadable manifest) — show the stderr message |
3. Read `/tmp/config-audit-rollback-restore.json` and report what the run
actually did. Render one line per entry in `restored[]` and `failed[]`, each
showing its own `{status}` from the payload — never a fixed verification
phrase, because the outcome differs per file and only the payload knows it:
```
5. **Report what rollback cannot undo.** A backup only holds files that already
existed, so files the implement step CREATED survive the restore. If the
manifest has a `created:` section (or `restoreBackup()` returns a non-empty
`createdNotRemoved`), list those paths and say plainly that they remain:
Restored 2 of 3 files from backup 20260403_163045
- /abs/path/.claude/settings.json — {status}
- /abs/path/hooks/hooks.json — {status}
- /abs/path/.claude/rules/typescript.md — {status}
```
When `{requiresApproval}` is true and nothing was restored, the run was
refused: list the `refused[]` paths, say plainly that no file was changed, and
offer to re-run with approval.
4. **Report what rollback cannot undo.** A backup only holds files that already
existed, so files the implement step CREATED survive the restore. When
`createdNotRemoved` is non-empty, list those paths and say plainly that they
remain:
```
Left in place — created by implement, no backup exists:
- .claude/rules/post-quality.md
@ -102,29 +136,24 @@ Use the Read tool on each backup's `manifest.yaml` (the list of changes captured
### Delete mode
If user says "delete" after listing, confirm and remove the backup directory.
If the user says "delete" after listing, confirm, then:
```bash
node ${CLAUDE_PLUGIN_ROOT}/scanners/rollback-cli.mjs --delete "<backup-id>" --output-file /tmp/config-audit-rollback-delete.json 2>/dev/null; echo $?
```
Exit 0 = removed (`deleted` is true in the payload); 3 = no backup with that ID
in either root — show the stderr message rather than reporting a deletion.
## Implementation
Use the backup and rollback libraries directly:
```javascript
import { listBackups, restoreBackup, deleteBackup } from '../scanners/rollback-engine.mjs';
import { parseManifest, getBackupDir } from '../scanners/lib/backup.mjs';
```
`scanners/rollback-cli.mjs` is the only entry point. It reads
`~/.claude/config-audit/backups` and falls back to the pre-v2.2.0
`~/.config-audit/backups`, so a backup made before the move still resolves; the
list payload flags those with `legacy: true`, and both roots are echoed in
`meta` so a report can name the one it used.
Both read `~/.claude/config-audit/backups` and fall back to the pre-v2.2.0
`~/.config-audit/backups`, so a backup made before the move still resolves;
`listBackups()` flags those with `legacy: true`. Prefer this API over ad-hoc
`cp` — it verifies the checksum before and after each write.
Or via Bash:
```bash
# List backups
ls -1 ~/.claude/config-audit/backups/
# Read manifest
cat ~/.claude/config-audit/backups/{id}/manifest.yaml
# Restore (copy back)
cp ~/.claude/config-audit/backups/{id}/files/{safeName} {originalPath}
```
The same CLI creates backups (`--create --target <path> …`), which is how
`/config-audit implement` records what it is about to change. Backup and restore
therefore share one manifest format, owned by `scanners/lib/backup.mjs` — the
format is never written or read by hand.