A command template is a caller with no compiler behind it. It names a scanner and
an argv; nothing checked that the scanner still accepts them. M-BUG-45 measured
what that costs: `--stale-after` arrived malformed, was ignored, and the command
reported "all 14 entries re-verified within the last 90 days" about a threshold
the user had just overridden.
The new guard builds the argv from each template's OWN text (#63 — a hand-typed
call is a path no user takes), reading all three forms a flag appears in,
including the comment-only `GLOBAL_FLAG="" # --global`; that third form is the
one that dies unobserved, since the default path leaves the variable empty.
Measured: 38 invocations, 54 (CLI, flag) pairs, 15 CLIs, 0 dead scanner paths.
Two premises in the plan text were falsified by measuring:
- "the flag exists in the CLI's BOOL_FLAGS/VALUE_FLAGS" — only 3 of 34 scanner
files declare such a surface. The contract is checked on BEHAVIOUR instead:
run the CLI, ask whether it calls the flag unknown.
- `--full-machine` was predicted dead on `posture`. It is live. The fasit was
wrong, not the code.
What the measurement found instead: `campaign-export-cli` was the only one of the
fifteen without the shared `requireValidArgs` gate. Its hand-rolled chain guards
every value branch with `argv[i + 1] !== undefined`, so a trailing `--repo` fell
past all of them to the `startsWith('--')` catch-all and was reported as an
unknown flag — about the flag the CLI itself requires. Classification of "value
flag, no value" across all fifteen: 14 correct, 1 wrong. It now uses ARG_SPEC +
requireValidArgs like the other twelve; valid argv reaches the existing loop
byte-for-byte unchanged. Special-casing it in the test would have rebuilt, in
test code, the prose exception Q1 deleted.
And what the guard itself got wrong, which is worse than what it was looking for:
probing a flag means RUNNING the CLI, and some flags are writers. Its first run
let `drift-cli --save` default its target to the working directory and overwrite
the operator's real ~/.config-audit/baselines/default.json — an ungated write
outside the repo, produced by the guard whose whole subject is ungated writes
outside the repo. Every probe now runs under hermeticEnv() with its own empty
cwd, and the cwd is asserted empty afterwards. Isolation that is only a
convention is not isolation. Side effect: 65s -> 13s, because a hermetic HOME
stops every probe from enumerating ~/.claude.
All six arms seen RED against their own defect, twice — including the ORIGINAL
class (remove --approve-scope from fix-cli) and the plan's own verification
(delete the write-scope-cli line from a template). The non-emptiness arm is
derived from the tree, not pinned to a count that would only be a drift point.
Suite 1707 -> 1724, frozen v5.0.0 + default-output snapshots 0 changed files.
Not fixed here, found while verifying and pre-existing at 749b710: the suite was
NOT green on HEAD. output-file-robustness fails on drift-cli, root cause
diff-engine.mjs:194 — `m.from.severity` where `m.from` is undefined in the moved
section of the drift report. It crashes after the scan, in formatting, so the
CLI exits 3 with no output file. Its own chunk, not this one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pj6UoTi6iPsAB2B2j6EZ1k
146 lines
5.1 KiB
JavaScript
146 lines
5.1 KiB
JavaScript
/**
|
|
* Extract the CLI invocations a command template tells the agent to run —
|
|
* from the template's OWN text, never hand-written.
|
|
*
|
|
* [[dogfood-the-command-not-the-cli]] (#63): a hand-typed argv is a path no user
|
|
* takes. The flag surface a template actually exercises is only knowable by
|
|
* reading the template, and it lives in THREE forms, all of them "the template's
|
|
* own text":
|
|
*
|
|
* 1. literal in the call `--apply`, `--json`
|
|
* 2. an `if` assignment `… then RAW_FLAG="--raw"; fi`
|
|
* 3. ONLY in a trailing comment `GLOBAL_FLAG="" # --global`
|
|
*
|
|
* Form 3 is the class that can die silently. The variable is empty on the
|
|
* default path, so a template run without the user flag proves nothing about
|
|
* whether the CLI still accepts it — exactly the shape of M-BUG-45, where
|
|
* `--stale-after` reached the CLI malformed and the command reported success.
|
|
* An extractor that reads only forms 1 and 2 is green on its own defect.
|
|
*/
|
|
|
|
import { readFile, readdir } from 'node:fs/promises';
|
|
import { resolve, dirname } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
const __dirname = dirname(fileURLToPath(import.meta.url));
|
|
export const COMMANDS_DIR = resolve(__dirname, '..', '..', 'commands');
|
|
|
|
/** `node ${CLAUDE_PLUGIN_ROOT}/scanners/<name>.mjs` — the only anchored form. */
|
|
const INVOCATION = /node\s+\$\{CLAUDE_PLUGIN_ROOT\}\/scanners\/([A-Za-z0-9._-]+\.mjs)/;
|
|
|
|
/** A bare `--flag` token. Stops at `=` so `--flag=value` still yields `--flag`. */
|
|
const FLAG_TOKEN = /(^|[\s"'])(--[a-z][a-z0-9-]*)(?=[\s"'=]|$)/g;
|
|
|
|
/** `$VAR` or `"$VAR"` or `${VAR}` in an argv position. */
|
|
const VAR_TOKEN = /\$\{?([A-Z_][A-Z0-9_]*)\}?/g;
|
|
|
|
/**
|
|
* Join shell line-continuations so a multi-line invocation is one logical line.
|
|
* Returns `[{ text, line }]` where `line` is the 1-based line the call STARTS on.
|
|
*/
|
|
function logicalLines(content) {
|
|
const raw = content.split('\n');
|
|
const out = [];
|
|
let buffer = null;
|
|
|
|
raw.forEach((text, i) => {
|
|
const continues = /\\\s*$/.test(text);
|
|
const body = text.replace(/\\\s*$/, '');
|
|
if (buffer) {
|
|
buffer.text += ' ' + body.trim();
|
|
if (!continues) { out.push(buffer); buffer = null; }
|
|
return;
|
|
}
|
|
if (continues) { buffer = { text: body, line: i + 1 }; return; }
|
|
out.push({ text, line: i + 1 });
|
|
});
|
|
|
|
if (buffer) out.push(buffer);
|
|
return out;
|
|
}
|
|
|
|
/**
|
|
* Every flag a shell variable can carry, resolved from the template itself.
|
|
* Returns `Map<varName, string[]>`.
|
|
*
|
|
* Both `VAR="--flag"` (any position, so the `if … then VAR="--raw"; fi` idiom is
|
|
* covered) and the comment form are read. The comment form is read from the
|
|
* WHOLE trailing comment, so `# e.g. SCOPE_FLAG="--full-machine" or
|
|
* SCOPE_FLAG="--global"` yields both candidates: a template that documents two
|
|
* possible values promises the CLI accepts both.
|
|
*/
|
|
export function resolveFlagVariables(content) {
|
|
const vars = new Map();
|
|
const add = (name, flag) => {
|
|
if (!vars.has(name)) vars.set(name, []);
|
|
if (!vars.get(name).includes(flag)) vars.get(name).push(flag);
|
|
};
|
|
|
|
for (const { text } of logicalLines(content)) {
|
|
// Form 2 — assignment to a literal flag, anywhere on the line.
|
|
for (const m of text.matchAll(/\b([A-Z_][A-Z0-9_]*)=["'](--[a-z][a-z0-9-]*)["']/g)) {
|
|
add(m[1], m[2]);
|
|
}
|
|
// Form 3 — an empty assignment whose real value lives in the comment.
|
|
const commented = text.match(/\b([A-Z_][A-Z0-9_]*)=["']{2}\s*#(.*)$/);
|
|
if (commented) {
|
|
for (const f of commented[2].matchAll(/(--[a-z][a-z0-9-]*)/g)) add(commented[1], f[1]);
|
|
}
|
|
}
|
|
|
|
return vars;
|
|
}
|
|
|
|
/**
|
|
* All invocations in one template.
|
|
* `[{ cli, flags: string[], line, source: Map<flag, 'literal'|'$VAR'> }]`
|
|
*/
|
|
export function extractInvocations(content) {
|
|
const vars = resolveFlagVariables(content);
|
|
const found = [];
|
|
|
|
for (const { text, line } of logicalLines(content)) {
|
|
const call = text.match(INVOCATION);
|
|
if (!call) continue;
|
|
|
|
// Only the part AFTER the script path is argv.
|
|
const argv = text.slice(text.indexOf(call[0]) + call[0].length);
|
|
const flags = [];
|
|
const source = new Map();
|
|
const push = (flag, from) => {
|
|
if (!flags.includes(flag)) { flags.push(flag); source.set(flag, from); }
|
|
};
|
|
|
|
for (const m of argv.matchAll(FLAG_TOKEN)) push(m[2], 'literal');
|
|
for (const m of argv.matchAll(VAR_TOKEN)) {
|
|
for (const flag of vars.get(m[1]) ?? []) push(flag, `$${m[1]}`);
|
|
}
|
|
|
|
found.push({ cli: call[1], flags, line, source });
|
|
}
|
|
|
|
return found;
|
|
}
|
|
|
|
/** Every command template, as `[{ file, content }]`, sorted. */
|
|
export async function commandTemplates() {
|
|
const entries = (await readdir(COMMANDS_DIR)).filter((e) => e.endsWith('.md')).sort();
|
|
return Promise.all(
|
|
entries.map(async (file) => ({
|
|
file,
|
|
content: await readFile(resolve(COMMANDS_DIR, file), 'utf-8'),
|
|
})),
|
|
);
|
|
}
|
|
|
|
/**
|
|
* The whole command layer's contract with the CLI layer:
|
|
* `[{ file, line, cli, flags, source }]`, one entry per invocation.
|
|
*/
|
|
export async function allInvocations() {
|
|
const out = [];
|
|
for (const { file, content } of await commandTemplates()) {
|
|
for (const inv of extractInvocations(content)) out.push({ file, ...inv });
|
|
}
|
|
return out;
|
|
}
|