feat(hooks): additionalContext injection advisory + filter-before lever (v5.10 B5) [skip-docs]
HKV now flags hooks that build hookSpecificOutput.additionalContext from un-grepped command output as an INFO advisory (weight 0, never severity-bearing — excluded from the self-audit nonInfo set). That field enters Claude's context every time the hook fires (plain stdout on exit 0 does not), so an unfiltered payload is a recurring per-turn token cost. - New lib scanners/lib/hook-additional-context.mjs: pure assessHookAdditionalContext (unit-tested, no IO) + IO wrapper assessHookContextForRepo (walk hooks->scripts). Heuristic: additionalContext + verbose-prone capture (cat/git log/execSync/…) && no filter (grep/head/jq/.slice). Deliberately low precision -> advisory only. - HKV: emits the advisory inline on scripts it already reads (after the M5 verbose check). Additive, info severity -> frozen v5.0.0 + SC-5 snapshots untouched. - feature-gap: new filterHookLeverFinding companion, fires ONLY when >=1 chatty hook is detected — surfaces the filter-before-Claude-reads lever (CC filter-test-output.sh). Silent otherwise (opportunity, not noise). - docs: README HKV + GAP scanner rows; scanner-internals.md HKV row + full B5 implementation note. CLAUDE.md kept lean ([skip-docs]); B5 fully documented in README + scanner-internals. Mechanism verified 2026-06-23 against code.claude.com/docs context-window.md (additionalContext enters context; plain stdout does not). Suite 1239 -> 1254 green. Version/badges/CHANGELOG wait for the v5.10 release cut. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
8f7e196046
commit
d2c45a3bb8
11 changed files with 468 additions and 4 deletions
|
|
@ -316,12 +316,12 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
|---------|--------|-----------------|
|
||||
| `claude-md-linter.mjs` | CML | Oversized files (line count **plus** a context-window-scaled char budget mirroring Claude Code's ~40.0k-char startup warning), missing sections, broken @imports, duplicates, stale TODOs |
|
||||
| `settings-validator.mjs` | SET | Schema violations, unknown/deprecated keys, type mismatches, permission issues |
|
||||
| `hook-validator.mjs` | HKV | Invalid format, missing scripts, wrong event names, timeout risks |
|
||||
| `hook-validator.mjs` | HKV | Invalid format, missing scripts, wrong event names, timeout risks, verbose-stdout scripts, and a low-precision **advisory** (info) when a hook injects un-grepped command output into `hookSpecificOutput.additionalContext` — that payload enters context on every fire (plain stdout does not) |
|
||||
| `rules-validator.mjs` | RUL | Bad glob patterns, orphaned rules, deprecated fields, unscoped rules |
|
||||
| `mcp-config-validator.mjs` | MCP | Invalid server types, exposed env vars, unknown fields |
|
||||
| `import-resolver.mjs` | IMP | Broken @imports, circular references, deep chains, tilde path issues |
|
||||
| `conflict-detector.mjs` | CNF | Settings contradictions across scopes, permission conflicts, hook duplicates |
|
||||
| `feature-gap-scanner.mjs` | GAP | 25 feature checks shown as opportunities, not grades — plus a conditional `disableBundledSkills` recommendation when the active skill listing is over budget |
|
||||
| `feature-gap-scanner.mjs` | GAP | 25 feature checks shown as opportunities, not grades — plus a conditional `disableBundledSkills` recommendation when the active skill listing is over budget, and a conditional **filter-before-Claude-reads** lever when a hook injects unfiltered output into `additionalContext` (companion to the HKV advisory; cites the documented `filter-test-output.sh` pattern) |
|
||||
| `token-hotspots.mjs` | TOK | Cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascades, bloated skill descriptions, MCP tool-schema budget, and stale `~/.claude/plugins/cache` versions (disk-cleanup, zero live-context impact) — cache-aware ranking excludes superseded plugin versions by default (`--no-exclude-cache` to include) |
|
||||
| `cache-prefix-scanner.mjs` | CPS | Volatile content in lines 31–150 of the CLAUDE.md cascade — beyond the cache-prefix window but still re-loaded every turn |
|
||||
| `disabled-in-schema-scanner.mjs` | DIS | Dead/ineffective permission entries: (1) tools in BOTH `permissions.deny` and `permissions.allow` — deny wins (incl. the `Tool(*)` deny-all glob, equivalent to a bare deny); (2) unanchored allow wildcards (`*`, `B*`, `mcp__*`) that Claude Code silently skips — valid only as `mcp__<server>__*`; (3) `Tool(param:value)` rules whose key is the tool's own canonicalizing field (`command`/`file_path`/`path`/`notebook_path`/`url`) — CC ignores these and emits a startup warning |
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ Scanner CLI: `node scanners/scan-orchestrator.mjs <path> [--global] [--full-mach
|
|||
|---------|--------|---------|
|
||||
| `claude-md-linter.mjs` | CML | Structure, length, sections, @imports, duplicates, TODOs |
|
||||
| `settings-validator.mjs` | SET | Schema, unknown/deprecated keys, type mismatches, permissions |
|
||||
| `hook-validator.mjs` | HKV | Format, script existence, event validity, timeouts |
|
||||
| `hook-validator.mjs` | HKV | Format, script existence, event validity, timeouts, verbose-stdout (low), unfiltered `additionalContext` injection (info advisory, v5.10 B5) |
|
||||
| `rules-validator.mjs` | RUL | Glob matching, orphan rules, deprecated fields, unscoped rules |
|
||||
| `mcp-config-validator.mjs` | MCP | Server types, env vars, unknown fields |
|
||||
| `import-resolver.mjs` | IMP | Broken @imports, circular refs, deep chains, tilde paths |
|
||||
|
|
@ -185,6 +185,40 @@ operations (CLI adds zero context tokens until invoked). Deferred MCP is effecti
|
|||
lever stays silent in the default case (opportunity, not noise — mirrors the bundledSkills lever).
|
||||
`alwaysLoad` was added to CA-MCP's `VALID_SERVER_FIELDS` so it is never flagged as an unknown field.
|
||||
|
||||
### hook-validator — unfiltered additionalContext advisory (v5.10 B5)
|
||||
|
||||
A hook that emits `hookSpecificOutput.additionalContext` has that payload injected into Claude's
|
||||
context **every time it fires** — plain stdout on exit 0 does NOT (it goes to the debug log only). A
|
||||
hook that dumps large, un-grepped command output into `additionalContext` is therefore a recurring,
|
||||
compaction-sensitive per-turn token cost. HKV flags it as an **`info` advisory** (weight 0 — never
|
||||
severity-bearing, excluded from the self-audit `nonInfo` set), paired with a feature-gap lever.
|
||||
|
||||
The heuristic lives in `lib/hook-additional-context.mjs` as a **pure** `assessHookAdditionalContext({scriptContent})`
|
||||
(unit-tested, no IO) plus a thin IO wrapper `assessHookContextForRepo(discovery)` (walk hooks → scripts
|
||||
→ assess) used by GAP; HKV calls the pure function inline on scripts it already reads. The signal:
|
||||
|
||||
| Condition | Detected by | Effect |
|
||||
|-----------|-------------|--------|
|
||||
| references `additionalContext` | `/additionalContext/` | gate (else not applicable) |
|
||||
| captures verbose-prone output | `cat`/`find`/`ls`/`git log\|diff\|status\|show`/`npm`/`pytest`/`jest`/`curl`/`execSync`/`readFileSync`… | `hasVerboseCapture` |
|
||||
| applies any truncating filter | `grep`/`head`/`tail`/`sed`/`awk`/`jq`/`cut`/`wc`/`uniq`/`sort` or `.slice`/`.substring` | suppresses (assumed bounded) |
|
||||
|
||||
`flagged = buildsAdditionalContext && hasVerboseCapture && !hasFilter`. A filtered capture (e.g.
|
||||
`cat … | grep ERROR`) or a cheap-only capture (`$(date)`) is not flagged.
|
||||
|
||||
**Why `info`, not a hard finding (Verifiseringsplikt).** This is deliberately **low precision** — a
|
||||
static scan cannot run the hook or measure the real payload, and a filter we don't recognise would be
|
||||
a false positive. So it ships as an advisory with the precision caveat in its own description, never a
|
||||
graded/severity-bearing finding. Mechanism verified 2026-06-23 against `code.claude.com/docs`:
|
||||
`context-window.md` — *"A PostToolUse hook … reports back via `hookSpecificOutput.additionalContext`.
|
||||
That field enters Claude's context. Plain stdout on exit 0 does not."* + the tip to keep output concise
|
||||
(it enters context without truncation).
|
||||
|
||||
**feature-gap companion.** `filterHookLeverFinding` (GAP) fires **only** when `assessHookContextForRepo`
|
||||
returns ≥1 chatty hook — surfaces the documented **filter-before-Claude-reads** lever (`filter-test-output.sh`:
|
||||
grep ERROR and return only matches instead of a 10,000-line log). No chatty hook → silent (opportunity,
|
||||
not noise — same contract as the cliOverMcp / bundledSkills levers).
|
||||
|
||||
### CML scanner — context-window-scaled char budget
|
||||
|
||||
Beyond the line-count checks (200/500 lines, both MEDIUM), the CML scanner mirrors
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ import { SEVERITY } from './lib/severity.mjs';
|
|||
import { findImports, parseJson, parseFrontmatter } from './lib/yaml-parser.mjs';
|
||||
import { measureActiveSkillListing, isBundledSkillsDisabled, BUDGET_CALIBRATION_NOTE } from './lib/skill-listing-budget.mjs';
|
||||
import { assessMcpDeferralForRepo } from './lib/mcp-deferral.mjs';
|
||||
import { assessHookContextForRepo } from './lib/hook-additional-context.mjs';
|
||||
|
||||
const SCANNER = 'GAP';
|
||||
|
||||
|
|
@ -175,6 +176,41 @@ export function cliOverMcpLeverFinding({ assessment } = {}) {
|
|||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* filter-before-Claude-reads lever — remediation companion to HKV's B5 advisory
|
||||
* (v5.10). Fires ONLY when ≥1 active hook was detected injecting unfiltered
|
||||
* command output into additionalContext, i.e. when there is a measured chatty
|
||||
* hook to fix. When no such hook exists there is nothing to recommend and we
|
||||
* stay silent — opportunity, not noise (same "fire only under measured pressure"
|
||||
* contract as the cliOverMcp / bundledSkills levers).
|
||||
*
|
||||
* Pure and exported for unit testing.
|
||||
*
|
||||
* @param {{ flaggedHooks: Array<{event:string, scriptPath:string}> }} args
|
||||
* @returns {object|null} a GAP finding, or null when no chatty hook was detected
|
||||
*/
|
||||
export function filterHookLeverFinding({ flaggedHooks } = {}) {
|
||||
const hooks = Array.isArray(flaggedHooks) ? flaggedHooks : [];
|
||||
if (hooks.length === 0) return null;
|
||||
const scripts = hooks.map((h) => h.scriptPath.split('/').slice(-1)[0]).join(', ');
|
||||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
title: 'Filter hook output before it enters context',
|
||||
description:
|
||||
`${hooks.length} active hook${hooks.length === 1 ? '' : 's'} build hookSpecificOutput.additionalContext ` +
|
||||
"from un-grepped command output (see HKV advisory). That field enters Claude's context on every fire, " +
|
||||
'so filtering verbose output down to what matters BEFORE Claude reads it reclaims per-turn tokens — the ' +
|
||||
'documented filter-test-output.sh pattern (grep ERROR and return only matches instead of a 10,000-line log).',
|
||||
evidence: `chatty_hooks=${hooks.length}; scripts=${scripts} (companion to HKV additionalContext advisory)`,
|
||||
recommendation:
|
||||
'In each flagged hook, pipe the command output through grep/head/jq to keep only the actionable lines ' +
|
||||
'before assigning additionalContext. Reserve additionalContext for concise signals; leave bulk diagnostics ' +
|
||||
'on plain stdout (exit 0) so they go to the debug log, not context.',
|
||||
category: 'token-efficiency',
|
||||
});
|
||||
}
|
||||
|
||||
/** @type {GapCheck[]} */
|
||||
const GAP_CHECKS = [
|
||||
// --- Tier 1: Foundation ---
|
||||
|
|
@ -489,6 +525,12 @@ export async function scan(targetPath, sharedDiscovery) {
|
|||
const cliLever = cliOverMcpLeverFinding({ assessment: mcpAssessment });
|
||||
if (cliLever) findings.push(cliLever);
|
||||
|
||||
// filter-before-Claude-reads lever — companion to HKV's B5 advisory: fires
|
||||
// only when an active hook injects unfiltered output into additionalContext.
|
||||
const flaggedHooks = await assessHookContextForRepo(discovery);
|
||||
const hookLever = filterHookLeverFinding({ flaggedHooks });
|
||||
if (hookLever) findings.push(hookLever);
|
||||
|
||||
const filesScanned = discovery.files.length;
|
||||
return scannerResult(SCANNER, 'ok', findings, filesScanned, Date.now() - start);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import { readTextFile, discoverConfigFiles } from './lib/file-discovery.mjs';
|
|||
import { finding, scannerResult } from './lib/output.mjs';
|
||||
import { SEVERITY } from './lib/severity.mjs';
|
||||
import { parseJson } from './lib/yaml-parser.mjs';
|
||||
import { assessHookAdditionalContext } from './lib/hook-additional-context.mjs';
|
||||
import { stat } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
|
||||
|
|
@ -248,6 +249,35 @@ async function validateHooksObject(hooks, file, findings, baseDir) {
|
|||
autoFixable: false,
|
||||
}));
|
||||
}
|
||||
|
||||
// v5.10 B5: advisory (info) — a hook that injects unfiltered
|
||||
// command output into hookSpecificOutput.additionalContext pays
|
||||
// that whole payload into Claude's context on every fire (plain
|
||||
// stdout does not). Low-precision static heuristic, so info only.
|
||||
const scriptContent = await readTextFile(scriptPath);
|
||||
const ac = assessHookAdditionalContext({ scriptContent });
|
||||
if (ac.flagged) {
|
||||
findings.push(finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
title: 'Hook injects unfiltered output into context',
|
||||
description:
|
||||
`${file.relPath}: "${event}" runs ${scriptPath.split('/').slice(-2).join('/')} ` +
|
||||
'which builds hookSpecificOutput.additionalContext from un-grepped command ' +
|
||||
"output. That field enters Claude's context every time the hook fires (plain " +
|
||||
'stdout does not), so an unfiltered payload is a recurring per-turn token cost. ' +
|
||||
'Advisory only — low-precision static heuristic; verify the real payload size.',
|
||||
file: scriptPath,
|
||||
evidence:
|
||||
'additional_context_unfiltered=true; ' +
|
||||
`verbose_capture=${ac.hasVerboseCapture}; filter_applied=${ac.hasFilter}`,
|
||||
recommendation:
|
||||
'Filter before Claude reads: grep/head the command output down to what matters ' +
|
||||
'before putting it in additionalContext (the documented filter-test-output.sh ' +
|
||||
'pattern), or keep large diagnostics on plain stdout so they stay out of context.',
|
||||
autoFixable: false,
|
||||
}));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
160
scanners/lib/hook-additional-context.mjs
Normal file
160
scanners/lib/hook-additional-context.mjs
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
/**
|
||||
* Hook additionalContext injection advisory (v5.10 B5).
|
||||
*
|
||||
* A hook that emits `hookSpecificOutput.additionalContext` has that content
|
||||
* injected into Claude's context EVERY time the hook fires. Plain stdout on
|
||||
* exit 0 does NOT enter context (it goes to the debug log only). So a hook that
|
||||
* dumps large, unfiltered command output into additionalContext is a recurring
|
||||
* per-turn token cost that compounds and is compaction-sensitive.
|
||||
*
|
||||
* This module is a STATIC heuristic over hook SCRIPT SOURCE. It is deliberately
|
||||
* LOW PRECISION — it cannot run the script or measure the real payload — so it
|
||||
* ships as an INFO advisory (weight 0, never severity-bearing), paired with a
|
||||
* feature-gap "filter-before-Claude-reads" lever. The signal:
|
||||
*
|
||||
* the script references `additionalContext`
|
||||
* AND captures output from a verbose-prone command (cat/find/git log/test/curl…)
|
||||
* AND applies no truncating/filtering tool anywhere (grep/head/jq/.slice…).
|
||||
*
|
||||
* A script that pipes through a filter, or only captures cheap output ($(date)),
|
||||
* is assumed bounded and is not flagged. Conversely a verbose capture with no
|
||||
* filter is the un-grepped pattern worth surfacing.
|
||||
*
|
||||
* Mechanism verified 2026-06-23 against code.claude.com/docs:
|
||||
* context-window.md — "A PostToolUse hook … reports back via
|
||||
* hookSpecificOutput.additionalContext. That field enters Claude's context.
|
||||
* Plain stdout on exit 0 does not." + tip: keep output concise; it enters
|
||||
* context without truncation. The remediation lever is the documented
|
||||
* filter-test-output.sh pattern (grep before Claude reads).
|
||||
*
|
||||
* The pure `assessHookAdditionalContext` takes already-read script text so it is
|
||||
* fully unit-testable without IO. `assessHookContextForRepo` is the thin IO
|
||||
* wrapper (walk hooks → scripts → assess) shared by feature-gap; the HKV scanner
|
||||
* calls the pure function inline on scripts it already reads.
|
||||
*/
|
||||
|
||||
import { readTextFile } from './file-discovery.mjs';
|
||||
import { parseJson } from './yaml-parser.mjs';
|
||||
import { stat } from 'node:fs/promises';
|
||||
import { resolve, dirname } from 'node:path';
|
||||
|
||||
// The field that actually enters Claude's context (vs. plain stdout / debug log).
|
||||
const ADDITIONAL_CONTEXT_RX = /additionalContext/;
|
||||
|
||||
// Commands whose UNfiltered output can be large. A capture invoking one of these
|
||||
// with no filter anywhere is the low-precision "un-grepped output" signal.
|
||||
// Shell substitution AND node child_process / file reads are both covered.
|
||||
const VERBOSE_CAPTURE_RX =
|
||||
/\b(?:cat|find|ls|git\s+(?:log|diff|status|show)|npm|yarn|pnpm|pytest|jest|go\s+test|cargo\s+test|curl|wget|env|printenv|dmesg|journalctl|execSync|spawnSync|readFileSync)\b/;
|
||||
|
||||
// Truncating / filtering tools that BOUND a payload before it reaches context.
|
||||
// Their presence anywhere in the script suppresses the advisory (assumed bounded).
|
||||
// Shell filters + the common node-side bounding operations.
|
||||
const FILTER_RX =
|
||||
/\b(?:grep|egrep|rg|head|tail|sed|awk|jq|cut|wc|uniq|sort)\b|\.(?:slice|substring|substr)\s*\(/;
|
||||
|
||||
/**
|
||||
* Assess one hook script's source for unfiltered additionalContext injection.
|
||||
*
|
||||
* @param {{ scriptContent?: string }} [args]
|
||||
* @returns {{
|
||||
* buildsAdditionalContext: boolean,
|
||||
* hasVerboseCapture: boolean,
|
||||
* hasFilter: boolean,
|
||||
* capturesUnfiltered: boolean,
|
||||
* flagged: boolean,
|
||||
* }}
|
||||
*/
|
||||
export function assessHookAdditionalContext({ scriptContent } = {}) {
|
||||
const content = typeof scriptContent === 'string' ? scriptContent : '';
|
||||
const buildsAdditionalContext = ADDITIONAL_CONTEXT_RX.test(content);
|
||||
const hasVerboseCapture = VERBOSE_CAPTURE_RX.test(content);
|
||||
const hasFilter = FILTER_RX.test(content);
|
||||
const capturesUnfiltered = hasVerboseCapture && !hasFilter;
|
||||
return {
|
||||
buildsAdditionalContext,
|
||||
hasVerboseCapture,
|
||||
hasFilter,
|
||||
capturesUnfiltered,
|
||||
flagged: buildsAdditionalContext && capturesUnfiltered,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract a filesystem script path from a hook command string.
|
||||
* Mirrors hook-validator's extractScriptPath (kept local so the shared lib has
|
||||
* no upward dependency on a scanner). Handles ${CLAUDE_PLUGIN_ROOT}.
|
||||
*/
|
||||
function extractScriptPath(command, baseDir) {
|
||||
const match = command.match(/(?:bash|node|sh)\s+(.+?)(?:\s|$)/);
|
||||
if (!match) return null;
|
||||
let scriptPath = match[1].trim();
|
||||
scriptPath = scriptPath.replace(/\$\{CLAUDE_PLUGIN_ROOT\}/g, resolve(baseDir, '..'));
|
||||
scriptPath = scriptPath.replace(/\$CLAUDE_PLUGIN_ROOT/g, resolve(baseDir, '..'));
|
||||
if (scriptPath.includes('$')) return null;
|
||||
return resolve(baseDir, scriptPath);
|
||||
}
|
||||
|
||||
/** Yield every command-hook { event, command } from a hooks object. */
|
||||
function* iterateCommandHooks(hooks) {
|
||||
if (!hooks || typeof hooks !== 'object' || Array.isArray(hooks)) return;
|
||||
for (const [event, handlers] of Object.entries(hooks)) {
|
||||
if (!Array.isArray(handlers)) continue;
|
||||
for (const group of handlers) {
|
||||
const hookList = group && Array.isArray(group.hooks) ? group.hooks : [];
|
||||
for (const hook of hookList) {
|
||||
if (hook && hook.type === 'command' && typeof hook.command === 'string') {
|
||||
yield { event, command: hook.command };
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* IO wrapper: walk discovered hooks (hooks.json + settings.json hooks), resolve
|
||||
* each command hook's script, and return the ones flagged by the heuristic.
|
||||
* Shared by feature-gap (the HKV scanner assesses inline on scripts it reads).
|
||||
*
|
||||
* @param {{ files: import('./file-discovery.mjs').ConfigFile[] }} discovery
|
||||
* @returns {Promise<Array<{ event: string, scriptPath: string, file: string,
|
||||
* assessment: ReturnType<typeof assessHookAdditionalContext> }>>}
|
||||
*/
|
||||
export async function assessHookContextForRepo(discovery) {
|
||||
const flagged = [];
|
||||
const files = (discovery && Array.isArray(discovery.files)) ? discovery.files : [];
|
||||
|
||||
const hooksObjects = [];
|
||||
for (const file of files.filter((f) => f.type === 'hooks-json')) {
|
||||
const content = await readTextFile(file.absPath);
|
||||
const parsed = content ? parseJson(content) : null;
|
||||
if (parsed) hooksObjects.push({ hooks: parsed.hooks || parsed, file });
|
||||
}
|
||||
for (const file of files.filter((f) => f.type === 'settings-json')) {
|
||||
const content = await readTextFile(file.absPath);
|
||||
const parsed = content ? parseJson(content) : null;
|
||||
if (parsed && parsed.hooks && !Array.isArray(parsed.hooks)) {
|
||||
hooksObjects.push({ hooks: parsed.hooks, file });
|
||||
}
|
||||
}
|
||||
|
||||
for (const { hooks, file } of hooksObjects) {
|
||||
const baseDir = dirname(file.absPath);
|
||||
for (const { event, command } of iterateCommandHooks(hooks)) {
|
||||
const scriptPath = extractScriptPath(command, baseDir);
|
||||
if (!scriptPath) continue;
|
||||
try {
|
||||
await stat(scriptPath);
|
||||
} catch {
|
||||
continue; // missing script — HKV reports that separately
|
||||
}
|
||||
const scriptContent = await readTextFile(scriptPath);
|
||||
const assessment = assessHookAdditionalContext({ scriptContent });
|
||||
if (assessment.flagged) {
|
||||
flagged.push({ event, scriptPath, file: file.absPath, assessment });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return flagged;
|
||||
}
|
||||
10
tests/fixtures/hooks-additional-context/hooks/hooks.json
vendored
Normal file
10
tests/fixtures/hooks-additional-context/hooks/hooks.json
vendored
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
{
|
||||
"hooks": {
|
||||
"SessionStart": [
|
||||
{ "hooks": [{ "type": "command", "command": "bash ./scripts/chatty.sh", "timeout": 5000 }] }
|
||||
],
|
||||
"PostToolUse": [
|
||||
{ "matcher": "Bash", "hooks": [{ "type": "command", "command": "bash ./scripts/filtered.sh", "timeout": 5000 }] }
|
||||
]
|
||||
}
|
||||
}
|
||||
6
tests/fixtures/hooks-additional-context/hooks/scripts/chatty.sh
vendored
Normal file
6
tests/fixtures/hooks-additional-context/hooks/scripts/chatty.sh
vendored
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
#!/usr/bin/env bash
|
||||
# Chatty hook: dumps unfiltered command output straight into additionalContext.
|
||||
# Every SessionStart this whole payload enters Claude's context (not just stdout).
|
||||
NOTES="$(cat ./STATE.md)"
|
||||
LOG="$(git log --oneline)"
|
||||
printf '{"hookSpecificOutput":{"hookEventName":"SessionStart","additionalContext":"%s\n%s"}}' "$NOTES" "$LOG"
|
||||
5
tests/fixtures/hooks-additional-context/hooks/scripts/filtered.sh
vendored
Normal file
5
tests/fixtures/hooks-additional-context/hooks/scripts/filtered.sh
vendored
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
#!/usr/bin/env bash
|
||||
# Bounded hook: greps the log down to only failures before injecting context.
|
||||
# This is the filter-before-Claude-reads pattern — should NOT be flagged.
|
||||
ERRORS="$(cat ./test.log | grep -E 'FAIL|ERROR' | head -20)"
|
||||
printf '{"hookSpecificOutput":{"hookEventName":"PostToolUse","additionalContext":"%s"}}' "$ERRORS"
|
||||
|
|
@ -5,7 +5,7 @@ import { fileURLToPath } from 'node:url';
|
|||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { resetCounter } from '../../scanners/lib/output.mjs';
|
||||
import { scan, opportunitySummary, bundledSkillsLeverFinding, cliOverMcpLeverFinding } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { scan, opportunitySummary, bundledSkillsLeverFinding, cliOverMcpLeverFinding, filterHookLeverFinding } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
||||
|
||||
|
|
@ -364,3 +364,50 @@ describe('cliOverMcpLeverFinding (CLI-over-MCP lever, v5.10 B4)', () => {
|
|||
assert.match(f.recommendation || '', /\bgh\b|\baws\b|\bgcloud\b/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('filterHookLeverFinding (filter-before-Claude-reads lever, v5.10 B5)', () => {
|
||||
it('returns null when no chatty hook was detected', () => {
|
||||
assert.equal(filterHookLeverFinding({ flaggedHooks: [] }), null);
|
||||
assert.equal(filterHookLeverFinding({}), null);
|
||||
assert.equal(filterHookLeverFinding(), null);
|
||||
});
|
||||
|
||||
it('fires an info opportunity when ≥1 chatty hook is detected', () => {
|
||||
resetCounter();
|
||||
const f = filterHookLeverFinding({
|
||||
flaggedHooks: [{ event: 'SessionStart', scriptPath: '/x/hooks/scripts/chatty.sh' }],
|
||||
});
|
||||
assert.ok(f, 'expected a lever finding');
|
||||
assert.equal(f.severity, 'info');
|
||||
assert.equal(f.category, 'token-efficiency');
|
||||
assert.match(f.description || '', /filter-test-output\.sh|filter/i);
|
||||
assert.match(f.evidence || '', /chatty_hooks=1/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('GAP scanner — filter-before lever wiring (chatty hook fixture)', () => {
|
||||
it('emits the filter-before lever when scanning a repo with a chatty hook', async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('hooks-additional-context');
|
||||
const result = await withHermeticHome(() =>
|
||||
scan(resolve(FIXTURES, 'hooks-additional-context'), discovery),
|
||||
);
|
||||
const lever = result.findings.find(
|
||||
f => f.scanner === 'GAP' && /chatty_hooks=/.test(f.evidence || ''),
|
||||
);
|
||||
assert.ok(lever, `expected filter-before lever; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
assert.equal(lever.severity, 'info');
|
||||
});
|
||||
|
||||
it('does NOT emit the lever for a repo with only quiet hooks', async () => {
|
||||
resetCounter();
|
||||
const discovery = await fixtureDiscovery('hooks-quiet');
|
||||
const result = await withHermeticHome(() =>
|
||||
scan(resolve(FIXTURES, 'hooks-quiet'), discovery),
|
||||
);
|
||||
const lever = result.findings.find(
|
||||
f => f.scanner === 'GAP' && /chatty_hooks=/.test(f.evidence || ''),
|
||||
);
|
||||
assert.equal(lever, undefined, `expected no filter-before lever; got id=${lever?.id}`);
|
||||
});
|
||||
});
|
||||
|
|
|
|||
90
tests/scanners/hook-additional-context.test.mjs
Normal file
90
tests/scanners/hook-additional-context.test.mjs
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { assessHookAdditionalContext } from '../../scanners/lib/hook-additional-context.mjs';
|
||||
|
||||
// B5 (v5.10) — pure heuristic: does a hook script build
|
||||
// hookSpecificOutput.additionalContext from un-grepped / verbose command output?
|
||||
// Low precision by design (ships as an info advisory, never severity-bearing).
|
||||
|
||||
describe('assessHookAdditionalContext — not applicable', () => {
|
||||
it('empty / missing input → not flagged', () => {
|
||||
assert.equal(assessHookAdditionalContext({}).flagged, false);
|
||||
assert.equal(assessHookAdditionalContext({ scriptContent: '' }).flagged, false);
|
||||
assert.equal(assessHookAdditionalContext().flagged, false);
|
||||
});
|
||||
|
||||
it('script with no additionalContext reference → not flagged even if it cats a file', () => {
|
||||
const a = assessHookAdditionalContext({
|
||||
scriptContent: 'CONTENT="$(cat /var/log/big.log)"\necho "$CONTENT"\n',
|
||||
});
|
||||
assert.equal(a.buildsAdditionalContext, false);
|
||||
assert.equal(a.flagged, false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('assessHookAdditionalContext — shell hooks', () => {
|
||||
it('additionalContext built from an unfiltered cat → flagged', () => {
|
||||
const a = assessHookAdditionalContext({
|
||||
scriptContent:
|
||||
'BODY="$(cat /tmp/notes.md)"\n' +
|
||||
'echo "{\\"hookSpecificOutput\\":{\\"additionalContext\\":\\"$BODY\\"}}"\n',
|
||||
});
|
||||
assert.equal(a.buildsAdditionalContext, true);
|
||||
assert.equal(a.hasVerboseCapture, true);
|
||||
assert.equal(a.hasFilter, false);
|
||||
assert.equal(a.capturesUnfiltered, true);
|
||||
assert.equal(a.flagged, true);
|
||||
});
|
||||
|
||||
it('additionalContext from a git log → flagged', () => {
|
||||
const a = assessHookAdditionalContext({
|
||||
scriptContent:
|
||||
'LOG="$(git log --oneline)"\n' +
|
||||
'printf \'{"hookSpecificOutput":{"additionalContext":"%s"}}\' "$LOG"\n',
|
||||
});
|
||||
assert.equal(a.flagged, true);
|
||||
});
|
||||
|
||||
it('additionalContext from cat piped through grep → NOT flagged (filtered)', () => {
|
||||
const a = assessHookAdditionalContext({
|
||||
scriptContent:
|
||||
'BODY="$(cat /tmp/test.log | grep ERROR)"\n' +
|
||||
'echo "{\\"hookSpecificOutput\\":{\\"additionalContext\\":\\"$BODY\\"}}"\n',
|
||||
});
|
||||
assert.equal(a.hasFilter, true);
|
||||
assert.equal(a.capturesUnfiltered, false);
|
||||
assert.equal(a.flagged, false);
|
||||
});
|
||||
|
||||
it('additionalContext from a cheap command only ($(date)) → NOT flagged', () => {
|
||||
const a = assessHookAdditionalContext({
|
||||
scriptContent:
|
||||
'NOW="$(date)"\n' +
|
||||
'echo "{\\"hookSpecificOutput\\":{\\"additionalContext\\":\\"$NOW\\"}}"\n',
|
||||
});
|
||||
assert.equal(a.hasVerboseCapture, false);
|
||||
assert.equal(a.flagged, false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('assessHookAdditionalContext — node hooks', () => {
|
||||
it('execSync(git log) into additionalContext with no slice → flagged', () => {
|
||||
const a = assessHookAdditionalContext({
|
||||
scriptContent:
|
||||
"const out = execSync('git log --oneline').toString();\n" +
|
||||
"process.stdout.write(JSON.stringify({ hookSpecificOutput: { additionalContext: out } }));\n",
|
||||
});
|
||||
assert.equal(a.flagged, true);
|
||||
});
|
||||
|
||||
it('readFileSync sliced before additionalContext → NOT flagged (bounded)', () => {
|
||||
const a = assessHookAdditionalContext({
|
||||
scriptContent:
|
||||
"const raw = readFileSync('big.log', 'utf8');\n" +
|
||||
"const out = raw.slice(0, 400);\n" +
|
||||
"process.stdout.write(JSON.stringify({ hookSpecificOutput: { additionalContext: out } }));\n",
|
||||
});
|
||||
assert.equal(a.hasFilter, true);
|
||||
assert.equal(a.flagged, false);
|
||||
});
|
||||
});
|
||||
|
|
@ -101,6 +101,46 @@ describe('HKV scanner — verbose hook output (v5 M5)', () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe('HKV scanner — additionalContext injection advisory (v5.10 B5)', () => {
|
||||
it('flags a hook that injects unfiltered command output into additionalContext (info advisory)', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'hooks-additional-context');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
const f = result.findings.find(
|
||||
x => x.scanner === 'HKV' && /additional_context_unfiltered=true/.test(x.evidence || ''),
|
||||
);
|
||||
assert.ok(f, `expected additionalContext advisory; got: ${result.findings.map(x => x.title).join(' | ')}`);
|
||||
assert.equal(f.severity, 'info', `expected info (advisory), got ${f.severity}`);
|
||||
// The chatty.sh script (SessionStart) is the one flagged, not filtered.sh.
|
||||
assert.match(f.file || '', /chatty\.sh$/);
|
||||
// Precision caveat must be disclosed in the description (low-precision advisory).
|
||||
assert.match(f.description || '', /every time|enters Claude's context|advisory/i);
|
||||
});
|
||||
|
||||
it('does NOT flag the filtered (grep|head) sibling hook', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'hooks-additional-context');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
const filtered = result.findings.find(
|
||||
x => x.scanner === 'HKV' && /additional_context_unfiltered=true/.test(x.evidence || '') && /filtered\.sh$/.test(x.file || ''),
|
||||
);
|
||||
assert.equal(filtered, undefined, `filtered.sh should not be flagged; got id=${filtered?.id}`);
|
||||
});
|
||||
|
||||
it('does NOT flag a quiet hook with no additionalContext', async () => {
|
||||
resetCounter();
|
||||
const path = resolve(FIXTURES, 'hooks-quiet');
|
||||
const discovery = await discoverConfigFiles(path);
|
||||
const result = await scan(path, discovery);
|
||||
const f = result.findings.find(
|
||||
x => x.scanner === 'HKV' && /additional_context_unfiltered=true/.test(x.evidence || ''),
|
||||
);
|
||||
assert.equal(f, undefined, `expected no additionalContext advisory; got id=${f?.id}`);
|
||||
});
|
||||
});
|
||||
|
||||
describe('HKV scanner — CC 2.1.152 MessageDisplay event (Batch 1 false-positive fix)', () => {
|
||||
// The pre-write path-guard blocks committing settings.json/hooks.json, so
|
||||
// this suite materializes a hermetic temp fixture at runtime.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue