feat(scanners): model/effort routing becomes a lever, not a 25th dimension (C4)
New GAP finding CA-GAP-028: authored subagents exist and not one of them names `model:` or `effort:`, so every delegated task runs on the main conversation's model (`model` defaults to `inherit`). Cites BP-MODEL-001/002, landed in C1. `whats-active` and `manifest` now carry `model`/`effort` per agent. Shipped as a conditional LEVER rather than a 25th dimension, and the choice was made by measurement: as a t3 dimension the agent-less marketplace-medium fixture would count it vacuously-present, moving the denominators 41->42 and utilization 44->45 — which flips `segment` "Developing"->"Competent" in the frozen v5.0.0 posture baseline, a field strip-retired-gap.mjs does not mask. A lever never enters those denominators. The general rule is now an invariant in CLAUDE.md. One check across both axes, not one per axis: it fires only when neither is used anywhere, so a deliberate everything-on-one-model policy stays silent. Cost is recall, chosen for precision. Found by dogfooding, fixed red-first: `model: inherit` is the documented default spelled out, so it must not count as routing — otherwise a config opts out of the opportunity without changing anything real. Two pre-existing defects surfaced and closed on the way: - The humanizer guard asserted TRANSLATIONS.GAP.static EQUALS the dimension titles, which forbade humanizing any lever — all three existing levers fell through to the generic "feature opportunity" default, wrong for a budget lever. Guard now requires coverage of every emittable title, seen red against those three before the entries were written. - Two hand-written copies of the lever list (finding-codes guard, humanizer guard) merged into one exported LEVERS registry carrying code AND title. - suppression-validation pinned CA-GAP-028 as an unoccupied number; C4 claimed it. Fixed structurally with a derived first-free id, not by picking a new literal — same class as #60's "bump this again". Suite 1596/0. Frozen v5.0.0 snapshots untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pq3nye21RVYk4pZLeT8pGz
This commit is contained in:
parent
e861e63a7b
commit
9ae4be26d2
16 changed files with 512 additions and 31 deletions
|
|
@ -30,7 +30,7 @@ Per-command flags, patterns, and feature lists live in `README.md` and `/config-
|
|||
|---------|-------------|
|
||||
| `/config-audit drift` | Compare current config against saved baseline |
|
||||
| `/config-audit plugin-health` | Audit plugin structure, frontmatter, cross-plugin coherence |
|
||||
| `/config-audit whats-active` | Read-only inventory of active plugins/skills/MCP/hooks/CLAUDE.md (with token estimates) |
|
||||
| `/config-audit whats-active` | Read-only inventory of active plugins/skills/agents/MCP/hooks/CLAUDE.md (with token estimates, and `model`/`effort` per agent) |
|
||||
| `/config-audit knowledge-refresh` | Refresh the best-practices register (stale check + web poll). Human-approved writes; **not byte-stable** |
|
||||
| `/config-audit campaign` | Machine-wide audit ledger + token bill across repos. Human-approved writes; **not byte-stable** |
|
||||
| `/config-audit discover` | Run discovery phase only |
|
||||
|
|
@ -79,6 +79,8 @@ Workflow: `/config-audit → discover + analyze (auto) → plan → implement
|
|||
|
||||
Finding ID format: `CA-{SCANNER}-{NNN}` — e.g. `CA-CML-001`, `CA-SET-003`, `CA-HKV-002`, `CA-RUL-005`, `CA-TOK-005`, `CA-CPS-001`, `CA-SKL-001`, `CA-OST-001`, `CA-OPT-001`, `CA-AGT-001`.
|
||||
|
||||
**GAP dimensions vs. levers (invariant).** `GAP_CHECKS` holds the 24 *dimensions* — always evaluated, always counted in the utilization denominators (`TIER_COUNTS` / `TOTAL_DIMENSIONS` in `scoring.mjs`, and `TITLE_TO_ID` there). A *lever* is a finding the scanner emits after the loop and only under a measured condition; it carries no tier, never enters those denominators, and is registered in the exported `LEVERS` object (code + title in one place, because the finding-code guard needs the code and the humanizer-coverage guard needs the title). Adding a dimension moves every user's utilization score and can flip the reported `segment` in a frozen baseline — adding a lever cannot. When a check is only meaningful for configs that already have some feature, it is a lever.
|
||||
|
||||
**`{NNN}` names the CHECK, never the emission position (invariant).** `scanners/lib/finding-codes.mjs` is the single authority: every `finding()` call passes a `code`, and an undeclared or missing one **throws** — there is no counter fallback, because a fallback lets a half-converted scanner ship IDs that look valid. Adding a check takes the next free number for that scanner, never the next source-order position; removing one moves its key to `RETIRED_CODES` and its number is never reissued. IDs are therefore **not unique per finding** — one check failing in three files emits three findings sharing an ID, and `(id, file, line)` is the instance key that `fix-engine` verification uses. Frozen `v5.0.0` baselines mask IDs (`tests/helpers/mask-finding-ids.mjs`) instead of re-deriving them; the check→number pairs are pinned exhaustively in `tests/lib/finding-codes.test.mjs`.
|
||||
|
||||
## Conventions
|
||||
|
|
|
|||
15
README.md
15
README.md
|
|
@ -180,9 +180,20 @@ The feature opportunity scanner checks 24 dimensions and groups recommendations
|
|||
| **Worth Considering** | Workflow efficiency | Path-scoped rules, modular `@imports`, custom agents |
|
||||
| **Explore** | Nice-to-have | Keybindings, status line, output styles, agent teams |
|
||||
|
||||
Alongside the dimensions sit four **conditional levers** — recommendations that only make
|
||||
sense under a measured condition, so they stay silent otherwise. One of them is model/effort
|
||||
routing (`CA-GAP-028`): when you have your own subagents and *not one* of them names a
|
||||
`model:` or an `effort:`, every delegated task runs on the main conversation's model, because
|
||||
`model` defaults to `inherit`. It fires only when you actually have agents, and goes quiet the
|
||||
moment any of them routes either axis — so a deliberate everything-on-one-model setup is not
|
||||
nagged. Writing `model: inherit` out in full does not count as routing; it is the default
|
||||
spelled out.
|
||||
|
||||
Each recommendation is **context-aware** — it considers what your project actually contains. A solo TypeScript project gets different suggestions than a team Python monorepo. Recommendations include *why* (backed by Anthropic's official guidance) and *how* (concrete steps).
|
||||
|
||||
Run `/config-audit feature-gap` to see what's relevant to your project.
|
||||
Run `/config-audit feature-gap` to see what's relevant to your project. To see what each agent
|
||||
currently runs on, `/config-audit whats-active` lists `model` and `effort` per agent, and
|
||||
`/config-audit manifest` shows them on the agent rows.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -296,7 +307,7 @@ By default, `/config-audit` auto-detects scope from your git context. Override w
|
|||
| `mcp-config-validator.mjs` | MCP | Invalid server types, exposed env vars, unknown fields |
|
||||
| `import-resolver.mjs` | IMP | Broken @imports, circular references, deep chains, tilde path issues |
|
||||
| `conflict-detector.mjs` | CNF | Settings contradictions across scopes, permission conflicts, hook duplicates |
|
||||
| `feature-gap-scanner.mjs` | GAP | 24 feature checks shown as opportunities, not grades — plus a conditional `disableBundledSkills` recommendation when the active skill listing is over budget, and a conditional **filter-before-Claude-reads** lever when a hook injects unfiltered output into `additionalContext` (companion to the HKV advisory; cites the documented `filter-test-output.sh` pattern) |
|
||||
| `feature-gap-scanner.mjs` | GAP | 24 feature checks shown as opportunities, not grades — plus four conditional levers: a `disableBundledSkills` recommendation when the active skill listing is over budget, a **CLI-over-MCP** lever when tool schemas are forced upfront, a **filter-before-Claude-reads** lever when a hook injects unfiltered output into `additionalContext` (companion to the HKV advisory; cites the documented `filter-test-output.sh` pattern), and **agent model/effort routing** (`CA-GAP-028`) when authored subagents exist and none pins either axis (cites `BP-MODEL-001/002`). Levers are not dimensions: they stay out of the utilization denominators |
|
||||
| `token-hotspots.mjs` | TOK | Cache-breaking volatile content, redundant tool permissions, deep import chains, oversized cascades, bloated skill descriptions, MCP tool-schema budget, and stale `~/.claude/plugins/cache` versions (disk-cleanup, zero live-context impact) — cache-aware ranking excludes superseded plugin versions by default (`--no-exclude-cache` to include) |
|
||||
| `cache-prefix-scanner.mjs` | CPS | Volatile content in lines 31–150 of the CLAUDE.md cascade — beyond Pattern A's top-30 window but still re-loaded every turn — **plus** volatile content inside `@import`-ed files (inlined into the cached prefix, one hop, otherwise invisible to per-file scans) |
|
||||
| `disabled-in-schema-scanner.mjs` | DIS | Dead/ineffective permission entries: (1) tools in BOTH `permissions.deny` and `permissions.allow` — deny wins (incl. the `Tool(*)` deny-all glob, equivalent to a bare deny); (2) unanchored allow wildcards (`*`, `B*`, `mcp__*`) that Claude Code silently skips — valid only as `mcp__<server>__*`; (3) `Tool(param:value)` rules whose key is the tool's own canonicalizing field (`command`/`file_path`/`path`/`notebook_path`/`url`) — CC ignores these and emits a startup warning |
|
||||
|
|
|
|||
|
|
@ -73,6 +73,7 @@ Use the Read tool on `/tmp/config-audit-manifest.json`. Extract `meta.repoPath`,
|
|||
| ... | ... | ... | ... | ... | ... |
|
||||
|
||||
_Load column: **always** / **on-demand** / **external**. Append `°` when `derivationConfidence` is `inferred` (no primary-doc row pins it exactly)._
|
||||
_Agent rows carry `model` and `effort`. When either is set, append it to the name — `` `reviewer` (haiku/low) `` — using `inherit` / `default` for the unset side. Leave the suffix off entirely when both are null; a row of "inherit/default" on every agent is noise, and `/config-audit feature-gap` is where that becomes a finding._
|
||||
_Estimates assume ~4 chars/token (Claude ballpark). Real token count varies ±15%._
|
||||
```
|
||||
|
||||
|
|
|
|||
|
|
@ -94,7 +94,17 @@ Render as markdown:
|
|||
|-------|--------|--------|
|
||||
| {name} | {source}{if pluginName: ` (${pluginName})`} | ~{estimatedTokens} |
|
||||
|
||||
### MCP Servers ({mcpServers.length}, ~{mcpServers subtotal} tokens)
|
||||
### Agents ({agents.length}, ~{agents subtotal} tokens)
|
||||
|
||||
| Agent | Source | Model | Effort | Tokens |
|
||||
|-------|--------|-------|--------|--------|
|
||||
| {name} | {source}{if pluginName: ` (${pluginName})`} | {model or "inherit"} | {effort or "session default"} | ~{estimatedTokens} |
|
||||
|
||||
Skip this section entirely when `agents` is empty. `model: null` is rendered as
|
||||
*inherit* and `effort: null` as *session default* — both are the documented
|
||||
defaults, so a blank cell would read as missing data rather than as the choice
|
||||
it is. If no agent pins either column, say so in one sentence: every delegated
|
||||
task then costs what the session costs.
|
||||
|
||||
| Server | Source | Status | Command |
|
||||
|--------|--------|--------|---------|
|
||||
|
|
|
|||
|
|
@ -221,6 +221,46 @@ returns ≥1 chatty hook — surfaces the documented **filter-before-Claude-read
|
|||
grep ERROR and return only matches instead of a 10,000-line log). No chatty hook → silent (opportunity,
|
||||
not noise — same contract as the cliOverMcp / bundledSkills levers).
|
||||
|
||||
### feature-gap — agent model/effort routing lever (v5.14 C4, `CA-GAP-028`)
|
||||
|
||||
`agentModelRoutingLeverFinding` fires only when the target has **authored** subagents (the same
|
||||
`isAuthoredConfig` set the presence checks use, so plugin-bundled and fixture agents cannot make a
|
||||
machine look routed — M-BUG-13) and **not one of them** names `model:` or `effort:`. Cites
|
||||
`BP-MODEL-001` (a subagent's `model` defaults to `inherit`, so omitting it is a choice to pay the
|
||||
session's rate) and `BP-MODEL-002` (effort is a separate axis with its own frontmatter field).
|
||||
|
||||
**Why a lever and not a 25th dimension — decided by measurement, not taste.** A dimension is always
|
||||
evaluated, so "no agents at all" would have to read as *present*, and present weight feeds the
|
||||
utilization score. Measured on `tests/fixtures/marketplace-medium` (hermetic HOME) before the change:
|
||||
`utilization.score` 44, `segment` "Developing", where the "Competent" boundary is 45. As a t3
|
||||
dimension the denominators move 41→42 and the vacuous present pushes 18/41→19/42 = **45** — flipping
|
||||
`segment` in the frozen `v5.0.0/posture.json`, which `strip-retired-gap.mjs` does **not** mask (it
|
||||
drops only `utilization.score`/`overhang` and `feature_coverage.score`). A lever leaves every
|
||||
denominator alone and cannot move a score it never enters. The general rule now lives in CLAUDE.md
|
||||
(*GAP dimensions vs. levers*).
|
||||
|
||||
**One check across both axes, not one per axis.** It fires only when *neither* axis is used anywhere,
|
||||
so a deliberate everything-on-one-model policy stays silent. The cost is recall: a config that pins
|
||||
`model:` everywhere but never `effort:` gets no nudge. That is the v1 boundary, chosen for precision.
|
||||
|
||||
**`model: inherit` is not routing** — found by dogfooding, where installed agents write it out
|
||||
explicitly. `inherit` is the documented default, so spelling it out changes nothing about what the
|
||||
agent costs; counting it as a pin would let a config opt out of the opportunity without changing
|
||||
anything real. Effort has no documented sentinel of this kind, so it has no counterpart rule.
|
||||
|
||||
**Two silences that must not be conflated.** "No authored agents" (owned by dimension `t2_6`,
|
||||
*No custom subagents*) and "the only agents on disk are plugin-bundled" produce the same quiet
|
||||
output for different reasons. `tests/scanners/gap-agent-model-routing.test.mjs` P5 pins the second
|
||||
one specifically — it asserts the agent file *was* discovered before asserting silence, so the arm
|
||||
cannot pass for P4's reason.
|
||||
|
||||
**Humanizer coverage is now a blanket invariant.** The old guard asserted `TRANSLATIONS.GAP.static`
|
||||
keys *equal* `GAP_CHECKS` titles, which forbade humanizing any lever — so all three existing levers
|
||||
fell through to the generic GAP `_default` ("You have a feature opportunity worth a look"), wrong for
|
||||
a budget lever. The guard now requires a static entry for **every title GAP can emit** (dimensions ∪
|
||||
levers), and it was seen red against those three before the four entries were written. `TITLE_TO_ID`
|
||||
keeps strict equality with `GAP_CHECKS`: levers are not dimensions and must stay out of scoring.
|
||||
|
||||
### cache-prefix-scanner — @import extension (v5.10 B6)
|
||||
|
||||
CPS originally scanned only the files discovery classifies as `claude-md`. But a CLAUDE.md can pull
|
||||
|
|
|
|||
|
|
@ -1,9 +1,10 @@
|
|||
/**
|
||||
* GAP Scanner — Feature Gap Scanner
|
||||
* Compares actual configuration against complete Claude Code feature register.
|
||||
* 25 gap dimensions across 4 tiers, plus a conditional disableBundledSkills
|
||||
* budget-lever check (remediation companion to SKL CA-SKL-002, fires only under
|
||||
* measured skill-listing pressure). Always runs with includeGlobal: true.
|
||||
* 24 gap dimensions across 4 tiers, plus four conditional levers (bundled-skills
|
||||
* budget, CLI-over-MCP, hook-output filtering, agent model/effort routing) which
|
||||
* fire only under a measured condition and are therefore NOT dimensions: they
|
||||
* stay out of the scoring denominators. Always runs with includeGlobal: true.
|
||||
* Finding IDs: CA-GAP-NNN
|
||||
*/
|
||||
|
||||
|
|
@ -115,6 +116,36 @@ const TIER_SEVERITY = {
|
|||
t4: SEVERITY.info,
|
||||
};
|
||||
|
||||
/**
|
||||
* Titles of the conditional levers — findings this scanner emits that are NOT
|
||||
* dimensions in GAP_CHECKS. They fire only under a measured condition, so they
|
||||
* carry no tier and never enter the scoring denominators (TIER_COUNTS /
|
||||
* TOTAL_DIMENSIONS) or the scoring TITLE_TO_ID map.
|
||||
*
|
||||
* Exported as the single source of both the code and the title: the
|
||||
* finding-code registry guard needs the codes, the humanizer coverage guard
|
||||
* needs the titles, and a hand-maintained copy of either list in a test is the
|
||||
* two-copies-drift class. One object so the two cannot disagree.
|
||||
*/
|
||||
export const LEVERS = {
|
||||
bundledSkills: {
|
||||
code: 'bundled-skills-lever',
|
||||
title: 'Bundled skills add to an over-budget skill listing',
|
||||
},
|
||||
cliOverMcp: {
|
||||
code: 'cli-over-mcp-lever',
|
||||
title: 'Prefer CLI over MCP for common operations',
|
||||
},
|
||||
filterHookOutput: {
|
||||
code: 'filter-hook-output-lever',
|
||||
title: 'Filter hook output before it enters context',
|
||||
},
|
||||
agentModelRouting: {
|
||||
code: 'agent-model-routing-lever',
|
||||
title: 'Subagents pin neither model nor effort',
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Lazily read and cache file content.
|
||||
* @param {CheckContext} ctx
|
||||
|
|
@ -177,8 +208,8 @@ export function bundledSkillsLeverFinding({ leverPulled, aggregate }) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.low,
|
||||
code: 'bundled-skills-lever',
|
||||
title: 'Bundled skills add to an over-budget skill listing',
|
||||
code: LEVERS.bundledSkills.code,
|
||||
title: LEVERS.bundledSkills.title,
|
||||
description:
|
||||
`Your ${aggregate.scanned} active skills already carry ~${aggregate.aggregateTokens} tokens of ` +
|
||||
`description text, over the ${aggregate.budgetTokens}-token listing budget Claude Code allots the ` +
|
||||
|
|
@ -222,8 +253,8 @@ export function cliOverMcpLeverFinding({ assessment } = {}) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.low,
|
||||
code: 'cli-over-mcp-lever',
|
||||
title: 'Prefer CLI over MCP for common operations',
|
||||
code: LEVERS.cliOverMcp.code,
|
||||
title: LEVERS.cliOverMcp.title,
|
||||
description:
|
||||
`Your active project MCP tool schemas (~${assessment.aggregateTokens} tokens) are forced into the ` +
|
||||
'always-loaded prefix every turn rather than deferred (see CA-TOK-006). CLI tools (gh, aws, gcloud, …) ' +
|
||||
|
|
@ -260,8 +291,8 @@ export function filterHookLeverFinding({ flaggedHooks } = {}) {
|
|||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
code: 'filter-hook-output-lever',
|
||||
title: 'Filter hook output before it enters context',
|
||||
code: LEVERS.filterHookOutput.code,
|
||||
title: LEVERS.filterHookOutput.title,
|
||||
description:
|
||||
`${hooks.length} active hook${hooks.length === 1 ? '' : 's'} build hookSpecificOutput.additionalContext ` +
|
||||
"from un-grepped command output (see HKV advisory). That field enters Claude's context on every fire, " +
|
||||
|
|
@ -276,6 +307,101 @@ export function filterHookLeverFinding({ flaggedHooks } = {}) {
|
|||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Agent model/effort routing lever (C4) — cites BP-MODEL-001/002.
|
||||
*
|
||||
* A LEVER rather than a GAP_CHECKS dimension, and deliberately so. The question
|
||||
* "do your subagents route model/effort?" has no meaningful reading on a config
|
||||
* with no subagents — the `No custom subagents` dimension (t2_6) owns that case,
|
||||
* and firing here too would just double-report it. A dimension can only express
|
||||
* "not applicable" as "present", which would also inflate the utilization
|
||||
* denominator for every agent-less config.
|
||||
*
|
||||
* ONE check across BOTH axes, not two: it fires only when NEITHER `model:` nor
|
||||
* `effort:` appears on ANY authored agent. A deliberate all-on-one-model setup
|
||||
* therefore stays silent, which is the precision the opportunity framing needs.
|
||||
* The cost is recall — a config that pins `model:` everywhere but never uses
|
||||
* `effort:` gets no nudge. That trade is the v1 boundary, not an oversight.
|
||||
*
|
||||
* Pure and exported for unit testing.
|
||||
*
|
||||
* @param {{ agentCount: number, modelPinned: number, effortPinned: number }} counts
|
||||
* @returns {object|null} a GAP finding, or null when there is no opportunity
|
||||
*/
|
||||
export function agentModelRoutingLeverFinding({ agentCount, modelPinned, effortPinned }) {
|
||||
if (!agentCount) return null;
|
||||
if (modelPinned > 0 || effortPinned > 0) return null;
|
||||
|
||||
return finding({
|
||||
scanner: SCANNER,
|
||||
severity: SEVERITY.info,
|
||||
code: LEVERS.agentModelRouting.code,
|
||||
title: LEVERS.agentModelRouting.title,
|
||||
description:
|
||||
`All ${agentCount} of your subagents name neither a \`model:\` nor an \`effort:\` in their frontmatter. ` +
|
||||
'The `model` field defaults to `inherit`, so each one runs on the main conversation\'s model — omitting ' +
|
||||
'it is not a neutral default but a choice to pay the session\'s rate for every delegated task ' +
|
||||
'(BP-MODEL-001, https://code.claude.com/docs/en/sub-agents). Reasoning effort is a separate axis with ' +
|
||||
'its own frontmatter field and its own default, so a subagent can be routed on either or both ' +
|
||||
'(BP-MODEL-002, https://code.claude.com/docs/en/model-config).',
|
||||
evidence:
|
||||
`authored_agents=${agentCount}; model_pinned=${modelPinned}; effort_pinned=${effortPinned}; ` +
|
||||
'lever=agent frontmatter `model:` / `effort:` (plugin-bundled and fixture agents excluded)',
|
||||
recommendation:
|
||||
'Pin a cheaper `model:` on the subagents whose work is mechanical or read-only (search, extraction, ' +
|
||||
'summarisation) and leave the orchestrating session on the stronger model; pin a lower `effort:` on the ' +
|
||||
'same ones and reserve the high levels for work whose product is judgement. If running everything on one ' +
|
||||
'model is a deliberate policy, suppress this with `CA-GAP-028` in `.config-audit-ignore`.',
|
||||
category: 'model-fit',
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Count authored agents and how many pin each routing axis.
|
||||
* Frontmatter-only read; an unparseable or frontmatter-less file counts as an
|
||||
* agent that pins nothing, matching what Claude Code would load.
|
||||
* @param {CheckContext} ctx
|
||||
* @returns {Promise<{ agentCount: number, modelPinned: number, effortPinned: number }>}
|
||||
*/
|
||||
async function countAgentRouting(ctx) {
|
||||
let agentCount = 0;
|
||||
let modelPinned = 0;
|
||||
let effortPinned = 0;
|
||||
for (const file of ctx.files.filter(f => f.type === 'agent-md')) {
|
||||
agentCount++;
|
||||
const content = await getContent(ctx, file.absPath);
|
||||
if (!content) continue;
|
||||
const { frontmatter } = parseFrontmatter(content);
|
||||
if (!frontmatter) continue;
|
||||
if (isRoutingValue(frontmatter.model) && !isDefaultModel(frontmatter.model)) modelPinned++;
|
||||
if (isRoutingValue(frontmatter.effort)) effortPinned++;
|
||||
}
|
||||
return { agentCount, modelPinned, effortPinned };
|
||||
}
|
||||
|
||||
/**
|
||||
* True for a frontmatter value that actually names something. An empty or
|
||||
* whitespace-only `model:` is a no-op in Claude Code, so it must not read as a pin.
|
||||
* @param {*} v
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function isRoutingValue(v) {
|
||||
return typeof v === 'string' ? v.trim().length > 0 : v != null && v !== false;
|
||||
}
|
||||
|
||||
/**
|
||||
* `inherit` IS the documented default for a subagent's `model` (BP-MODEL-001),
|
||||
* so writing it explicitly routes nothing — the agent still runs on the main
|
||||
* conversation's model. Spelling out a default must not buy silence, or a config
|
||||
* can opt out of the opportunity without changing a single thing about cost.
|
||||
* Effort has no documented sentinel of this kind, so it has no counterpart here.
|
||||
* @param {*} v
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function isDefaultModel(v) {
|
||||
return typeof v === 'string' && v.trim().toLowerCase() === 'inherit';
|
||||
}
|
||||
|
||||
/** @type {GapCheck[]} */
|
||||
export const GAP_CHECKS = [
|
||||
// --- Tier 1: Foundation ---
|
||||
|
|
@ -608,6 +734,13 @@ export async function scan(targetPath, sharedDiscovery) {
|
|||
const hookLever = filterHookLeverFinding({ flaggedHooks });
|
||||
if (hookLever) findings.push(hookLever);
|
||||
|
||||
// Agent model/effort routing lever (C4) — fires only when authored agents
|
||||
// exist and not one of them uses either routing axis. Reads the SAME authored
|
||||
// set as the presence checks, so plugin-bundled and fixture agents cannot
|
||||
// make a machine look routed (M-BUG-13).
|
||||
const routingLever = agentModelRoutingLeverFinding(await countAgentRouting(ctx));
|
||||
if (routingLever) findings.push(routingLever);
|
||||
|
||||
const filesScanned = discovery.files.length;
|
||||
return scannerResult(SCANNER, 'ok', findings, filesScanned, Date.now() - start);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -817,7 +817,7 @@ export async function enumerateRules(repoPath, pluginList = []) {
|
|||
*
|
||||
* @param {string} repoPath
|
||||
* @param {Array<{name:string, path:string}>} [pluginList]
|
||||
* @returns {Promise<Array<{name:string, source:string, pluginName:string|null, path:string, bytes:number, estimatedTokens:number, loadPattern:string, survivesCompaction:string, derivationConfidence:string}>>}
|
||||
* @returns {Promise<Array<{name:string, source:string, pluginName:string|null, path:string, bytes:number, estimatedTokens:number, model:string|null, effort:string|null, loadPattern:string, survivesCompaction:string, derivationConfidence:string}>>}
|
||||
*/
|
||||
export async function enumerateAgents(repoPath, pluginList = []) {
|
||||
const out = [];
|
||||
|
|
@ -842,6 +842,11 @@ export async function enumerateAgents(repoPath, pluginList = []) {
|
|||
path: f.path,
|
||||
bytes: f.size,
|
||||
estimatedTokens: estimateTokens(f.size, 'frontmatter'),
|
||||
// Routing axes (C4). Explicit null rather than an absent key: `model`
|
||||
// defaults to `inherit` and `effort` to the session level, so a consumer
|
||||
// must be able to read "not pinned" without guessing (BP-MODEL-001/002).
|
||||
model: hasText(frontmatter && frontmatter.model) ? frontmatter.model.trim() : null,
|
||||
effort: hasText(frontmatter && frontmatter.effort) ? frontmatter.effort.trim() : null,
|
||||
...lp,
|
||||
});
|
||||
}
|
||||
|
|
|
|||
|
|
@ -209,8 +209,11 @@ export const FINDING_CODES = {
|
|||
},
|
||||
|
||||
// ── GAP: feature-gap-scanner ────────────────────────────────────────────
|
||||
// Keys are GAP_CHECKS[].id (already stable). Dimensions 1–24 in table order,
|
||||
// then the three conditional levers, which the scanner emits after the loop.
|
||||
// Keys are GAP_CHECKS[].id for dimensions, and the lever code for the
|
||||
// conditional levers the scanner emits after the loop. Numbers 1–24 happen to
|
||||
// follow the current table order because that is how the dimensions were first
|
||||
// published — NOT because position determines the number. A new check takes the
|
||||
// next free number wherever it sits in the file (M-BUG-28).
|
||||
GAP: {
|
||||
t1_1: 1,
|
||||
t1_2: 2,
|
||||
|
|
@ -239,6 +242,7 @@ export const FINDING_CODES = {
|
|||
'bundled-skills-lever': 25,
|
||||
'cli-over-mcp-lever': 26,
|
||||
'filter-hook-output-lever': 27,
|
||||
'agent-model-routing-lever': 28,
|
||||
},
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -524,6 +524,30 @@ export const TRANSLATIONS = {
|
|||
description: 'Language-server connections let Claude see types, error messages, and definitions the same way your editor does.',
|
||||
recommendation: 'Set up LSP integration if you work in a typed language.',
|
||||
},
|
||||
// Conditional levers. These are not "a feature you haven't set up" — they
|
||||
// fire only under a measured condition, so the generic _default would
|
||||
// misdescribe them. Every title the scanner can emit needs an entry here
|
||||
// (guarded in tests/scanners/feature-gap-scanner.test.mjs).
|
||||
'Bundled skills add to an over-budget skill listing': {
|
||||
title: 'Built-in skills are crowding an already-full skill list',
|
||||
description: 'Claude Code loads its own built-in skills into the same limited list as yours. Your list is already over budget, so entries risk being cut off and Claude may miss the right skill.',
|
||||
recommendation: 'Turn off the built-in skills to free up room — unless you use them, in which case shorten your own skill descriptions instead.',
|
||||
},
|
||||
'Prefer CLI over MCP for common operations': {
|
||||
title: 'Some connected services load their full tool list every turn',
|
||||
description: 'Most connected services only cost tokens when used, but yours are set to load everything upfront. That weight is there whether you use them or not.',
|
||||
recommendation: 'For services with a command-line equivalent (like `gh` or `aws`), the command line costs nothing until you run it.',
|
||||
},
|
||||
'Filter hook output before it enters context': {
|
||||
title: 'An automation is pasting its full output into the conversation',
|
||||
description: 'An automation that injects its output adds it to every turn that follows. Unfiltered command output can be much larger than the part that actually matters.',
|
||||
recommendation: 'Trim the output inside the script itself, so only the useful lines reach the conversation.',
|
||||
},
|
||||
'Subagents pin neither model nor effort': {
|
||||
title: 'Your helper agents all run at the same cost as your main session',
|
||||
description: 'A subagent that names no model inherits the one you are using, so routine delegated work costs the same as your hardest work. Reasoning effort is a separate dial with the same default.',
|
||||
recommendation: 'Give mechanical agents (search, extraction, summarizing) a smaller model or a lower effort level, and keep the strong settings for the work that needs judgement.',
|
||||
},
|
||||
},
|
||||
patterns: [],
|
||||
_default: {
|
||||
|
|
|
|||
|
|
@ -119,6 +119,10 @@ export function buildManifest(activeConfig) {
|
|||
name: a.name,
|
||||
source: sourceLabel(a, 'project'),
|
||||
estimated_tokens: a.estimatedTokens || 0,
|
||||
// Routing axes (C4) — named explicitly because withLoadPattern copies the
|
||||
// row plus the load-pattern triple, nothing else from the enumeration.
|
||||
model: a.model ?? null,
|
||||
effort: a.effort ?? null,
|
||||
}, a));
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1075,6 +1075,32 @@ describe('enumerateAgents (v5.6)', () => {
|
|||
assert.deepEqual(agents.map(a => a.name).sort(), ['reviewer']);
|
||||
});
|
||||
|
||||
// C4: the two routing axes are part of the inventory, so `whats-active` and
|
||||
// `manifest` can answer "what does each agent actually run on?" without the
|
||||
// reader having to open the files again.
|
||||
it('C4: surfaces model and effort per agent', async () => {
|
||||
const dir = join(root, '.claude', 'agents');
|
||||
await mkdir(dir, { recursive: true });
|
||||
await writeFile(join(dir, 'cheap.md'), '---\nname: cheap\ndescription: mechanical work\nmodel: haiku\neffort: low\n---\nbody\n');
|
||||
const agents = await enumerateAgents(root, []);
|
||||
const a = agents.find(x => x.name === 'cheap');
|
||||
assert.equal(a.model, 'haiku');
|
||||
assert.equal(a.effort, 'low');
|
||||
});
|
||||
|
||||
// Explicit null, not an absent key: `inherit` is the documented default, so a
|
||||
// consumer must be able to tell "not pinned" apart from "field unknown".
|
||||
it('C4: reports null for an agent that pins neither axis', async () => {
|
||||
const dir = join(root, '.claude', 'agents');
|
||||
await mkdir(dir, { recursive: true });
|
||||
await writeFile(join(dir, 'plain.md'), '---\nname: plain\ndescription: no pins\n---\nbody\n');
|
||||
const agents = await enumerateAgents(root, []);
|
||||
const a = agents.find(x => x.name === 'plain');
|
||||
assert.ok('model' in a && 'effort' in a, 'both keys must be present');
|
||||
assert.equal(a.model, null);
|
||||
assert.equal(a.effort, null);
|
||||
});
|
||||
|
||||
// M-BUG-3: CC scans agents dirs recursively, so a valid agent in a subfolder
|
||||
// (e.g. agents/review/security.md) is registered and must be counted.
|
||||
it('M-BUG-3: recurses into agent subdirectories', async () => {
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { FINDING_CODES, RETIRED_CODES, codeNumber, findingId, allFindingIds } from '../../scanners/lib/finding-codes.mjs';
|
||||
import { GAP_CHECKS } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { GAP_CHECKS, LEVERS } from '../../scanners/feature-gap-scanner.mjs';
|
||||
|
||||
describe('finding-code registry', () => {
|
||||
it('gives every check a distinct number within its scanner', () => {
|
||||
|
|
@ -59,7 +59,9 @@ describe('finding-code registry', () => {
|
|||
const missing = shipped.filter((id) => !declared.has(id));
|
||||
assert.deepEqual(missing, [], 'a GAP dimension has no declared code');
|
||||
|
||||
const levers = ['bundled-skills-lever', 'cli-over-mcp-lever', 'filter-hook-output-lever'];
|
||||
// Derived from the scanner, not listed again here: a hand-written copy of
|
||||
// this list is the drift class the registry exists to prevent.
|
||||
const levers = Object.values(LEVERS).map((l) => l.code);
|
||||
const orphans = [...declared].filter((k) => !shipped.includes(k) && !levers.includes(k));
|
||||
assert.deepEqual(orphans, [], 'a declared GAP code matches no shipped dimension');
|
||||
});
|
||||
|
|
|
|||
|
|
@ -11,6 +11,19 @@
|
|||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { parseIgnoreFile, unknownSuppressions } from '../../scanners/lib/suppression.mjs';
|
||||
import { FINDING_CODES } from '../../scanners/lib/finding-codes.mjs';
|
||||
|
||||
/**
|
||||
* The next number no check in `scanner` occupies, DERIVED rather than written
|
||||
* down. An unoccupied number is a moving target — every added check claims one —
|
||||
* so a literal here expires the moment the registry grows, which is what C4
|
||||
* (claiming CA-GAP-028) demonstrated. Derived, the input is unoccupied by
|
||||
* construction; the assertion it feeds is unchanged.
|
||||
*/
|
||||
function firstFreeId(scanner) {
|
||||
const n = Math.max(...Object.values(FINDING_CODES[scanner])) + 1;
|
||||
return `CA-${scanner}-${String(n).padStart(3, '0')}`;
|
||||
}
|
||||
|
||||
describe('unknownSuppressions', () => {
|
||||
it('accepts an exact ID that names a declared check', () => {
|
||||
|
|
@ -19,16 +32,18 @@ describe('unknownSuppressions', () => {
|
|||
});
|
||||
|
||||
it('reports an exact ID that names no declared check', () => {
|
||||
// CA-GAP-099 has never existed; CA-PLH-021 is past the end of PLH's range.
|
||||
const s = parseIgnoreFile('CA-GAP-099\nCA-PLH-021\n');
|
||||
assert.deepEqual(unknownSuppressions(s), ['CA-GAP-099', 'CA-PLH-021']);
|
||||
// CA-GAP-099 has never existed; the PLH one is past the end of PLH's range.
|
||||
const pastEnd = firstFreeId('PLH');
|
||||
const s = parseIgnoreFile(`CA-GAP-099\n${pastEnd}\n`);
|
||||
assert.deepEqual(unknownSuppressions(s), ['CA-GAP-099', pastEnd]);
|
||||
});
|
||||
|
||||
it('reports an ID whose number was retired rather than pretending it matches', () => {
|
||||
// GAP's retired autoMode dimension sat at 25 under the registry's numbering
|
||||
// had it survived; nothing occupies it now.
|
||||
const s = parseIgnoreFile('CA-GAP-028\n');
|
||||
assert.deepEqual(unknownSuppressions(s), ['CA-GAP-028']);
|
||||
it('reports an ID whose number no check occupies rather than pretending it matches', () => {
|
||||
// The registry never reissues a retired key's number, so an ID can name a
|
||||
// hole. Any unoccupied number exercises the same path.
|
||||
const free = firstFreeId('GAP');
|
||||
const s = parseIgnoreFile(`${free}\n`);
|
||||
assert.deepEqual(unknownSuppressions(s), [free]);
|
||||
});
|
||||
|
||||
it('accepts a scanner-wide glob for a real scanner', () => {
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { resolve, join, dirname } from 'node:path';
|
|||
import { fileURLToPath } from 'node:url';
|
||||
import { mkdtemp, mkdir, writeFile, rm } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { scan, opportunitySummary, bundledSkillsLeverFinding, cliOverMcpLeverFinding, filterHookLeverFinding, GAP_CHECKS } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { scan, opportunitySummary, bundledSkillsLeverFinding, cliOverMcpLeverFinding, filterHookLeverFinding, GAP_CHECKS, LEVERS } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { TITLE_TO_ID as GAP_TITLE_TO_ID, TIER_COUNTS, TOTAL_DIMENSIONS } from '../../scanners/lib/scoring.mjs';
|
||||
import { TRANSLATIONS } from '../../scanners/lib/humanizer-data.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
|
|
@ -535,12 +535,21 @@ describe('GAP scanner — retired dimensions (D1)', () => {
|
|||
}
|
||||
});
|
||||
|
||||
it('keeps the three title tables carrying exactly the same titles', () => {
|
||||
it('keeps the scoring title table carrying exactly the dimension titles', () => {
|
||||
const checks = GAP_CHECKS.map(g => g.title).sort();
|
||||
const scoring = Object.keys(GAP_TITLE_TO_ID).sort();
|
||||
const humanizer = Object.keys(TRANSLATIONS.GAP.static).sort();
|
||||
assert.deepEqual(scoring, checks, 'scoring TITLE_TO_ID drifted from GAP_CHECKS');
|
||||
assert.deepEqual(humanizer, checks, 'humanizer TRANSLATIONS.GAP drifted from GAP_CHECKS');
|
||||
});
|
||||
|
||||
// The humanizer's table is NOT the dimension table: levers are findings too,
|
||||
// and a title with no static entry falls through to the generic GAP _default
|
||||
// ("You have a feature opportunity worth a look") — wrong for a budget lever.
|
||||
// The invariant is therefore coverage of EVERY title GAP can emit, asserted
|
||||
// blanket rather than as a relation between the two dimension tables.
|
||||
it('humanizes every title the scanner can emit — dimensions AND levers', () => {
|
||||
const emittable = [...GAP_CHECKS.map(g => g.title), ...Object.values(LEVERS).map(l => l.title)].sort();
|
||||
const humanizer = Object.keys(TRANSLATIONS.GAP.static).sort();
|
||||
assert.deepEqual(humanizer, emittable, 'humanizer TRANSLATIONS.GAP drifted from the emittable titles');
|
||||
});
|
||||
|
||||
// The scoring denominators are a FOURTH copy of the dimension inventory, and
|
||||
|
|
|
|||
187
tests/scanners/gap-agent-model-routing.test.mjs
Normal file
187
tests/scanners/gap-agent-model-routing.test.mjs
Normal file
|
|
@ -0,0 +1,187 @@
|
|||
/**
|
||||
* C4 — agent model/effort routing lever (CA-GAP-028).
|
||||
*
|
||||
* A LEVER, not a dimension: it fires only when authored agents exist, so it has
|
||||
* no meaningful "present/absent" reading on a config with no agents at all —
|
||||
* exactly the shape the three existing levers already have. The design was
|
||||
* decided by measurement, not taste: as a GAP_CHECKS dimension it would have
|
||||
* counted as vacuously-present on the agent-less marketplace-medium fixture,
|
||||
* moving MAX_WEIGHTED 41→42 and utilization 44→45, which flips `segment`
|
||||
* "Developing"→"Competent" in the frozen v5.0.0 posture baseline (segment is
|
||||
* NOT among the fields strip-retired-gap.mjs drops from comparison).
|
||||
*
|
||||
* The silence has two independent causes and they must not be conflated:
|
||||
* - no authored agents at all (t2_6 "No custom subagents" owns that case), and
|
||||
* - the only agents on disk being plugin-bundled, which isAuthoredConfig
|
||||
* excludes.
|
||||
* P5 below pins the second one specifically, so that arm cannot pass for the
|
||||
* first one's reason.
|
||||
*/
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { resolve, join } from 'node:path';
|
||||
import { mkdtemp, mkdir, writeFile } from 'node:fs/promises';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { scan, agentModelRoutingLeverFinding } from '../../scanners/feature-gap-scanner.mjs';
|
||||
import { discoverConfigFiles } from '../../scanners/lib/file-discovery.mjs';
|
||||
import { withHermeticHome } from '../helpers/hermetic-home.mjs';
|
||||
|
||||
const LEVER_TITLE = 'Subagents pin neither model nor effort';
|
||||
|
||||
/** Build a throwaway project; `agents` maps a relative dir to agent frontmatter lines. */
|
||||
async function makeProject(agents) {
|
||||
const root = await mkdtemp(join(tmpdir(), 'config-audit-c4-'));
|
||||
// A CLAUDE.md keeps the fixture from being a bare directory; irrelevant to the lever.
|
||||
await writeFile(join(root, 'CLAUDE.md'), '# Project\n');
|
||||
for (const [relDir, files] of Object.entries(agents)) {
|
||||
const dir = join(root, relDir);
|
||||
await mkdir(dir, { recursive: true });
|
||||
for (const [name, frontmatter] of Object.entries(files)) {
|
||||
await writeFile(
|
||||
join(dir, name),
|
||||
`---\nname: ${name.replace(/\.md$/, '')}\ndescription: does a thing\n${frontmatter}---\nBody.\n`,
|
||||
);
|
||||
}
|
||||
}
|
||||
return root;
|
||||
}
|
||||
|
||||
/** Scan a project with hermetic HOME and no global discovery. */
|
||||
async function scanProject(root) {
|
||||
const discovery = await discoverConfigFiles(resolve(root));
|
||||
const result = await withHermeticHome(() => scan(resolve(root), discovery));
|
||||
return { result, discovery };
|
||||
}
|
||||
|
||||
const leverFindings = (result) => result.findings.filter(f => f.title === LEVER_TITLE);
|
||||
const hasNoSubagentsGap = (result) => result.findings.some(f => f.title === 'No custom subagents');
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Unit — the pure lever function
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('agentModelRoutingLeverFinding (pure)', () => {
|
||||
it('stays silent when no authored agents exist', () => {
|
||||
assert.equal(
|
||||
agentModelRoutingLeverFinding({ agentCount: 0, modelPinned: 0, effortPinned: 0 }),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
it('stays silent when at least one agent pins model', () => {
|
||||
assert.equal(
|
||||
agentModelRoutingLeverFinding({ agentCount: 3, modelPinned: 1, effortPinned: 0 }),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
it('stays silent when at least one agent pins effort (the separate axis)', () => {
|
||||
assert.equal(
|
||||
agentModelRoutingLeverFinding({ agentCount: 3, modelPinned: 0, effortPinned: 1 }),
|
||||
null,
|
||||
);
|
||||
});
|
||||
|
||||
it('fires when agents exist and neither axis is used anywhere', () => {
|
||||
const f = agentModelRoutingLeverFinding({ agentCount: 3, modelPinned: 0, effortPinned: 0 });
|
||||
assert.ok(f, 'expected a finding');
|
||||
assert.equal(f.id, 'CA-GAP-028');
|
||||
assert.equal(f.scanner, 'GAP');
|
||||
assert.equal(f.severity, 'info');
|
||||
assert.equal(f.category, 'model-fit');
|
||||
assert.equal(f.title, LEVER_TITLE);
|
||||
});
|
||||
|
||||
it('cites both register entries and their primary sources', () => {
|
||||
const f = agentModelRoutingLeverFinding({ agentCount: 2, modelPinned: 0, effortPinned: 0 });
|
||||
const text = `${f.description} ${f.recommendation}`;
|
||||
assert.match(text, /BP-MODEL-001/);
|
||||
assert.match(text, /BP-MODEL-002/);
|
||||
assert.match(text, /code\.claude\.com\/docs\/en\/sub-agents/);
|
||||
assert.match(text, /code\.claude\.com\/docs\/en\/model-config/);
|
||||
});
|
||||
|
||||
it('carries the measured counts as evidence', () => {
|
||||
const f = agentModelRoutingLeverFinding({ agentCount: 4, modelPinned: 0, effortPinned: 0 });
|
||||
assert.match(f.evidence, /authored_agents=4/);
|
||||
assert.match(f.evidence, /model_pinned=0/);
|
||||
assert.match(f.evidence, /effort_pinned=0/);
|
||||
});
|
||||
|
||||
it('frames the opportunity without asserting the config is wrong', () => {
|
||||
const f = agentModelRoutingLeverFinding({ agentCount: 2, modelPinned: 0, effortPinned: 0 });
|
||||
// `inherit` is the documented default and the reason the opportunity exists.
|
||||
assert.match(f.description, /inherit/i);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Integration — fire/silent matrix through scan()
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('GAP scanner — C4 fire/silent matrix', () => {
|
||||
it('P1: fires once when authored agents pin neither axis', async () => {
|
||||
const root = await makeProject({
|
||||
'.claude/agents': { 'alpha.md': '', 'beta.md': '' },
|
||||
});
|
||||
const { result } = await scanProject(root);
|
||||
const hits = leverFindings(result);
|
||||
assert.equal(hits.length, 1, 'expected exactly one lever finding');
|
||||
assert.equal(hits[0].id, 'CA-GAP-028');
|
||||
});
|
||||
|
||||
// Found by dogfooding this repo's own machine: several installed agents write
|
||||
// `model: inherit` explicitly. `inherit` IS the documented default
|
||||
// (BP-MODEL-001), so naming it routes nothing and must not buy silence — the
|
||||
// opportunity is exactly as open as with the field absent.
|
||||
it('P2b: fires when the only "pin" is the default value spelled out', async () => {
|
||||
const root = await makeProject({
|
||||
'.claude/agents': { 'alpha.md': 'model: inherit\n', 'beta.md': '' },
|
||||
});
|
||||
const { result } = await scanProject(root);
|
||||
const hits = leverFindings(result);
|
||||
assert.equal(hits.length, 1, 'model: inherit must not count as routing');
|
||||
assert.match(hits[0].evidence, /model_pinned=0/);
|
||||
});
|
||||
|
||||
it('P2: silent when one agent pins model', async () => {
|
||||
const root = await makeProject({
|
||||
'.claude/agents': { 'alpha.md': 'model: haiku\n', 'beta.md': '' },
|
||||
});
|
||||
const { result } = await scanProject(root);
|
||||
assert.equal(leverFindings(result).length, 0);
|
||||
});
|
||||
|
||||
it('P3: silent when one agent pins effort and none pins model', async () => {
|
||||
const root = await makeProject({
|
||||
'.claude/agents': { 'alpha.md': 'effort: low\n', 'beta.md': '' },
|
||||
});
|
||||
const { result } = await scanProject(root);
|
||||
assert.equal(leverFindings(result).length, 0);
|
||||
});
|
||||
|
||||
it('P4: silent when there are no agents at all (t2_6 owns that case)', async () => {
|
||||
const root = await makeProject({});
|
||||
const { result } = await scanProject(root);
|
||||
assert.equal(leverFindings(result).length, 0);
|
||||
assert.ok(hasNoSubagentsGap(result), 'expected the "No custom subagents" dimension instead');
|
||||
});
|
||||
|
||||
it('P5: silent for the EXCLUSION reason when the only agent is plugin-bundled', async () => {
|
||||
const root = await makeProject({
|
||||
'.claude/plugins/somePlugin/agents': { 'vendored.md': '' },
|
||||
});
|
||||
const { result, discovery } = await scanProject(root);
|
||||
// The discriminator: the file IS on disk and IS discovered as an agent —
|
||||
// so silence here cannot be the P4 "no agent files" reason.
|
||||
assert.ok(
|
||||
discovery.files.some(f => f.type === 'agent-md'),
|
||||
'fixture invalid: no agent file was discovered at all',
|
||||
);
|
||||
assert.equal(leverFindings(result).length, 0);
|
||||
assert.ok(
|
||||
hasNoSubagentsGap(result),
|
||||
'the plugin-bundled agent must not count as an authored subagent either',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
|
@ -123,7 +123,7 @@ describe('buildManifest — load-pattern accounting (unit)', () => {
|
|||
],
|
||||
agents: [
|
||||
{ name: 'a1', source: 'project', pluginName: null, estimatedTokens: 5,
|
||||
...deriveLoadPattern('agent') },
|
||||
model: 'sonnet', effort: null, ...deriveLoadPattern('agent') },
|
||||
],
|
||||
outputStyles: [
|
||||
{ name: 's1', source: 'project', pluginName: null, estimatedTokens: 7,
|
||||
|
|
@ -165,6 +165,14 @@ describe('buildManifest — load-pattern accounting (unit)', () => {
|
|||
assert.equal(byName('output-style', 's1').loadPattern, 'always');
|
||||
});
|
||||
|
||||
// C4. withLoadPattern copies the explicit row object plus three load-pattern
|
||||
// fields and nothing else, so routing fields do NOT ride along from the
|
||||
// enumeration — they have to be named in the row.
|
||||
it('C4: carries model and effort on agent rows', () => {
|
||||
assert.equal(byName('agent', 'a1').model, 'sonnet');
|
||||
assert.equal(byName('agent', 'a1').effort, null);
|
||||
});
|
||||
|
||||
it('tags MCP servers always and hooks external', () => {
|
||||
assert.equal(byName('mcp-server', 'm1').loadPattern, 'always');
|
||||
assert.equal(byName('hook', 'PreToolUse:Edit').loadPattern, 'external');
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue