feat(voyage): W2 impl (S4) — gate resolver model, adopt native effort:, doc-truth
S4 of the 2.1.181 upgrade — implementation, not a gate. TDD: failing test written first for the resolver gate, then the fix; suite green throughout. - Resolver MAJOR (FIX): lib/profiles/phase-signal-resolver.mjs now imports BASE_ALLOWED_MODELS from profile-validator and gates `model` (if 'model' in entry && BASE_ALLOWED_MODELS.includes(entry.model)), mirroring the EFFORT_LEVELS gate one line up. Out-of-allowlist models (gpt-4, haiku) are dropped instead of handed to an agent spawn — defense-in-depth behind brief-validator's validation-time check. No circular import (brief-validator already imports the same symbol). +2 tests (drops-invalid / keeps-valid). - Native effort: (SHIP, static additive): effort: frontmatter on 8 agents — retrieval (task-finder, git-historian, dependency-tracer, architecture-mapper) = medium; adversarial-reasoning (plan-critic, risk-assessor, contrarian-researcher, review-coordinator) = high. The other 15 stay unset -> inherit Opus-4.8 default (high). This per-spawn REASONING effort is a different axis from brief phase_signals.effort (ORCHESTRATION shape) per the S3 decision. - Doc-truth + axis distinction: new canonical docs/profiles.md §Model & effort axes (opus->Opus 4.8 default-high; orchestration vs reasoning effort table; native-effort precedence; per-agent levels). Short notes in CLAUDE.md (after Agents table) and README.md (Cost profile), both pointing to profiles.md. - Open (non-blocking, unchanged): only STATIC effort shipped — the verified-safe minimum. Profile-driven DYNAMIC effort still needs verification of the per-spawn effort param or env-var injection. Matrix: new "S4 resolutions" section. Tests 582 total / 580 pass / 0 fail / 2 skip (was 578 pass; +2). claude plugin validate passes (only pre-existing root-CLAUDE.md warning). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
This commit is contained in:
parent
fbf83f2271
commit
fdd3ad80d7
14 changed files with 102 additions and 1 deletions
|
|
@ -52,6 +52,8 @@ Full flag reference for each command (modes, `--gates`, `--profile`, breaking ch
|
|||
| contrarian-researcher | opus | Counter-evidence, overlooked alternatives |
|
||||
| gemini-bridge | opus | Gemini Deep Research second opinion (conditional) |
|
||||
|
||||
> **Model & effort (CC 2.1.154+).** `opus` resolves to **Opus 4.8** (default reasoning effort `high`); `sonnet` to Sonnet 4.6. Select agents carry native `effort:` — retrieval agents (`task-finder`, `git-historian`, `dependency-tracer`, `architecture-mapper`) at `medium`, adversarial-reasoning agents (`plan-critic`, `risk-assessor`, `contrarian-researcher`, `review-coordinator`) at `high`. This native per-spawn **reasoning** effort is a different axis from brief `phase_signals.effort` (orchestration shape — which agents/passes run). See `docs/profiles.md` §Model & effort axes.
|
||||
|
||||
## Reference docs (read on demand)
|
||||
|
||||
- **Architecture, workflows, project-directory contract, state, terminology:** `docs/architecture.md`
|
||||
|
|
|
|||
|
|
@ -777,6 +777,8 @@ Default JSONL stats stream (`${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl`) is unchan
|
|||
|
||||
Opus runs the orchestrators (one per command) and the executor (one per plan session). Sonnet runs the exploration and review swarms (5–10 agents per command, with effort/turn limits). The pipeline front-loads cheap Sonnet work so Opus only does synthesis and execution. Typical total: comparable to a long single Claude Code session — the per-command cost is published in `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` if you want exact numbers.
|
||||
|
||||
The `opus` alias resolves to **Opus 4.8** (default reasoning effort `high`) and `sonnet` to Sonnet 4.6. Note two distinct effort axes that share the word "effort": brief `phase_signals.effort` (low/standard/high) tunes *orchestration shape* — how many agents and passes run — while native `effort:` on selected agents (retrieval at `medium`, adversarial-reasoning at `high`) tunes the *per-spawn reasoning budget*. See [`docs/profiles.md`](docs/profiles.md) § Model & effort axes.
|
||||
|
||||
For per-profile cost estimates, see [`docs/profiles.md`](docs/profiles.md).
|
||||
|
||||
## Requirements
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ description: |
|
|||
</commentary>
|
||||
</example>
|
||||
model: opus
|
||||
effort: medium
|
||||
color: cyan
|
||||
tools: ["Read", "Glob", "Grep", "Bash"]
|
||||
---
|
||||
|
|
|
|||
|
|
@ -25,6 +25,7 @@ description: |
|
|||
</commentary>
|
||||
</example>
|
||||
model: opus
|
||||
effort: high
|
||||
color: red
|
||||
tools: ["WebSearch", "WebFetch", "mcp__tavily__tavily_search", "mcp__tavily__tavily_research"]
|
||||
---
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ description: |
|
|||
</commentary>
|
||||
</example>
|
||||
model: opus
|
||||
effort: medium
|
||||
color: blue
|
||||
tools: ["Read", "Glob", "Grep", "Bash"]
|
||||
---
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ description: |
|
|||
</commentary>
|
||||
</example>
|
||||
model: opus
|
||||
effort: medium
|
||||
color: yellow
|
||||
tools: ["Bash", "Read", "Glob", "Grep"]
|
||||
---
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ description: |
|
|||
</commentary>
|
||||
</example>
|
||||
model: opus
|
||||
effort: high
|
||||
color: red
|
||||
tools: ["Read", "Glob", "Grep"]
|
||||
---
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ description: |
|
|||
Judge filters, Cloudflare reasonableness filter, verdict computation.
|
||||
Synthesis-level inference across files is forbidden in v1.0.
|
||||
model: opus
|
||||
effort: high
|
||||
color: yellow
|
||||
tools: ["Read", "Glob", "Grep"]
|
||||
---
|
||||
|
|
|
|||
|
|
@ -22,6 +22,7 @@ description: |
|
|||
</commentary>
|
||||
</example>
|
||||
model: opus
|
||||
effort: high
|
||||
color: yellow
|
||||
tools: ["Read", "Glob", "Grep", "Bash"]
|
||||
---
|
||||
|
|
|
|||
|
|
@ -23,6 +23,7 @@ description: |
|
|||
</commentary>
|
||||
</example>
|
||||
model: opus
|
||||
effort: medium
|
||||
color: green
|
||||
tools: ["Read", "Glob", "Grep", "Bash"]
|
||||
---
|
||||
|
|
|
|||
|
|
@ -108,6 +108,16 @@ S3 was a decision-gate. Two-track evidence (codebase map of `phase_signals`/`pha
|
|||
|
||||
**S4 scope (now unblocked):** (1) resolver MAJOR model-gate fix; (2) additive native `effort:` — verified-safe minimum is static per-agent frontmatter (retrieval agents `task-finder`/`git-historian`/`dependency-tracer`/`architecture-mapper` lower; reasoning agents `plan-critic`/`risk-assessor`/`contrarian-researcher`/`review-coordinator` stay high); (3) doc-truth model/effort updates; (4) document the effort-axis distinction. **Open (non-blocking):** profile-driven *dynamic* effort needs verification — per-spawn `effort` param is **unverified** (only `model` is documented per-spawn); dynamic would otherwise need env-var injection.
|
||||
|
||||
## S4 resolutions (W2 implementation — 2026-06-18)
|
||||
|
||||
S4 implemented the four scope items above. TDD: a failing test was written first for the resolver gate, then the fix; full suite green throughout.
|
||||
|
||||
- **Resolver MAJOR — FIXED.** `lib/profiles/phase-signal-resolver.mjs` now imports `BASE_ALLOWED_MODELS` from `profile-validator.mjs` and gates `model` (`if ('model' in entry && BASE_ALLOWED_MODELS.includes(entry.model))`), mirroring the `EFFORT_LEVELS` gate one line above. Out-of-allowlist models (e.g. `gpt-4`, `haiku`) are now dropped (treated as absent) instead of handed through to an agent spawn — defense-in-depth behind `brief-validator`'s validation-time check. 2 new tests in `tests/lib/phase-signal-resolver.test.mjs` (drops-invalid + keeps-valid). No circular import (`brief-validator` already imports the same symbol; `profile-validator` depends only on `util/`).
|
||||
- **Native `effort:` — SHIPPED (static, additive).** Added `effort:` frontmatter to the 8 named agents: retrieval (`task-finder`, `git-historian`, `dependency-tracer`, `architecture-mapper`) → `medium`; adversarial-reasoning (`plan-critic`, `risk-assessor`, `contrarian-researcher`, `review-coordinator`) → `high`. The other 15 agents stay unset → inherit Opus-4.8 default (`high`). `claude plugin validate` passes (effort is a valid frontmatter field; only the pre-existing root-CLAUDE.md warning remains).
|
||||
- **Doc-truth + axis distinction — DONE.** New canonical section `docs/profiles.md` §Model & effort axes (opus→Opus 4.8 default-`high`; orchestration `phase_signals.effort` vs native reasoning `effort:` table; native-effort precedence; which agents carry which level). Short note added to `CLAUDE.md` (after Agents table) and `README.md` (Cost profile section), both pointing to the profiles.md section.
|
||||
- **Open (non-blocking, unchanged).** Only *static* per-agent effort shipped — the verified-safe minimum. Profile-driven *dynamic* effort still needs verification of the per-spawn `effort` param (only `model` is documented per-spawn) or env-var injection (`CLAUDE_CODE_EFFORT_LEVEL`). Deferred to W2 profile design.
|
||||
- **Tests:** 582 total / 580 pass / 0 fail / 2 skip (was 578 pass; +2 new resolver tests).
|
||||
|
||||
## W1 — Orchestration architecture (multi-week, empirical)
|
||||
|
||||
| ID | Change (version) | Type | Voyage relevance | Decision | Rationale |
|
||||
|
|
|
|||
|
|
@ -26,6 +26,45 @@ runs (Step 17 calibration was deferred — see
|
|||
`tests/synthetic/profile-jaccard-calibration.md`). If you observe
|
||||
economy-plan quality regressions, fall back to `balanced`.
|
||||
|
||||
## Model & effort axes
|
||||
|
||||
`opus` and `sonnet` are model **aliases**, not pinned ids. As of Claude Code
|
||||
2.1.154 the `opus` alias resolves to **Opus 4.8**, whose default reasoning
|
||||
effort is **`high`**; `sonnet` resolves to Sonnet 4.6. The profile table above
|
||||
selects *which alias* runs each phase — it does not touch reasoning effort.
|
||||
|
||||
Two different things share the word "effort" in Voyage. They are **orthogonal
|
||||
axes** — same name, different mechanism:
|
||||
|
||||
| | **Orchestration effort** | **Reasoning effort (native)** |
|
||||
|---|---|---|
|
||||
| Where | brief `phase_signals.effort`, consumed by command prose | native Claude Code `effort:` in agent frontmatter |
|
||||
| Values | `low` / `standard` / `high` | `low` / `medium` / `high` / `xhigh` / `max` |
|
||||
| Controls | *which agents/passes/gates run* (swarm cardinality, gate strictness, sequential-vs-parallel) | *per-spawn reasoning-token budget* the harness gives one agent |
|
||||
| `low` means | "run fewer agents" (the `--quick`-equivalent code-path) | "think with a smaller budget" |
|
||||
| Applied by | Voyage command logic | the harness, at spawn time |
|
||||
|
||||
The `phase-signal-resolver.mjs` helper only reads the **orchestration** axis
|
||||
(`phase_signals.effort`, gated against `low/standard/high`) plus the optional
|
||||
per-phase `model` (gated against `['sonnet','opus']`). It never emits native
|
||||
`effort:`.
|
||||
|
||||
**Native `effort:` on agents.** Voyage sets the reasoning axis statically on
|
||||
selected agents, additively over the Opus-4.8 default:
|
||||
|
||||
- **Retrieval agents → `medium`:** `task-finder`, `git-historian`,
|
||||
`dependency-tracer`, `architecture-mapper` (structured discovery, not deep
|
||||
multi-step reasoning).
|
||||
- **Adversarial-reasoning agents → `high`:** `plan-critic`, `risk-assessor`,
|
||||
`contrarian-researcher`, `review-coordinator` (synthesis and stress-testing
|
||||
where reasoning depth pays off).
|
||||
- **All other agents:** unset → inherit the model default (Opus 4.8 = `high`).
|
||||
|
||||
Native-effort precedence (harness): env `CLAUDE_CODE_EFFORT_LEVEL` > frontmatter
|
||||
`effort:` > session setting > model default. `MAX_THINKING_TOKENS=0` (or
|
||||
`--thinking disabled`) overrides effort entirely. Enterprise `availableModels`
|
||||
constrains the *model* alias only — it does **not** bound effort.
|
||||
|
||||
## Decision tree
|
||||
|
||||
```
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@
|
|||
import { readFileSync, existsSync } from 'node:fs';
|
||||
import { parseDocument } from '../util/frontmatter.mjs';
|
||||
import { PHASE_SIGNAL_PHASES, EFFORT_LEVELS } from '../validators/brief-validator.mjs';
|
||||
import { BASE_ALLOWED_MODELS } from '../validators/profile-validator.mjs';
|
||||
|
||||
/**
|
||||
* Resolve a brief's phase_signal entry for one phase.
|
||||
|
|
@ -27,6 +28,8 @@ import { PHASE_SIGNAL_PHASES, EFFORT_LEVELS } from '../validators/brief-validato
|
|||
* - No entry for the requested phase
|
||||
*
|
||||
* Returns partial `{effort}` (with `model: undefined`) when the signal omits model.
|
||||
* `effort` is gated against EFFORT_LEVELS and `model` against BASE_ALLOWED_MODELS;
|
||||
* values outside those allowlists are dropped (treated as absent).
|
||||
*/
|
||||
export function resolvePhaseSignal(briefFrontmatter, phase) {
|
||||
if (!briefFrontmatter || typeof briefFrontmatter !== 'object') return null;
|
||||
|
|
@ -37,7 +40,11 @@ export function resolvePhaseSignal(briefFrontmatter, phase) {
|
|||
if (entry && typeof entry === 'object' && entry.phase === phase) {
|
||||
const out = {};
|
||||
if ('effort' in entry && EFFORT_LEVELS.includes(entry.effort)) out.effort = entry.effort;
|
||||
if ('model' in entry) out.model = entry.model;
|
||||
// MAJOR fix (S4): gate `model` against BASE_ALLOWED_MODELS, mirroring the
|
||||
// effort gate above. brief-validator already rejects out-of-allowlist
|
||||
// models at validation time; this is defense-in-depth so a brief that
|
||||
// slipped validation cannot hand a junk model to an agent spawn.
|
||||
if ('model' in entry && BASE_ALLOWED_MODELS.includes(entry.model)) out.model = entry.model;
|
||||
return out;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -48,6 +48,39 @@ test('resolvePhaseSignal — defensive: null/non-object input returns null', ()
|
|||
assert.equal(resolvePhaseSignal({ phase_signals: 'not-array' }, 'plan'), null);
|
||||
});
|
||||
|
||||
test('resolvePhaseSignal — drops model not in BASE_ALLOWED_MODELS (defense-in-depth gate)', () => {
|
||||
// MAJOR fix (S4): line that copies `model` must gate against the same
|
||||
// allowlist brief-validator uses (BASE_ALLOWED_MODELS = ['sonnet','opus']),
|
||||
// mirroring how effort is gated against EFFORT_LEVELS. A brief that slipped
|
||||
// validation (hand-edited, validation skipped) must not hand a junk model
|
||||
// string to a command that then spawns an agent with `model: <junk>`.
|
||||
const fm = {
|
||||
phase_signals: [
|
||||
{ phase: 'plan', effort: 'high', model: 'gpt-4' }, // invalid model
|
||||
{ phase: 'review', effort: 'standard', model: 'haiku' }, // not in BASE_ALLOWED_MODELS
|
||||
],
|
||||
};
|
||||
const plan = resolvePhaseSignal(fm, 'plan');
|
||||
assert.equal(plan.effort, 'high', 'valid effort still resolves');
|
||||
assert.equal(plan.model, undefined, 'invalid model "gpt-4" must be dropped');
|
||||
assert.ok(!('model' in plan), 'model key absent when value not in allowlist');
|
||||
|
||||
const review = resolvePhaseSignal(fm, 'review');
|
||||
assert.equal(review.model, undefined, 'haiku must be dropped (not in BASE_ALLOWED_MODELS)');
|
||||
assert.ok(!('model' in review), 'model key absent for haiku');
|
||||
});
|
||||
|
||||
test('resolvePhaseSignal — keeps valid models (sonnet, opus) after gating', () => {
|
||||
const fm = {
|
||||
phase_signals: [
|
||||
{ phase: 'research', effort: 'low', model: 'sonnet' },
|
||||
{ phase: 'execute', effort: 'high', model: 'opus' },
|
||||
],
|
||||
};
|
||||
assert.equal(resolvePhaseSignal(fm, 'research').model, 'sonnet');
|
||||
assert.equal(resolvePhaseSignal(fm, 'execute').model, 'opus');
|
||||
});
|
||||
|
||||
test('resolvePhaseSignalFromFile + CLI shim — writes JSON to stdout, exit 0', () => {
|
||||
const fixture = join(tmpdir(), `phase-signal-test-${process.pid}.md`);
|
||||
writeFileSync(fixture, `---
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue