diff --git a/README.md b/README.md index 9f25ca6..10b7732 100644 --- a/README.md +++ b/README.md @@ -192,7 +192,7 @@ If you say "stop" or "enough" during Phase 4, the current review findings are su Deep, multi-phase research that combines local codebase analysis with external knowledge. Uses specialized agent swarms to investigate multiple dimensions in parallel, then triangulates findings. -A parallel swarm of up to 5 local + 4 external Sonnet agents investigates 3–8 research dimensions, with optional Gemini Deep Research as an independent second opinion. Findings are triangulated (local vs. external, confidence per dimension, contradictions flagged) and synthesized into a structured research brief. +A parallel swarm of up to 5 local + 4 external agents investigates 3–8 research dimensions, with optional Gemini Deep Research as an independent second opinion. Findings are triangulated (local vs. external, confidence per dimension, contradictions flagged) and synthesized into a structured research brief. Output: - With `--project `: `{dir}/research/{NN}-{slug}.md` (auto-incremented index) @@ -220,7 +220,7 @@ Research uses up to 5 local agents (architecture-mapper, dependency-tracer, task Produces an implementation plan detailed enough for autonomous execution. **v2.0 breaking change:** requires `--brief` or `--project`. There is no longer an interview inside `/trekplan` — use `/trekbrief` first. -After `brief-reviewer` validates the input brief, 6–8 Sonnet exploration agents analyze the codebase in parallel and merge findings into a synthesis. Optional research briefs (`--research`, or auto-discovered in `{project_dir}/research/`) enrich the plan; `architecture/overview.md` priors are loaded if an opt-in upstream architect plugin (not bundled) produced one. Opus then writes the plan with per-step YAML manifests, which `plan-critic` (9 dimensions) and `scope-guardian` adversarially review before handoff. +After `brief-reviewer` validates the input brief, 6–8 exploration agents analyze the codebase in parallel and merge findings into a synthesis. Optional research briefs (`--research`, or auto-discovered in `{project_dir}/research/`) enrich the plan; `architecture/overview.md` priors are loaded if an opt-in upstream architect plugin (not bundled) produced one. Opus then writes the plan with per-step YAML manifests, which `plan-critic` (9 dimensions) and `scope-guardian` adversarially review before handoff. Output: - With `--project `: `{dir}/plan.md` @@ -263,7 +263,7 @@ Every implementation step includes: - **Checkpoint:** — git commit after success - **Manifest:** — the objective completion predicate (Hard Rule 17) -Exploration uses 6–8 Sonnet agents in parallel (architecture-mapper, dependency-tracer, task-finder, test-strategist, git-historian, risk-assessor, plus convention-scanner on medium+ codebases and research-scout when unfamiliar tech is detected). Adversarial review then runs `brief-reviewer`, `plan-critic` (9 dimensions, no-placeholder enforcement, manifest audit), and `scope-guardian` (creep + gap detection). Per-agent details in [`agents/`](agents/). +Exploration uses 6–8 agents in parallel (architecture-mapper, dependency-tracer, task-finder, test-strategist, git-historian, risk-assessor, plus convention-scanner on medium+ codebases and research-scout when unfamiliar tech is detected). Adversarial review then runs `brief-reviewer`, `plan-critic` (9 dimensions, no-placeholder enforcement, manifest audit), and `scope-guardian` (creep + gap detection). Per-agent details in [`agents/`](agents/). --- @@ -755,16 +755,16 @@ Three built-in model profiles plus operator-defined `.yaml` (drop in `li | Profile | Brief | Research | Plan | Execute | Review | Continue | Use case | |---------|-------|----------|------|---------|--------|----------|----------| -| `economy` | sonnet | sonnet | sonnet | sonnet | sonnet | sonnet | Lowest cost; high-confidence small-scope tasks | -| `balanced` (default) | sonnet | sonnet | opus | sonnet | opus | sonnet | Default — opus where reasoning depth pays off | -| `premium` | opus | sonnet | opus | sonnet | opus | sonnet | Critical-path planning + review when budget allows | +| `economy` | sonnet | sonnet | sonnet | sonnet | sonnet | sonnet | Lowest cost; high-confidence small-scope tasks (opt-in via `--profile economy`) | +| `balanced` | sonnet | sonnet | opus | sonnet | opus | sonnet | Mixed — opus where reasoning depth pays off (opt-in via `--profile balanced`) | +| `premium` (default) | opus | opus | opus | opus | opus | opus | Maximum quality — Opus on every phase (default since the 2026-05-13 operator decision) | Lookup order: 1. Explicit `--profile ` flag passed to the command 2. Plan-file frontmatter `profile:` (when resuming via `/trekexecute --resume` or `/trekcontinue`) 3. `VOYAGE_PROFILE` environment variable -4. Default `balanced` +4. Default `premium` See [`docs/profiles.md`](docs/profiles.md) for the decision tree, custom-profile authoring, and cost estimation disclaimer (the per-profile cost numbers are *anslag*, not contractual SLAs). @@ -782,7 +782,7 @@ Default JSONL stats stream (`${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl`) is unchan ## Cost profile -Opus runs the orchestrators (one per command) and the executor (one per plan session). Sonnet runs the exploration and review swarms (5–10 agents per command, with effort/turn limits). The pipeline front-loads cheap Sonnet work so Opus only does synthesis and execution. Typical total: comparable to a long single Claude Code session — the per-command cost is published in `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` if you want exact numbers. +The default `premium` profile runs **Opus on every phase** — the orchestrator (one per command), the exploration and review swarms (5–10 sub-agents per command, all `model: opus`-pinned in `agents/*.md`), and the executor (one per plan session). The model is **uniform per phase**: there is no "Opus orchestrates, Sonnet runs the swarms" split — a phase resolves to one model and both the orchestrator and its sub-agents use it. For cheaper runs, opt into `--profile balanced` (Sonnet on brief/research/execute/continue, Opus on plan + review) or `--profile economy` (Sonnet everywhere). Per-command cost is published in `${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl` if you want exact numbers. The `opus` alias resolves to **Opus 4.8** (default reasoning effort `high`) and `sonnet` to Sonnet 4.6. Note two distinct effort axes that share the word "effort": brief `phase_signals.effort` (low/standard/high) tunes *orchestration shape* — how many agents and passes run — while native `effort:` on selected agents (retrieval at `medium`, adversarial-reasoning at `high`) tunes the *per-spawn reasoning budget*. See [`docs/profiles.md`](docs/profiles.md) § Model & effort axes. @@ -801,7 +801,7 @@ Top-level layout: ``` trekplan/ -├── agents/ 23 specialized agents (sonnet for exploration + review, opus for orchestration) +├── agents/ 23 specialized agents (all `model: opus`-pinned; per-phase model set by `--profile`) ├── commands/ 6 slash commands (trekbrief, trekresearch, trekplan, trekexecute, trekreview, trekcontinue) + trekplan-end-session helper ├── templates/ Frontmatter templates for brief, research, plan, session, launch ├── hooks/ 5 hooks (pre-bash, pre-write, session-title, post-bash-stats, pre-compact-flush) @@ -844,7 +844,7 @@ synthesis. The default for `/trekbrief`, `/trekresearch`, `/trekplan`, and `/trekexecute` is `opus` (deep reasoning). To run on Sonnet for cost or latency, search-and-replace -the frontmatter in three files: +the frontmatter in these four command files: ```bash sed -i.bak 's/^model: opus$/model: sonnet/' \ @@ -854,7 +854,9 @@ sed -i.bak 's/^model: opus$/model: sonnet/' \ commands/trekexecute.md ``` -The exploration agents stay on Sonnet — only the orchestrator is bumped. +This flips only the four command orchestrators. The exploration and review +sub-agents in `agents/*.md` are separately `opus`-pinned — flip those too, or +just run `--profile economy`, to put the whole pipeline on Sonnet. ### Disable external research diff --git a/docs/operations.md b/docs/operations.md index 0530d8d..fb0c771 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -32,14 +32,14 @@ Three built-in model profiles plus operator-defined `.yaml`. Each profil |---------|-------|----------|------|---------|--------|----------|----------| | `economy` | sonnet | sonnet | sonnet | sonnet | sonnet | sonnet | Lowest cost; high-confidence small-scope tasks (operator-opt-in via `--profile economy`) | | `balanced` | sonnet | sonnet | opus | sonnet | opus | sonnet | Mixed — opus where reasoning depth pays off (operator-opt-in via `--profile balanced`) | -| `premium` (default) | opus | opus | opus | opus | opus | opus | Maximum quality — Opus on every phase. Default since 2026-05-13 operator request; also the hardcoded resolver default at `lib/profiles/resolver.mjs:145` | +| `premium` (default) | opus | opus | opus | opus | opus | opus | Maximum quality — Opus on every phase. Default since 2026-05-13 operator request; also the hardcoded resolver default returned by `resolveProfile()` in `lib/profiles/resolver.mjs` | ### Lookup order 1. Explicit `--profile ` flag passed to the command 2. Plan-file frontmatter `profile:` (when resuming via `/trekexecute --resume` or `/trekcontinue`) 3. `VOYAGE_PROFILE` environment variable -4. Default `balanced` +4. Default `premium` ### Custom profiles diff --git a/docs/profiles.md b/docs/profiles.md index 8c6817d..0b69ef6 100644 --- a/docs/profiles.md +++ b/docs/profiles.md @@ -12,13 +12,17 @@ Three pre-defined tiers ship with v4.1, located at | Profile | Brief | Research | Plan | Execute | Review | Continue | Use case | |---------|-------|----------|------|---------|--------|----------|----------| | `economy` | sonnet | sonnet | sonnet | sonnet | sonnet | sonnet | Lowest cost; small-scope tasks where you have high confidence the brief is right | -| `balanced` (default) | sonnet | sonnet | opus | sonnet | opus | sonnet | Default — opus where reasoning depth pays off (plan synthesis + adversarial review) | -| `premium` | opus | sonnet | opus | sonnet | opus | sonnet | Critical-path planning + review when budget allows | +| `balanced` | sonnet | sonnet | opus | sonnet | opus | sonnet | Mixed — opus where reasoning depth pays off (plan synthesis + adversarial review); opt-in via `--profile balanced` | +| `premium` (default) | opus | opus | opus | opus | opus | opus | Maximum quality — Opus on every phase + external research on (default since the 2026-05-13 operator decision) | -`balanced` is the v4.1 default. It puts opus on the two phases where -quality matters most (Plan synthesis + Review) and sonnet everywhere -else. This lands the cost/quality trade-off that solo-developers and -small teams actually want. +`premium` is the default tier — set by the 2026-05-13 operator decision and +matched by the hardcoded resolver default in `lib/profiles/resolver.mjs`. It +runs opus on every phase and turns external research on: maximum quality, at +roughly 5× the sub-agent cost of an all-sonnet run, accepted as a deliberate +trade-off. Drop to `--profile balanced` (opus only on the two phases where +quality matters most — Plan synthesis + Review — and sonnet everywhere else) +or `--profile economy` (sonnet everywhere) when cost or latency matters more +than depth. `economy` is *strictly experimental* in v4.1. The cross-tier Jaccard floor (0.55) is grounded in parked-synthetic fixtures, not empirical @@ -92,7 +96,7 @@ Voyage resolves the profile in this priority order: 2. **Plan-file frontmatter `profile:`** — when resuming via `/trekexecute --resume` or `/trekcontinue` 3. **`VOYAGE_PROFILE` environment variable** — useful for headless CI -4. **Default `balanced`** — final fallback +4. **Default `premium`** — final fallback The resolved value is recorded in two places: @@ -116,10 +120,10 @@ The validator (`lib/validators/profile-validator.mjs`) enforces: one of the canonical short names - All six phases must be present (no partial profiles) -Custom profiles override built-ins of the same name (lookup is -alphabetical with `` taking precedence). You may NOT redefine -`balanced` (the default tier is locked to prevent accidental override -of headless CI behaviour); use a different name and reference it via +The three built-in names (`economy`, `balanced`, `premium`) resolve to their +bundled yaml first — `findProfilePath()` returns the built-in before consulting +`voyage-profiles/`, so a same-named custom file is ignored and cannot shadow a +built-in. To customize, give your profile a new name and reference it via `--profile ` or `VOYAGE_PROFILE=`. ### Example custom profile diff --git a/tests/lib/doc-consistency.test.mjs b/tests/lib/doc-consistency.test.mjs index b4f05b0..65465dc 100644 --- a/tests/lib/doc-consistency.test.mjs +++ b/tests/lib/doc-consistency.test.mjs @@ -11,6 +11,7 @@ import { readFileSync, readdirSync } from 'node:fs'; import { join, dirname } from 'node:path'; import { fileURLToPath } from 'node:url'; import { parseDocument } from '../../lib/util/frontmatter.mjs'; +import { resolveProfile, loadProfile } from '../../lib/profiles/resolver.mjs'; const HERE = dirname(fileURLToPath(import.meta.url)); const ROOT = join(HERE, '..', '..'); @@ -709,3 +710,79 @@ test('v5.5 — voyage README.md mentions framing enforcement / brief_version 2.2 assert.ok(t.includes('framing') && t.includes('brief_version 2.2'), 'voyage README.md must carry a "What\'s new" note for framing enforcement (brief_version 2.2)'); }); + +// --- S15 — default-profile name + per-phase model claims (cost-claim truth-pass) --- +// The resolver's hardcoded default (lib/profiles/resolver.mjs) is the single +// source of truth for which built-in profile a fresh user gets. README, +// docs/profiles.md, and docs/operations.md must all mark THAT profile as the +// default and must not mark any other built-in as default. Guards the +// code-vs-docs contradiction the S14 audit surfaced (Addendum finding #3). +const PROFILE_DOCS = ['README.md', 'docs/profiles.md', 'docs/operations.md']; + +test('S15: default-profile name is consistent across resolver + all profile docs', () => { + // Empty env so the operator's VOYAGE_PROFILE does not mask the hardcoded default. + const { profile: def, profile_source } = resolveProfile({}, {}); + assert.equal(profile_source, 'default', 'resolveProfile({}, {}) must report source=default'); + assert.equal(def, 'premium', 'resolver hardcoded default is premium (operator decision 2026-05-13, commit 40d8742)'); + + const OTHERS = ['economy', 'balanced', 'premium'].filter((p) => p !== def); + for (const doc of PROFILE_DOCS) { + const body = read(doc); + assert.ok( + body.includes(`\`${def}\` (default)`), + `${doc}: profile table must mark \`${def}\` as "(default)" to match resolver`, + ); + for (const other of OTHERS) { + assert.ok( + !body.includes(`\`${other}\` (default)`), + `${doc}: must NOT mark \`${other}\` as "(default)" — resolver default is ${def}`, + ); + assert.ok( + !body.includes(`Default \`${other}\``), + `${doc}: lookup-order must not say "Default \`${other}\`" — resolver default is ${def}`, + ); + } + } +}); + +test('S15: profile tables encode each built-in yaml phase_models exactly', () => { + // Column order in every profile table: Profile | Brief | Research | Plan | Execute | Review | Continue | Use case + const PHASES = ['brief', 'research', 'plan', 'execute', 'review', 'continue']; + for (const name of ['economy', 'balanced', 'premium']) { + const pm = loadProfile(name).phase_models; // {brief:'opus', ...} + const expected = PHASES.map((ph) => pm[ph]); + for (const doc of PROFILE_DOCS) { + const row = read(doc) + .split('\n') + .find((l) => new RegExp(`^\\|\\s*\`${name}\``).test(l)); + assert.ok(row, `${doc}: profile table is missing a \`${name}\` row`); + const cells = row.split('|').map((c) => c.trim()); + // cells[0] is '' (leading pipe), cells[1] is the name, cells[2..7] are the 6 models + const docModels = cells.slice(2, 8); + assert.deepEqual( + docModels, + expected, + `${doc}: \`${name}\` row models ${JSON.stringify(docModels)} must equal lib/profiles/${name}.yaml ${JSON.stringify(expected)}`, + ); + } + } +}); + +test('S15: README cost prose does not resurrect the false Sonnet-swarm claim', () => { + // All 24 sub-agents are model: opus (operator-pinned, commit 40d8742) and the + // model is uniform per phase — there is no "Opus orchestrates / Sonnet runs + // the swarms" split. These phrases asserted the false split and must stay out. + const md = read('README.md'); + const BANNED = [ + 'Sonnet exploration', + 'Sonnet runs the exploration', + 'front-loads cheap Sonnet', + 'exploration agents stay on Sonnet', + ]; + for (const phrase of BANNED) { + assert.ok( + !md.includes(phrase), + `README.md must not claim "${phrase}" — sub-agents are opus-pinned; default profile runs Opus on every phase`, + ); + } +});