docs(voyage): S15 — cost-claim truth-pass + premium default-profile pins

Resolve the S14 default-profile contradiction. Investigation overturned
the audit addendum's guess ("code is the bug → balanced"): commit 40d8742
(2026-05-13, "pin all sub-agents to Opus permanently (operator request)")
plus VOYAGE_PROFILE=premium in ~/.zshenv establish premium as the deliberate
default. Operator confirmed in-session: premium is the shipped default; fix
the stale docs, not the code. No code or behaviour changed (lib/ untouched).

Docs (default-name → premium, consistent across resolver + all three docs):
- README + docs/profiles.md + docs/operations.md: 3 lookup-order sites and
  3 profile tables now mark `premium` as the default.
- premium table row corrected to all-opus (matches premium.yaml — a third
  inconsistency the audit missed; README/profiles.md showed opus/sonnet/...).

Cost narrative made honest (premium = all-Opus reality):
- §Cost profile rewritten: uniform model per phase, no orchestrator-vs-swarm
  split; cheaper via --profile balanced/economy.
- Removed false "Sonnet exploration/review swarm" claims (README 195/223/266
  model-neutral; 804 parenthetical; the "Switch the planning model" note).
- profiles.md custom-profile prose corrected: built-in wins over same-named
  custom (findProfilePath), dropping the bogus "balanced is the locked default".

Pins (TDD red→green, doc-consistency.test.mjs):
- default-profile name invariant (resolveProfile ↔ README/profiles/operations)
- profile tables ↔ each built-in yaml phase_models (structural, catches drift)
- cost-claim regression guard (no resurrected Sonnet-swarm phrasing)

S16/S18 surface untouched: counts (23 agents, 9/10 dims, 5/6 dims), versions,
framing gates unchanged. Full suite 686 (684 pass / 2 skip / 0 fail).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
This commit is contained in:
Kjell Tore Guttormsen 2026-06-18 21:06:36 +02:00
commit 7a5749ddcc
4 changed files with 108 additions and 25 deletions

View file

@ -11,6 +11,7 @@ import { readFileSync, readdirSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
import { parseDocument } from '../../lib/util/frontmatter.mjs';
import { resolveProfile, loadProfile } from '../../lib/profiles/resolver.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
@ -709,3 +710,79 @@ test('v5.5 — voyage README.md mentions framing enforcement / brief_version 2.2
assert.ok(t.includes('framing') && t.includes('brief_version 2.2'),
'voyage README.md must carry a "What\'s new" note for framing enforcement (brief_version 2.2)');
});
// --- S15 — default-profile name + per-phase model claims (cost-claim truth-pass) ---
// The resolver's hardcoded default (lib/profiles/resolver.mjs) is the single
// source of truth for which built-in profile a fresh user gets. README,
// docs/profiles.md, and docs/operations.md must all mark THAT profile as the
// default and must not mark any other built-in as default. Guards the
// code-vs-docs contradiction the S14 audit surfaced (Addendum finding #3).
const PROFILE_DOCS = ['README.md', 'docs/profiles.md', 'docs/operations.md'];
test('S15: default-profile name is consistent across resolver + all profile docs', () => {
// Empty env so the operator's VOYAGE_PROFILE does not mask the hardcoded default.
const { profile: def, profile_source } = resolveProfile({}, {});
assert.equal(profile_source, 'default', 'resolveProfile({}, {}) must report source=default');
assert.equal(def, 'premium', 'resolver hardcoded default is premium (operator decision 2026-05-13, commit 40d8742)');
const OTHERS = ['economy', 'balanced', 'premium'].filter((p) => p !== def);
for (const doc of PROFILE_DOCS) {
const body = read(doc);
assert.ok(
body.includes(`\`${def}\` (default)`),
`${doc}: profile table must mark \`${def}\` as "(default)" to match resolver`,
);
for (const other of OTHERS) {
assert.ok(
!body.includes(`\`${other}\` (default)`),
`${doc}: must NOT mark \`${other}\` as "(default)" — resolver default is ${def}`,
);
assert.ok(
!body.includes(`Default \`${other}\``),
`${doc}: lookup-order must not say "Default \`${other}\`" — resolver default is ${def}`,
);
}
}
});
test('S15: profile tables encode each built-in yaml phase_models exactly', () => {
// Column order in every profile table: Profile | Brief | Research | Plan | Execute | Review | Continue | Use case
const PHASES = ['brief', 'research', 'plan', 'execute', 'review', 'continue'];
for (const name of ['economy', 'balanced', 'premium']) {
const pm = loadProfile(name).phase_models; // {brief:'opus', ...}
const expected = PHASES.map((ph) => pm[ph]);
for (const doc of PROFILE_DOCS) {
const row = read(doc)
.split('\n')
.find((l) => new RegExp(`^\\|\\s*\`${name}\``).test(l));
assert.ok(row, `${doc}: profile table is missing a \`${name}\` row`);
const cells = row.split('|').map((c) => c.trim());
// cells[0] is '' (leading pipe), cells[1] is the name, cells[2..7] are the 6 models
const docModels = cells.slice(2, 8);
assert.deepEqual(
docModels,
expected,
`${doc}: \`${name}\` row models ${JSON.stringify(docModels)} must equal lib/profiles/${name}.yaml ${JSON.stringify(expected)}`,
);
}
}
});
test('S15: README cost prose does not resurrect the false Sonnet-swarm claim', () => {
// All 24 sub-agents are model: opus (operator-pinned, commit 40d8742) and the
// model is uniform per phase — there is no "Opus orchestrates / Sonnet runs
// the swarms" split. These phrases asserted the false split and must stay out.
const md = read('README.md');
const BANNED = [
'Sonnet exploration',
'Sonnet runs the exploration',
'front-loads cheap Sonnet',
'exploration agents stay on Sonnet',
];
for (const phrase of BANNED) {
assert.ok(
!md.includes(phrase),
`README.md must not claim "${phrase}" — sub-agents are opus-pinned; default profile runs Opus on every phase`,
);
}
});