chore(voyage): S34 — V30 economy-profile self-declares experimental (uncalibrated Jaccard floor)
The economy profile's cross-tier Jaccard floor (0.55) rests on parked-synthetic fixtures; empirical Step-17 calibration is v4.2-budget-gated ($60–120, unauthorized). Fork resolved as label-not-calibrate: the experimental status, previously prose-only in docs/profiles.md, now lives in the profile data and is machine-checked. No new user-facing capability — honest labeling + a guard. - lib/profiles/economy.yaml: add `experimental: true` (with rationale comment). - lib/validators/profile-validator.mjs: recognize `experimental` as an OPTIONAL boolean; non-boolean → PROFILE_INVALID_ENUM. Absent ⇒ tier is stable (premium/balanced unaffected, profile_version stays 1.0 — additive). - README.md + docs/operations.md + docs/profiles.md: flag the `economy` table row "⚠ Experimental (uncalibrated Jaccard floor)". - tests/synthetic/profile-jaccard-calibration.md + analysis §6/§10 + backlog plan §S34: cross-reference the marker; mark V30/S34 done. +5 tests (739 → 744, 742/2/0): economy declares experimental:true; premium and balanced do not; validator rejects non-boolean experimental; every profile-doc economy row is flagged; the flag tracks the calibration's parked-synthetic status (must drop in the same change that lands real calibration). Closes the balance backlog (4/4, S31–S34). claude plugin validate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
This commit is contained in:
parent
2df0cbb372
commit
213cf388de
10 changed files with 146 additions and 13 deletions
|
|
@ -961,6 +961,46 @@ test('S15: profile tables encode each built-in yaml phase_models exactly', () =>
|
|||
}
|
||||
});
|
||||
|
||||
// --- S34 (V30) — economy is self-declared experimental until the cross-tier
|
||||
// Jaccard floor (0.55) is empirically calibrated (Step-17 calibration deferred
|
||||
// to v4.2). The status must be visible in BOTH the profile data
|
||||
// (lib/profiles/economy.yaml `experimental: true`) and every profile doc's
|
||||
// economy row. Fix the SOURCE doc/profile, not these pins.
|
||||
test('S34: economy profile row is flagged experimental across all profile docs', () => {
|
||||
for (const doc of PROFILE_DOCS) {
|
||||
const body = read(doc);
|
||||
const row = body.split('\n').find((l) => /^\|\s*`economy`/.test(l));
|
||||
assert.ok(row, `${doc}: profile table missing an \`economy\` row`);
|
||||
assert.match(
|
||||
row,
|
||||
/experimental/i,
|
||||
`${doc}: the \`economy\` table row must flag it experimental ` +
|
||||
`(uncalibrated Jaccard floor, S34/V30) — matches lib/profiles/economy.yaml experimental: true`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('S34: economy experimental flag agrees with parked-synthetic calibration status', () => {
|
||||
// economy.yaml experimental:true is honest ONLY while calibration is still
|
||||
// parked-synthetic. When empirical calibration lands (status: empirical),
|
||||
// drop experimental:true in the SAME change — this pin enforces the linkage.
|
||||
const eco = parseDocument(read('lib/profiles/economy.yaml')).parsed.frontmatter;
|
||||
const cal = parseDocument(read('tests/synthetic/profile-jaccard-calibration.md')).parsed.frontmatter;
|
||||
if (cal.status === 'parked-synthetic') {
|
||||
assert.equal(
|
||||
eco.experimental,
|
||||
true,
|
||||
'while calibration is parked-synthetic, economy.yaml must stay experimental: true',
|
||||
);
|
||||
} else {
|
||||
assert.notEqual(
|
||||
eco.experimental,
|
||||
true,
|
||||
'calibration is no longer parked-synthetic — drop economy experimental: true in the same change',
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test('S15: README cost prose does not resurrect the false Sonnet-swarm claim', () => {
|
||||
// All 24 sub-agents are model: opus (operator-pinned, commit 40d8742) and the
|
||||
// model is uniform per phase — there is no "Opus orchestrates / Sonnet runs
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue