chore(voyage): S34 — V30 economy-profile self-declares experimental (uncalibrated Jaccard floor)
The economy profile's cross-tier Jaccard floor (0.55) rests on parked-synthetic fixtures; empirical Step-17 calibration is v4.2-budget-gated ($60–120, unauthorized). Fork resolved as label-not-calibrate: the experimental status, previously prose-only in docs/profiles.md, now lives in the profile data and is machine-checked. No new user-facing capability — honest labeling + a guard. - lib/profiles/economy.yaml: add `experimental: true` (with rationale comment). - lib/validators/profile-validator.mjs: recognize `experimental` as an OPTIONAL boolean; non-boolean → PROFILE_INVALID_ENUM. Absent ⇒ tier is stable (premium/balanced unaffected, profile_version stays 1.0 — additive). - README.md + docs/operations.md + docs/profiles.md: flag the `economy` table row "⚠ Experimental (uncalibrated Jaccard floor)". - tests/synthetic/profile-jaccard-calibration.md + analysis §6/§10 + backlog plan §S34: cross-reference the marker; mark V30/S34 done. +5 tests (739 → 744, 742/2/0): economy declares experimental:true; premium and balanced do not; validator rejects non-boolean experimental; every profile-doc economy row is flagged; the flag tracks the calibration's parked-synthetic status (must drop in the same change that lands real calibration). Closes the balance backlog (4/4, S31–S34). claude plugin validate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
This commit is contained in:
parent
2df0cbb372
commit
213cf388de
10 changed files with 146 additions and 13 deletions
|
|
@ -148,3 +148,57 @@ test('PROFILE_REQUIRED_PHASES export drift-pin', () => {
|
|||
'PROFILE_REQUIRED_PHASES contract drift — pin contract',
|
||||
);
|
||||
});
|
||||
|
||||
// S34 (V30) — economy carries an explicit `experimental` marker.
|
||||
// The cross-tier Jaccard floor (0.55) rests on parked-synthetic fixtures;
|
||||
// empirical Step-17 calibration is deferred to v4.2 ($60-120 LLM budget,
|
||||
// unauthorized). Until a real calibration lands, `economy` is self-declared
|
||||
// experimental in the profile DATA — not only in prose — so the status
|
||||
// survives a docs rewrite. See tests/synthetic/profile-jaccard-calibration.md.
|
||||
|
||||
test('S34: economy.yaml declares experimental: true (uncalibrated Jaccard floor)', () => {
|
||||
const r = validateProfile(join(REPO_ROOT, 'lib', 'profiles', 'economy.yaml'));
|
||||
assert.equal(r.valid, true, `economy.yaml should validate: ${JSON.stringify(r.errors)}`);
|
||||
assert.equal(r.parsed.frontmatter.experimental, true,
|
||||
'economy.yaml must carry experimental: true until the Jaccard floor is empirically calibrated (S34/V30)');
|
||||
});
|
||||
|
||||
test('S34: premium + balanced are NOT flagged experimental', () => {
|
||||
for (const name of ['premium', 'balanced']) {
|
||||
const r = validateProfile(join(REPO_ROOT, 'lib', 'profiles', `${name}.yaml`));
|
||||
assert.equal(r.valid, true, `${name}.yaml should validate: ${JSON.stringify(r.errors)}`);
|
||||
assert.notEqual(r.parsed.frontmatter.experimental, true,
|
||||
`${name}.yaml must not be flagged experimental`);
|
||||
}
|
||||
});
|
||||
|
||||
test('S34: validator type-checks experimental as boolean (PROFILE_INVALID_ENUM)', () => {
|
||||
const bad = `---
|
||||
profile_version: "1.0"
|
||||
name: bad-experimental
|
||||
phase_models:
|
||||
- phase: brief
|
||||
model: sonnet
|
||||
- phase: research
|
||||
model: sonnet
|
||||
- phase: plan
|
||||
model: sonnet
|
||||
- phase: execute
|
||||
model: sonnet
|
||||
- phase: review
|
||||
model: sonnet
|
||||
- phase: continue
|
||||
model: sonnet
|
||||
parallel_agents_min: 2
|
||||
parallel_agents_max: 3
|
||||
external_research_enabled: false
|
||||
brief_reviewer_iter_cap: 1
|
||||
experimental: "yes"
|
||||
---
|
||||
`;
|
||||
const r = validateProfileContent(bad);
|
||||
assert.equal(r.valid, false, 'experimental: "yes" (string) must be rejected');
|
||||
const found = r.errors.find(e => e.code === 'PROFILE_INVALID_ENUM' && /experimental/.test(e.message));
|
||||
assert.ok(found, `expected PROFILE_INVALID_ENUM for experimental, got: ${JSON.stringify(r.errors)}`);
|
||||
assert.match(found.message, /boolean/);
|
||||
});
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue