Kjell Tore Guttormsen
d311f3b49a
Seven `claude plugin eval` cases under evals/, runs: 1, deterministic graders only (regex, tool_used, file_exists), no ablation. Each case tests one thing Voyage promises and stops within minutes, headless: - plan-requires-brief, plan-project-not-initialized, plan-rejects-unknown-export, review-requires-project: argument guards stop before any Agent or Write. - plan-halts-without-phase-signals: a brief_version 2.1 brief without phase_signals halts /trekplan at the sequencing gate, before the swarm. - review-validate-flags-bad-finding-id (known-positive) and review-validate-passes-clean-review (known-negative): /trekreview --validate names REVIEW_BAD_FINDING_ID on a planted bad ID and stays clean on a valid file. Each case's expected_outcome is committed here, before the suite has run. Fixtures are written by an inline scaffold.sh (needs --scaffold): chose inline heredocs because a run cannot read the eval directory, and the add_dirs path mapping is not documented. evals/results/ is ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
439 B
439 B
| description | expected_outcome | tags | runs | max_turns | timeout_seconds | allowed_tools | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| /trekplan on a brief_version 2.1 brief without phase_signals halts at the sequencing gate. | brief-validator runs, reports BRIEF_V51_MISSING_SIGNALS, and /trekplan halts with the phase_signals hint; no Agent call (no exploration swarm), no plan.md. |
|
1 | 15 | 300 |
|
/voyage:trekplan --project proj