voyage/tests/fixtures
Kjell Tore Guttormsen abc3083c1e
feat(execute): one fresh helper per step, a controller on the diff, proof by command
/trekexecute no longer builds steps itself. Each plan step is built by one
fresh helper (new agent step-implementer: opus, effort high, no Agent tool)
and judged by one controller (a general helper on opus) that reads the
diff from a recorded start_sha and gives two verdicts, spec and quality.
Fix rounds go up, maximum 5: three fixes by a fresh helper, then back to
the plan, then to the operator. No cheaper-model ladder.

Proof is a command, never the model's word:
- lib/verification/red-proof.mjs: the step's test must fail at the start
  point and pass at the result (exit 0/1/2; 2 is never green). Reproduces
  the 02243c6 red-first measurement (6 of 55) through the shim fixture.
- lib/verification/test-integrity.mjs: the floor guard on the diff
  (removed asserts, skipped/deleted tests, muted checks, stubs, loosened
  thresholds). Over the last 150 commits: 149 clean, 1 real finding.
- lib/verification/done-line.mjs: DONE step N only when the test command
  exits 0.

Test first is required from plan_version 1.8. The test lock (K8) is used
through its CLI and reported as not-locked while it is missing. A run
stops only on a closed list of four classes; every other call goes into a
decision log with its cost if wrong. Without a passing run-the-thing
criterion the result is not verified end to end, never green.

Handover 5 gains optional per-step start_sha, report_path, red_proof,
test_lock, fix_rounds (0-5) and run-level decisions and end_to_end; old
progress files still validate and resume.

No isolation on the helper: measured, a worktree-isolated subagent
branches from the default branch (02f320f), not the executor's HEAD
(2c89a6b), so it would miss the earlier steps.

Borrowings credited in commands/trekexecute.md ## Credits (Superpowers,
ai-native-sdlc, Kiro, GitHub Spec Kit, agent-skills, Cherny).

Tests: 1411 -> 1457 (1455 pass, 0 fail, 2 skipped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 19:58:18 +02:00
..
bakeoff feat(voyage): S10 — NW2 part A (Workflow port + fidelity harness + smoke) 2026-06-18 14:40:19 +02:00
bakeoff-rich feat(eval): SKAL-1·4b offline gold-scored output eval 2026-06-30 09:00:33 +02:00
red-first feat(measure): the yardstick prints Voyage's three numbers with denominators from the source, and it is RED 2026-09-22 21:50:06 +02:00
session-orders feat(execute)!: hand waves over as session orders, never start Claude Code (green) 2026-09-27 15:35:22 +02:00
session-state test(ultraplan-local): add session-state fixtures 2026-05-01 20:21:50 +02:00
synthesis feat(voyage): S12 — NW3 synthesis-agent built + measured → declined per measurement [skip-docs] 2026-06-18 17:58:39 +02:00
test-integrity feat(execute): one fresh helper per step, a controller on the diff, proof by command 2026-10-06 19:58:18 +02:00
token-usage feat(stats): add pure token-usage parser + cache-aware cost derivation 2026-06-26 14:32:10 +02:00
trekreview feat(voyage)!: bulk content rewrite ultra -> voyage/trek prose [skip-docs] 2026-05-05 15:08:20 +02:00
brief-effort-fable.md test(validators): cover fable accept + unknown-model reject in both gate layers 2026-07-02 16:56:26 +02:00
brief-effort-high.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-effort-low.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-effort-standard.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-success-criteria.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
brief-v21-no-signals.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-with-phase-signals.md feat(voyage): bump trekbrief-template to brief_version 2.1 + add phase_signals fixtures 2026-05-13 21:09:57 +02:00
brief-without-phase-signals.md feat(voyage): bump trekbrief-template to brief_version 2.1 + add phase_signals fixtures 2026-05-13 21:09:57 +02:00
criteria-exit-0.sh fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
criteria-exit-1.sh fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
end-state-frozen.json fix(end-state): a behaviour probe must FELL a stub, not just name a test 2026-09-18 02:56:22 +02:00
expected.prom feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
jsonl-schemas.md feat(execute): one fresh helper per step, a controller on the diff, proof by command 2026-10-06 19:58:18 +02:00
plan-fase-narrative.md chore(release): ultraplan-local v3.1.0 2026-05-01 06:31:42 +02:00
plan-profile-drift.md feat(voyage): emit MANIFEST_PROFILE_DRIFT warning in plan-validator strict mode — brief assumption 7 2026-05-09 10:02:53 +02:00
plan-verification-fails.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
plan-verification-passes.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
plan-with-profile.md feat(voyage): add lib/profiles/resolver.mjs — locked interface SC #5-#9 2026-05-09 09:29:01 +02:00
plan-without-profile.md feat(voyage): add lib/profiles/resolver.mjs — locked interface SC #5-#9 2026-05-09 09:29:01 +02:00
profile-invalid-enum.yaml feat(voyage): add lib/validators/profile-validator.mjs — SC #1, #2, #3 2026-05-09 09:26:23 +02:00
profile-invalid-model.yaml feat(voyage): add lib/validators/profile-validator.mjs — SC #1, #2, #3 2026-05-09 09:26:23 +02:00
research-deep-research-adapted.md test(trekresearch): pin deep-research adapter output contract 2026-06-30 13:38:09 +02:00
stats-sample.jsonl feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
stats-with-profile.jsonl feat(voyage): add 5 additive profile fields to JSONL stats — SC #11 2026-05-09 09:40:21 +02:00