voyage/tests/fixtures
Kjell Tore Guttormsen c56f852ab6
fix(validate): a check that can never run is caught when the brief or plan is written
voyage-lekerepo end-to-end trial (2026-10-06), report points 2, 3, 4, 9:
11 of 13 brief criteria were NOT RUN at review with no warning in /trekbrief;
a plan `git grep` criterion passed validator, gate and three critic rounds,
then felled the run in Phase 7; the End-to-end field and Phase 7's text
search never met; a prose Verify left the test command undefined.

- criteria-runner: runnableCheck() screens a declared check without running
  it (same allowlist; an npm run script is not looked up at write time);
  /bin/bash tests/<script> allowed; the plan report carries end_to_end,
  verified when a passed criterion runs a step's End-to-end command (the
  older text form still counts).
- plan-validator 1.8: PLAN_VERIFICATION_NOT_RUNNABLE,
  PLAN_STEP_VERIFY_NOT_RUNNABLE, PLAN_END_TO_END_NOT_RUNNABLE,
  PLAN_END_TO_END_NOT_VERIFIED (the gate inherits them).
- brief-validator: BRIEF_SC_NOT_RUNNABLE warning; /trekbrief offers a
  tests/ script; brief-reviewer no longer proposes unrunnable commands.
- trekexecute: Phase 7 copies end_to_end from the runner; Verify's test
  command is its first backticked span.

Measured on the trial's own brief and plan: 11 warnings (the 11 NOT RUN),
and the plan stops on V5 git grep and the unlisted End-to-end command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 06:42:33 +02:00
..
bakeoff feat(voyage): S10 — NW2 part A (Workflow port + fidelity harness + smoke) 2026-06-18 14:40:19 +02:00
bakeoff-rich feat(eval): SKAL-1·4b offline gold-scored output eval 2026-06-30 09:00:33 +02:00
red-first feat(measure): the yardstick prints Voyage's three numbers with denominators from the source, and it is RED 2026-09-22 21:50:06 +02:00
session-orders feat(execute)!: hand waves over as session orders, never start Claude Code (green) 2026-09-27 15:35:22 +02:00
session-state test(ultraplan-local): add session-state fixtures 2026-05-01 20:21:50 +02:00
synthesis feat(voyage): S12 — NW3 synthesis-agent built + measured → declined per measurement [skip-docs] 2026-06-18 17:58:39 +02:00
test-integrity feat(execute): one fresh helper per step, a controller on the diff, proof by command 2026-10-06 19:58:18 +02:00
token-usage feat(stats): add pure token-usage parser + cache-aware cost derivation 2026-06-26 14:32:10 +02:00
trekreview feat(review): the review is a gate - verdict in the file, CC engine, project memory 2026-10-06 19:45:52 +02:00
brief-effort-fable.md test(validators): cover fable accept + unknown-model reject in both gate layers 2026-07-02 16:56:26 +02:00
brief-effort-high.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-effort-low.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-effort-standard.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-success-criteria.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
brief-v21-no-signals.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-with-phase-signals.md feat(voyage): bump trekbrief-template to brief_version 2.1 + add phase_signals fixtures 2026-05-13 21:09:57 +02:00
brief-without-phase-signals.md feat(voyage): bump trekbrief-template to brief_version 2.1 + add phase_signals fixtures 2026-05-13 21:09:57 +02:00
criteria-exit-0.sh fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
criteria-exit-1.sh fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
end-state-frozen.json fix(end-state): a behaviour probe must FELL a stub, not just name a test 2026-09-18 02:56:22 +02:00
expected.prom feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
jsonl-schemas.md fix(validate): a check that can never run is caught when the brief or plan is written 2026-10-08 06:42:33 +02:00
plan-fase-narrative.md chore(release): ultraplan-local v3.1.0 2026-05-01 06:31:42 +02:00
plan-profile-drift.md feat(voyage): emit MANIFEST_PROFILE_DRIFT warning in plan-validator strict mode — brief assumption 7 2026-05-09 10:02:53 +02:00
plan-verification-fails.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
plan-verification-passes.md fix(verification): the criteria runner screens with an ALLOWLIST, not a denylist 2026-09-18 02:51:40 +02:00
plan-with-profile.md feat(voyage): add lib/profiles/resolver.mjs — locked interface SC #5-#9 2026-05-09 09:29:01 +02:00
plan-without-profile.md feat(voyage): add lib/profiles/resolver.mjs — locked interface SC #5-#9 2026-05-09 09:29:01 +02:00
profile-invalid-enum.yaml feat(voyage): add lib/validators/profile-validator.mjs — SC #1, #2, #3 2026-05-09 09:26:23 +02:00
profile-invalid-model.yaml feat(voyage): add lib/validators/profile-validator.mjs — SC #1, #2, #3 2026-05-09 09:26:23 +02:00
research-deep-research-adapted.md test(trekresearch): pin deep-research adapter output contract 2026-06-30 13:38:09 +02:00
stats-sample.jsonl feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
stats-with-profile.jsonl feat(voyage): add 5 additive profile fields to JSONL stats — SC #11 2026-05-09 09:40:21 +02:00