merge: main (wave 4 order A) into order B before B goes into main
Conflicts on the union lines only: CHANGELOG Unreleased (both orders' entries kept), CLAUDE.md (/trekexecute row from A, /trekreview row from B), tests/fixtures/jsonl-schemas.md (trekplan line_ref from B, trekexecute line_ref from A). Suite on the merged tree 2682 (2680/0/2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
commit
45eaedc6b4
24 changed files with 274 additions and 45 deletions
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "/trekplan on a brief that passes the sequencing gate but carries no intent approval halts at the intent gate (veikart steg 1)."
|
||||
expected_outcome: "intent-approval.mjs --check runs and reports BRIEF_INTENT_NOT_APPROVED; /trekplan halts and names /trekbrief --approve; no Agent call (no exploration swarm), no plan.md."
|
||||
tags: [proevesett]
|
||||
tags: [proevesett, from=proevesett-intent-gate]
|
||||
runs: 1
|
||||
max_turns: 15
|
||||
timeout_seconds: 300
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "/trekplan on a brief_version 2.1 brief without phase_signals halts at the sequencing gate."
|
||||
expected_outcome: "brief-validator runs, reports BRIEF_V51_MISSING_SIGNALS, and /trekplan halts with the phase_signals hint; no Agent call (no exploration swarm), no plan.md."
|
||||
tags: [proevesett]
|
||||
tags: [proevesett, from=proevesett-eval-suite]
|
||||
runs: 1
|
||||
max_turns: 15
|
||||
timeout_seconds: 300
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "/trekplan --project on a missing folder stops before any exploration."
|
||||
expected_outcome: "'Error: project directory not initialized. Run /trekbrief to create it.'; no Agent call, no plan.md."
|
||||
tags: [proevesett]
|
||||
tags: [proevesett, from=proevesett-eval-suite]
|
||||
runs: 1
|
||||
max_turns: 8
|
||||
timeout_seconds: 300
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "/trekplan --export with a removed format names the format and stops."
|
||||
expected_outcome: "'Error: export format 'pr' is not supported.'; no Agent call, no file written."
|
||||
tags: [proevesett]
|
||||
tags: [proevesett, from=proevesett-eval-suite]
|
||||
runs: 1
|
||||
max_turns: 8
|
||||
timeout_seconds: 300
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "/trekplan with no arguments prints usage and stops."
|
||||
expected_outcome: "Usage block ending in 'A brief is required. Produce one with /trekbrief first.'; no Agent call, no file written."
|
||||
tags: [proevesett]
|
||||
tags: [proevesett, from=proevesett-eval-suite]
|
||||
runs: 1
|
||||
max_turns: 8
|
||||
timeout_seconds: 300
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "/trekreview with no arguments demands --project and stops."
|
||||
expected_outcome: "'Error: --project <dir> is required.' plus usage; no Agent call, no file written."
|
||||
tags: [proevesett]
|
||||
tags: [proevesett, from=proevesett-eval-suite]
|
||||
runs: 1
|
||||
max_turns: 8
|
||||
timeout_seconds: 300
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "KNOWN-POSITIVE. /trekreview --validate on a review.md whose finding ID is not 40-char hex fails and names the rule."
|
||||
expected_outcome: "review-validator runs and reports REVIEW_BAD_FINDING_ID; the one-line summary says FAIL; no Agent call, no file written."
|
||||
tags: [proevesett]
|
||||
tags: [proevesett, from=proevesett-eval-suite]
|
||||
runs: 1
|
||||
max_turns: 10
|
||||
timeout_seconds: 300
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "KNOWN-NEGATIVE. /trekreview --validate on a valid review.md passes and raises no error code."
|
||||
expected_outcome: "review-validator runs; the one-line summary says PASS; no REVIEW_* error code appears anywhere in the trace; no Agent call, no file written."
|
||||
tags: [proevesett]
|
||||
tags: [proevesett, from=proevesett-eval-suite]
|
||||
runs: 1
|
||||
max_turns: 10
|
||||
timeout_seconds: 300
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
---
|
||||
description: "Routing case: the writing-criteria skill must stay quiet here. Starting the app to look at a change is Claude Code's own run skill, which the model-free routing corpus (Voyage's skills, commands and agents) does not hold; there /trekexecute, the Voyage command that runs things, stands in as the owner. The model-run grader only checks that writing-criteria is not called."
|
||||
description: "Routing case: the writing-criteria skill must stay quiet here. Starting the app to look at a change is Claude Code's own run skill. It is outside the model-free routing corpus (Voyage's skills, commands and agents), so the routing test only checks that writing-criteria does not rank first. The model-run grader only checks that writing-criteria is not called."
|
||||
expected_outcome: "Claude never calls the Skill tool for voyage:writing-criteria."
|
||||
tags: [skill-no-trigger, skill=writing-criteria, owner=trekexecute, from=order-7c-negatives]
|
||||
tags: [skill-no-trigger, skill=writing-criteria, owner=run, from=order-7c-negatives]
|
||||
runs: 3
|
||||
max_turns: 4
|
||||
allowed_tools: [Read, Glob, Grep, Skill]
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue