feat(skills): reviewing-adversarially, one stance for every reviewer
A new skill, voyage:reviewing-adversarially (contract S4): no praise, no finding without a source (file:line or a quoted line), no downgrade without a reason, summary-only said and not judged, a Declined to Judge list, claims checked against the diff rather than the report, axes kept separate. Red baseline first: six review runs by helpers without the skill (two invented fixtures) had no separate Declined to Judge list in 6 of 6, an unsourced finding in 6 of 6, praise in 2 of 6 and a verdict on something unmeasured in 4 of 6. With the skill: 6 of 6, 0, 0 and 0. - brief-conformance-reviewer, code-correctness-reviewer, review-coordinator and scope-guardian preload it through skills:; the 21-line "Interaction Awareness - MANDATORY OVERRIDE" block copied into three of them is gone. Each agent keeps its own role. - The two reviewers and scope-guardian end with their own Declined to Judge list; /trekreview names the skill at spawn and passes the reviewers' lines to the coordinator, which copies them into review.md as prose (the JSON declined_to_judge array and the verdict are unchanged). - Eval cases 3 / 2 / 1; pins in tests/lib/skills-review-prose-pins.test.mjs. - tests/fixtures/jsonl-schemas.md: the trekreview-stats line_ref moves with Phase 8 (636-686). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
31c77fd8d6
commit
f906ec9252
26 changed files with 601 additions and 67 deletions
|
|
@ -0,0 +1,8 @@
|
|||
---
|
||||
type: tool_used
|
||||
tool: Skill
|
||||
input_match: '"skill"\s*:\s*"(?:voyage:)?reviewing-adversarially"'
|
||||
min: 0
|
||||
max: 0
|
||||
arm: both
|
||||
---
|
||||
10
evals/skill-reviewing-adversarially-quiet-2/prompt.md
Normal file
10
evals/skill-reviewing-adversarially-quiet-2/prompt.md
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
---
|
||||
description: "Routing case: the reviewing-adversarially skill must stay quiet on a request to tidy the wording of a text; that is editing, not a verdict. /trekresearch produces research briefs and is named as the owner."
|
||||
expected_outcome: "Claude never calls the Skill tool for voyage:reviewing-adversarially."
|
||||
tags: [skill-no-trigger, skill=reviewing-adversarially, owner=trekresearch, from=order-7d-negatives]
|
||||
runs: 3
|
||||
max_turns: 4
|
||||
allowed_tools: [Read, Glob, Grep, Skill]
|
||||
---
|
||||
|
||||
Smooth the wording of my research brief's summary and fix the typos. Do not change what it says.
|
||||
Loading…
Add table
Add a link
Reference in a new issue