feat(skills): skill workshop, test-first and debugging preloaded into the step helper
The workshop (ordre 7a, contracts S1 and S2): a form check (lib/skills/skill-lint.mjs) that follows Claude Code's own skill fields, and a model-free routing test (lib/skills/routing.mjs) over every skill, the 7 commands and the 21 spawnable agents, reading the eval cases themselves (evals/skill-*, 3 fire / 2 quiet / 1 effect, tagged with the event they came from). Thresholds pinned, never lowered. The two skills were written against failures measured first in helpers without them: red only as an import crash, the whole suite run from a hand-made file list instead of the repo's own command, and an unreproduced fix shipped under authority pressure. step-implementer preloads both through its skills: line; no tool list change. Credits: addyosmani/agent-skills @9d0c60d, obra/superpowers @8ca22db, mattpocock/skills @c55ee46 (MIT), Anthropic skill-creator (Apache-2.0). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
parent
4ee711a6ac
commit
cc832fd430
41 changed files with 1996 additions and 0 deletions
4
evals/skill-test-first-effect/case.yaml
Normal file
4
evals/skill-test-first-effect/case.yaml
Normal file
|
|
@ -0,0 +1,4 @@
|
|||
schema_version: "1.1"
|
||||
name: skill-test-first-effect
|
||||
context:
|
||||
scaffold_script: scaffold.sh
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
---
|
||||
type: llm
|
||||
focus: trace
|
||||
---
|
||||
|
||||
PASS only if, before src/rota.mjs has a working bookedCost, the transcript shows a test run in which at least one bookedCost assertion fails on a value (an AssertionError naming an expected and an actual value). A run that only fails to load the test file (for example "does not provide an export named 'bookedCost'", a SyntaxError, or a missing module) does not count as that failure. FAIL if the only red run before the implementation is such a load failure, or if there is no red run before the implementation at all.
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
---
|
||||
type: llm
|
||||
focus:
|
||||
source: file
|
||||
path: tests/rota.test.mjs
|
||||
---
|
||||
|
||||
PASS only if every expected value in the bookedCost tests is a literal number (for example 28.13), not computed in the test by calling bookedMinutes, parseDuration or bookedCost, and not by repeating the code's own formula (minutes / 60 * rate). FAIL if any bookedCost assertion derives its expected value from the code under test or from the same formula.
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
---
|
||||
type: tool_used
|
||||
tool: Bash
|
||||
input_match: 'npm(?:\s+--prefix\s+\S+)?\s+(?:run\s+)?test\b'
|
||||
---
|
||||
21
evals/skill-test-first-effect/prompt.md
Normal file
21
evals/skill-test-first-effect/prompt.md
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
---
|
||||
description: "Effect case for test-first (two arms: with and without the plugin). The scenario where Voyage's baseline helpers failed without the skill: red only as an import crash, and a hand-made file list instead of the repo's own test command. The operator grants Write, Edit and Bash for node/npm, and sets the judge model explicitly (never Haiku)."
|
||||
expected_outcome: "A test run shows a bookedCost assertion failing on a value before the code passes; the whole suite runs as npm test; expected values are independent literals."
|
||||
tags: [skill-effect, skill=test-first, from=baseline-tfd-crash-red]
|
||||
runs: 3
|
||||
max_turns: 40
|
||||
timeout_seconds: 900
|
||||
allowed_tools: [Read, Glob, Grep, Skill, Bash, Write, Edit]
|
||||
---
|
||||
|
||||
You are implementing step 5 of an approved plan in the small Node project in the current directory. It is not a git repository; do not try to commit.
|
||||
|
||||
### Step 5: Travel allowance
|
||||
- Files: src/rota.mjs, tests/rota.test.mjs
|
||||
- Changes: add and export bookedCost(shifts, ratePerHour): the travel allowance for a volunteer's shifts, which is the booked minutes divided by 60 times the hourly rate, rounded to whole cents (two decimals).
|
||||
- Test first: tests/rota.test.mjs — bookedCost for a list of shifts at a given rate.
|
||||
- Verify: node --test tests/rota.test.mjs
|
||||
|
||||
The treasurer needs the allowance numbers for tonight's payout run, so be quick about it.
|
||||
|
||||
Do the step now. When you are done, reply with what you changed and every command you ran, with its result.
|
||||
42
evals/skill-test-first-effect/scaffold.sh
Executable file
42
evals/skill-test-first-effect/scaffold.sh
Executable file
|
|
@ -0,0 +1,42 @@
|
|||
#!/bin/bash
|
||||
# Seeds the empty run workspace with a small volunteer-rota app (Node, node --test).
|
||||
# Self-contained on purpose: the run cannot read the eval directory, and the
|
||||
# fixture is written inline, with no path outside the workspace and no git.
|
||||
set -eu
|
||||
mkdir -p src tests
|
||||
cat > package.json <<'J'
|
||||
{ "name": "rota", "version": "0.3.0", "type": "module", "private": true,
|
||||
"scripts": { "test": "node --test" } }
|
||||
J
|
||||
cat > src/duration.mjs <<'J'
|
||||
// Parses a shift length such as "90m" or "45s" into minutes.
|
||||
export function parseDuration(text) {
|
||||
const m = /^(\d+)([a-z]*)$/.exec(String(text).trim());
|
||||
if (!m) throw new Error(`bad duration: ${text}`);
|
||||
const n = Number(m[1]);
|
||||
if (m[2] === 'm') return n;
|
||||
if (m[2] === 's') return n / 60;
|
||||
return n; // a bare number is minutes
|
||||
}
|
||||
J
|
||||
cat > src/rota.mjs <<'J'
|
||||
import { parseDuration } from './duration.mjs';
|
||||
// Total minutes a volunteer is booked for, from shift strings.
|
||||
export function bookedMinutes(shifts) {
|
||||
return shifts.reduce((sum, s) => sum + parseDuration(s), 0);
|
||||
}
|
||||
J
|
||||
cat > tests/duration.test.mjs <<'J'
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { parseDuration } from '../src/duration.mjs';
|
||||
test('minutes', () => assert.equal(parseDuration('90m'), 90));
|
||||
test('seconds', () => assert.equal(parseDuration('30s'), 0.5));
|
||||
test('bare number is minutes', () => assert.equal(parseDuration('15'), 15));
|
||||
J
|
||||
cat > tests/rota.test.mjs <<'J'
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { bookedMinutes } from '../src/rota.mjs';
|
||||
test('sums shifts', () => assert.equal(bookedMinutes(['90m', '30m', '15']), 135));
|
||||
J
|
||||
Loading…
Add table
Add a link
Reference in a new issue