voyage/tests/fixtures
Kjell Tore Guttormsen 52b87978cb
fix(verification): the runner refuses writes a brief may not perform
/trekreview now runs the commands a BRIEF declares, and a brief is an artifact
that can arrive from outside the repo. Measured 2026-09-18 on 6cafb4c: the
executor denylist stopped a download piped into a shell, but the remote-writing
git subcommand and a recursive delete of a path both RAN. The denylist screens
catastrophe (root deletion, fork bombs, mkfs); it was never meant to screen an
artifact under review.

A second screen, in the runner and ahead of the denylist, refuses four classes:

- a remote-writing git subcommand. The subcommand is found by walking git's own
  options (`-C`, `-c`, `--git-dir`, ... take a value), so `git status` and
  `git log` still run and `git -C sub push` does not.
- a recursive delete: any `rm` carrying `-r`/`-rf`/`--recursive`. A plain
  `rm build/artifact.txt` still runs.
- a download piped straight into a shell (also caught by the denylist; pinned
  here so the runner does not depend on another file for it).
- a write outside the working tree. `/dev/null`-class devices are fine, and so
  is anything under the working tree; `~/...`, an absolute path elsewhere, and
  a target carrying an unexpanded `$VAR` are refused - the runner cannot know
  where a variable points, and guessing is how a screen stops screening.

A refusal is its own outcome, REFUSED_BY_POLICY: the command never reaches a
shell, and `summary.ok` is false in both plan and brief mode. For the reviewer,
REFUSED is like NOT RUN - the absence of a measurement, never on its own a
finding - and the rubric and the evidence block both say so.

Chosen deliberately, and it is stricter than today's habit: writing scratch to
/tmp is refused too. The repo's own example plan does `> /tmp/out`. Verification
output belongs in the working tree; exempting the whole system temp dir would
have made the rule unstatable, since a working tree created under /tmp then
contains its own escape hatch.

NOT covered, stated rather than implied:
- other writing git subcommands (tag, remote, config, gc) - only push is listed
- writes through a wrapper: `sh -c '...'`, `xargs`, `find -exec`, a Makefile
  target, a script the criterion invokes. The screen reads the command it is
  given, not what that command goes on to do.
- `>` inside a quoted string reads as a redirect, so a criterion echoing a
  literal `>` is refused. Fail-closed, on purpose.
- the whole surface still runs with the invoking process's permissions; this is
  a refusal list, not a sandbox.

The denylist-layer test now uses a stand-in command with a screen double: the
refusal list catches a recursive delete first, so naming one there would have
stopped exercising the denylist layer at all.

Red first: the 6 new tests failed before this change (`refuseCommand` did not
exist), and the fixture brief's four writes ran.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 02:14:01 +02:00
..
bakeoff feat(voyage): S10 — NW2 part A (Workflow port + fidelity harness + smoke) 2026-06-18 14:40:19 +02:00
bakeoff-rich feat(eval): SKAL-1·4b offline gold-scored output eval 2026-06-30 09:00:33 +02:00
session-state test(ultraplan-local): add session-state fixtures 2026-05-01 20:21:50 +02:00
synthesis feat(voyage): S12 — NW3 synthesis-agent built + measured → declined per measurement [skip-docs] 2026-06-18 17:58:39 +02:00
token-usage feat(stats): add pure token-usage parser + cache-aware cost derivation 2026-06-26 14:32:10 +02:00
trekreview feat(voyage)!: bulk content rewrite ultra -> voyage/trek prose [skip-docs] 2026-05-05 15:08:20 +02:00
brief-effort-fable.md test(validators): cover fable accept + unknown-model reject in both gate layers 2026-07-02 16:56:26 +02:00
brief-effort-high.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-effort-low.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-effort-standard.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-refused-commands.md fix(verification): the runner refuses writes a brief may not perform 2026-09-18 02:14:01 +02:00
brief-success-criteria.md fix(review): run the success-criteria commands and hand the reviewer the result (D-04) 2026-09-18 01:34:18 +02:00
brief-v21-no-signals.md test(voyage): add 4 brief fixtures for v5.1.1 runtime scenarios 2026-05-14 21:34:51 +02:00
brief-with-phase-signals.md feat(voyage): bump trekbrief-template to brief_version 2.1 + add phase_signals fixtures 2026-05-13 21:09:57 +02:00
brief-without-phase-signals.md feat(voyage): bump trekbrief-template to brief_version 2.1 + add phase_signals fixtures 2026-05-13 21:09:57 +02:00
end-state-frozen.json fix(end-state): the D-03/D-04 probes RUN a named test instead of grepping for a symbol 2026-09-18 02:08:40 +02:00
expected.prom feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
jsonl-schemas.md fix(storm-measure): check BOTH halves of the activation SC, not just the count delta 2026-08-12 23:01:59 +02:00
plan-fase-narrative.md chore(release): ultraplan-local v3.1.0 2026-05-01 06:31:42 +02:00
plan-profile-drift.md feat(voyage): emit MANIFEST_PROFILE_DRIFT warning in plan-validator strict mode — brief assumption 7 2026-05-09 10:02:53 +02:00
plan-verification-fails.md fix(execute): run the plan's Verification on the single-session path (D-03) 2026-09-18 01:30:57 +02:00
plan-verification-passes.md fix(execute): run the plan's Verification on the single-session path (D-03) 2026-09-18 01:30:57 +02:00
plan-with-profile.md feat(voyage): add lib/profiles/resolver.mjs — locked interface SC #5-#9 2026-05-09 09:29:01 +02:00
plan-without-profile.md feat(voyage): add lib/profiles/resolver.mjs — locked interface SC #5-#9 2026-05-09 09:29:01 +02:00
profile-invalid-enum.yaml feat(voyage): add lib/validators/profile-validator.mjs — SC #1, #2, #3 2026-05-09 09:26:23 +02:00
profile-invalid-model.yaml feat(voyage): add lib/validators/profile-validator.mjs — SC #1, #2, #3 2026-05-09 09:26:23 +02:00
research-deep-research-adapted.md test(trekresearch): pin deep-research adapter output contract 2026-06-30 13:38:09 +02:00
stats-sample.jsonl feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
stats-with-profile.jsonl feat(voyage): add 5 additive profile fields to JSONL stats — SC #11 2026-05-09 09:40:21 +02:00