voyage/tests/lib
Kjell Tore Guttormsen 52b87978cb
fix(verification): the runner refuses writes a brief may not perform
/trekreview now runs the commands a BRIEF declares, and a brief is an artifact
that can arrive from outside the repo. Measured 2026-09-18 on 6cafb4c: the
executor denylist stopped a download piped into a shell, but the remote-writing
git subcommand and a recursive delete of a path both RAN. The denylist screens
catastrophe (root deletion, fork bombs, mkfs); it was never meant to screen an
artifact under review.

A second screen, in the runner and ahead of the denylist, refuses four classes:

- a remote-writing git subcommand. The subcommand is found by walking git's own
  options (`-C`, `-c`, `--git-dir`, ... take a value), so `git status` and
  `git log` still run and `git -C sub push` does not.
- a recursive delete: any `rm` carrying `-r`/`-rf`/`--recursive`. A plain
  `rm build/artifact.txt` still runs.
- a download piped straight into a shell (also caught by the denylist; pinned
  here so the runner does not depend on another file for it).
- a write outside the working tree. `/dev/null`-class devices are fine, and so
  is anything under the working tree; `~/...`, an absolute path elsewhere, and
  a target carrying an unexpanded `$VAR` are refused - the runner cannot know
  where a variable points, and guessing is how a screen stops screening.

A refusal is its own outcome, REFUSED_BY_POLICY: the command never reaches a
shell, and `summary.ok` is false in both plan and brief mode. For the reviewer,
REFUSED is like NOT RUN - the absence of a measurement, never on its own a
finding - and the rubric and the evidence block both say so.

Chosen deliberately, and it is stricter than today's habit: writing scratch to
/tmp is refused too. The repo's own example plan does `> /tmp/out`. Verification
output belongs in the working tree; exempting the whole system temp dir would
have made the rule unstatable, since a working tree created under /tmp then
contains its own escape hatch.

NOT covered, stated rather than implied:
- other writing git subcommands (tag, remote, config, gc) - only push is listed
- writes through a wrapper: `sh -c '...'`, `xargs`, `find -exec`, a Makefile
  target, a script the criterion invokes. The screen reads the command it is
  given, not what that command goes on to do.
- `>` inside a quoted string reads as a redirect, so a criterion echoing a
  literal `>` is refused. Fail-closed, on purpose.
- the whole surface still runs with the invoking process's permissions; this is
  a refusal list, not a sandbox.

The denylist-layer test now uses a stand-in command with a screen double: the
refusal list catches a recursive delete first, so naming one there would have
stopped exercising the denylist layer at all.

Red first: the 6 new tests failed before this change (`refuseCommand` did not
exist), and the fixture brief's four writes ran.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 02:14:01 +02:00
..
agent-frontmatter.test.mjs release(v5.10.1): drop gemini-bridge from the pipeline; correct the T1 §6 PoC status 2026-09-03 20:29:39 +02:00
arg-parser.test.mjs feat(voyage): S11 — NW2 part B, integrate opt-in --workflow flag 2026-06-18 17:22:09 +02:00
atomic-write.test.mjs refactor(ultraplan-local): extract atomicWriteJson to lib/util 2026-05-01 20:21:15 +02:00
autonomy-gate.test.mjs feat(ultraplan-local): add autonomy-gate state machine + manifest schema extensions for skip_commit_check + memory_write 2026-05-04 06:28:47 +02:00
bash-normalize.test.mjs feat(ultraplan-local): Spor 1 wave 1 — lib/parsers + 66 tests grønn 2026-05-01 05:35:28 +02:00
cache-analyzer.test.mjs test(observability): close 2 MAJOR test gaps from SKAL-2 review 2026-06-26 17:14:03 +02:00
cleanup.test.mjs feat(voyage)!: rename produced_by field values + validator comments [skip-docs] 2026-05-05 14:42:21 +02:00
coordinator-contract.test.mjs fix(review): an anonymous invalid payload is unattributable, not a reviewer named "unnamed reviewer" 2026-09-01 22:54:31 +02:00
criteria-runner.test.mjs fix(verification): the runner refuses writes a brief may not perform 2026-09-18 02:14:01 +02:00
doc-consistency.test.mjs fix(verification): a backticked span is only run when it IS a command 2026-09-18 02:04:46 +02:00
fidelity-diff.test.mjs feat(voyage): S10 — NW2 part A (Workflow port + fidelity harness + smoke) 2026-06-18 14:40:19 +02:00
finding-id.test.mjs feat(ultraplan-local): add lib/parsers/finding-id.mjs (stable SHA1) 2026-05-01 13:28:05 +02:00
findings-schema.test.mjs feat(voyage): S9 — NW1 reviewer-output schema contract (TDD, ungated) 2026-06-18 14:06:07 +02:00
frontmatter.test.mjs feat(voyage)!: bulk content rewrite ultra -> voyage/trek prose [skip-docs] 2026-05-05 15:08:20 +02:00
gates-flag-coverage.test.mjs feat(voyage)!: rename commands, templates, fixtures for v4.0.0 [skip-docs] 2026-05-05 14:13:44 +02:00
gold-corpus.test.mjs feat(eval): add gold.json golden corpus + loader/validator test 2026-06-26 11:51:46 +02:00
gold-eval.test.mjs feat(eval): SKAL-1·4b offline gold-scored output eval 2026-06-30 09:00:33 +02:00
gold-scorer.test.mjs feat(eval): SKAL-1·4b offline gold-scored output eval 2026-06-30 09:00:33 +02:00
jaccard.test.mjs feat(ultraplan-local): add lib/parsers/jaccard.mjs 2026-05-01 13:28:44 +02:00
main-merge-gate.test.mjs feat(voyage)!: rename commands, templates, fixtures for v4.0.0 [skip-docs] 2026-05-05 14:13:44 +02:00
manifest-schema-extensions.test.mjs feat(voyage): add OPTIONAL_STRING_KEYS path to manifest-yaml — profile_used additive 2026-05-09 09:23:32 +02:00
manifest-yaml.test.mjs feat(graceful-handoff): 2.0 — migrate to skills/ with disable-model-invocation [skip-docs] 2026-05-01 05:45:26 +02:00
phase-signal-resolver.test.mjs test(validators): cover fable accept + unknown-model reject in both gate layers 2026-07-02 16:56:26 +02:00
plan-review-dedup.test.mjs fix(voyage): S23 — make /trekplan Phase 9 dedup executable (defect #1) 2026-06-19 21:12:36 +02:00
plan-schema.test.mjs fix(voyage): S26 — parse plan_version prose form (S22 defect #2/#3) 2026-06-19 21:59:31 +02:00
profile-application.test.mjs test(profiles): pin fable profile resolution end-to-end 2026-07-02 17:00:43 +02:00
profile-flag-coverage.test.mjs feat(voyage): document --profile flag in all 6 commands — SC #4 + arv-policy 2026-05-09 09:38:36 +02:00
profile-resolver.test.mjs test(profiles): pin fable profile resolution end-to-end 2026-07-02 17:00:43 +02:00
profile-stats-fields.test.mjs feat(trekresearch): emit and export the five STORM measurement fields 2026-08-09 15:01:15 +02:00
project-discovery.test.mjs feat(voyage)!: bulk content rewrite ultra -> voyage/trek prose [skip-docs] 2026-05-05 15:08:20 +02:00
research-loop-cap.test.mjs feat(research-loop-cap): give the discovery ceiling a reader, not just a sentence 2026-08-12 23:09:11 +02:00
review-determinism.test.mjs feat(voyage)!: rename commands, templates, fixtures for v4.0.0 [skip-docs] 2026-05-05 14:13:44 +02:00
rule-catalogue.test.mjs feat(ultraplan-local): add lib/review/rule-catalogue.mjs (12 rule keys) 2026-05-01 13:27:29 +02:00
source-findings.test.mjs feat(voyage)!: bulk content rewrite ultra -> voyage/trek prose [skip-docs] 2026-05-05 15:08:20 +02:00
stats-event-emit.test.mjs feat(voyage)!: rename stats filenames, settings keys, hook prefixes [skip-docs] 2026-05-05 14:49:03 +02:00
synthesis-digest-schema.test.mjs feat(voyage): S12 — NW3 synthesis-agent built + measured → declined per measurement [skip-docs] 2026-06-18 17:58:39 +02:00
test-census.test.mjs feat(eval): SKAL-1·4b offline gold-scored output eval 2026-06-30 09:00:33 +02:00
token-usage.test.mjs feat(stats): add claude-fable-5 to PRICE_TABLE 2026-07-02 17:15:36 +02:00