voyage/docs/operations.md
Kjell Tore Guttormsen 213cf388de chore(voyage): S34 — V30 economy-profile self-declares experimental (uncalibrated Jaccard floor)
The economy profile's cross-tier Jaccard floor (0.55) rests on parked-synthetic
fixtures; empirical Step-17 calibration is v4.2-budget-gated ($60–120,
unauthorized). Fork resolved as label-not-calibrate: the experimental status,
previously prose-only in docs/profiles.md, now lives in the profile data and is
machine-checked. No new user-facing capability — honest labeling + a guard.

- lib/profiles/economy.yaml: add `experimental: true` (with rationale comment).
- lib/validators/profile-validator.mjs: recognize `experimental` as an OPTIONAL
  boolean; non-boolean → PROFILE_INVALID_ENUM. Absent ⇒ tier is stable
  (premium/balanced unaffected, profile_version stays 1.0 — additive).
- README.md + docs/operations.md + docs/profiles.md: flag the `economy` table
  row "⚠ Experimental (uncalibrated Jaccard floor)".
- tests/synthetic/profile-jaccard-calibration.md + analysis §6/§10 + backlog
  plan §S34: cross-reference the marker; mark V30/S34 done.

+5 tests (739 → 744, 742/2/0): economy declares experimental:true; premium and
balanced do not; validator rejects non-boolean experimental; every profile-doc
economy row is flagged; the flag tracks the calibration's parked-synthetic
status (must drop in the same change that lands real calibration).

Closes the balance backlog (4/4, S31–S34). claude plugin validate green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
2026-06-20 10:18:38 +02:00

6.6 KiB

Voyage — Autonomy gates, profile system, observability

Imported from CLAUDE.md via pointer.

Autonomy mode (--gates, v3.4.0)

All four pipeline commands accept a boolean --gates {true|false} flag. Presence (--gates true, or bare --gates) turns gating on — the run pauses at autonomy boundaries for operator confirmation. Absence (or --gates false, the default) runs phases continuously without pausing.

/trekexecute additionally refines how strict the gating is via a gates_mode policy derived from the brief's effort signal (an explicit operator --gates flag takes precedence over the brief signal — see commands/trekexecute.md § High-effort behavior (v5.1.1)):

gates_mode Derived from (effort) Behavior
adaptive standard / absent (default) Stop only at meaningful boundaries (manifest-audit FAIL, plan-critic BLOCKER, main-merge gate)
closed high Stop at every autonomy boundary; operator confirms each transition
open low Skip optional checkpoints; trust manifests + verify gates only

Under the hood: lib/util/autonomy-gate.mjs runs the state machine idle → gates_on | auto_running → paused_for_gate → completed. start routes to gates_on when --gates true, else auto_running; a phase_boundary from gates_on pauses at paused_for_gate (awaiting resume); finish reaches the terminal completed. The module is pure data with no I/O. Separately, lib/stats/event-emit.mjs records named lifecycle events (brief-approved, main-merge-gate, user_input) to ${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl. The main-merge gate is the final autonomy boundary before HEAD lands on main.

Path A/B/C decision (v3.4.0; Path C closed 2026-05-05)

Three architectural options were considered for the speedup work:

  • Path A — cache-first (drop --allowedTools per child to recover cross-phase cache sharing): REJECTED. Inverts the security model; plugin hooks don't fire reliably in claude -p (research/06 GH #36071).
  • Path B — sequential --no-ff parallel waves with manifest-driven failure recovery: CHOSEN. Ships in v3.4.0. Phase 2.6 of /trekexecute runs the wave executor with hardenings for plugin-in-monorepo + gitignored-state topology.
  • Path C — hybrid (cache-warm sentinel + identical-tool parallel): CLOSED 2026-05-05. Q3 experiment measured median cache_creation_input_tokens = 163,903 across 3 fork-children at 186K parent context (CC v2.1.128, Sonnet 4.6). Master-plan thresholds: ≤ 1,500 POSITIVE / ≥ 3,500 NEGATIVE. Result is solidly NEGATIVE — CLAUDE_CODE_FORK_SUBAGENT does not preserve cache prefix across identical-tool children at our context size. Path C migration is deferred indefinitely; reassessment is appropriate when CC v2.2.xxx ships fork-cache-relevant features. Harness: scripts/q3-cache-prefix-experiment.mjs. Companion analyser: lib/stats/cache-analyzer.mjs.

A revived Path C (post-v2.2.xxx) would require: (1) re-architecting tool-list to be identical across all wave children, (2) cache-telemetry analysis confirming the new fork-cache behaviour holds, (3) prompt-level deny re-enablement to compensate for tool scoping rollback.

Profile system (--profile, v4.1.0)

Three built-in model profiles plus operator-defined <custom>.yaml. Each profile pins phase_models for the six pipeline phases (brief, research, plan, execute, review, continue). Profile is recorded in plan.md frontmatter as profile: <name> and emitted to ${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl for cost-attribution.

Profile Brief Research Plan Execute Review Continue Use case
economy sonnet sonnet sonnet sonnet sonnet sonnet Experimental (uncalibrated Jaccard floor) — lowest cost; high-confidence small-scope tasks (operator-opt-in via --profile economy)
balanced sonnet sonnet opus sonnet opus sonnet Mixed — opus where reasoning depth pays off (operator-opt-in via --profile balanced)
premium (default) opus opus opus opus opus opus Maximum quality — Opus on every phase. Default since 2026-05-13 operator request; also the hardcoded resolver default returned by resolveProfile() in lib/profiles/resolver.mjs

Lookup order

  1. Explicit --profile <name> flag passed to the command
  2. Plan-file frontmatter profile: (when resuming via /trekexecute --resume or /trekcontinue)
  3. VOYAGE_PROFILE environment variable
  4. Default premium

Custom profiles

Create voyage-profiles/<custom>.yaml in the repo root (or ~/.claude/voyage-profiles/<custom>.yaml) to define a new tier — the name must not be a built-in. The validator (lib/validators/profile-validator.mjs) enforces: every phase_models[].phase must be a known phase enum; every phase_models[].model must match ^(opus|sonnet)(\b|-).* or one of the canonical short names. findProfilePath (lib/profiles/resolver.mjs) resolves built-in first (lib/profiles/<name>.yaml for economy/balanced/premium), then repo-root voyage-profiles/, then ~/.claude/voyage-profiles/. A custom file named after a built-in therefore cannot shadow it (custom profiles must use new names); for the same custom name, repo-root takes precedence over home.

Drift between plan-frontmatter profile: and step-manifest profile_used: emits a MANIFEST_PROFILE_DRIFT warning from plan-validator --strict (Step 20). Plan remains valid; the warning surfaces accidental tier-mismatch.

Observability (Stop hook, v4.1.0)

The Stop hook in hooks/hooks.json runs hooks/scripts/otel-export.mjs at session-end. The hook is opt-in — when VOYAGE_EXPORT_MODE is unset or off, no work is done.

Mode Output Endpoint env-var
off (default) (no export)
textfile voyage.prom (Prometheus exposition format) VOYAGE_TEXTFILE_DIR
otlp OTLP/JSON POST VOYAGE_OTEL_ENDPOINT

Endpoint validation: VOYAGE_OTEL_ALLOW_PRIVATE=1 is required to send to loopback or RFC1918 destinations (CWE-918 SSRF mitigation). Allowlist lib/exporters/field-allowlist.mjs redacts records before export (CWE-212). Path validation (lib/exporters/path-validator.mjs) rejects symlink + traversal (CWE-22).

Local Docker Compose stack: examples/observability/. Operator docs: docs/observability.md. Both pin minimum versions per CVE history (prom/prometheus:v3.0.1, grafana/grafana:11.4.0, otel/opentelemetry-collector-contrib:0.115.0).