open/voyage
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity Actions
voyage/docs/operations.md
Kjell Tore Guttormsen 7a5749ddcc docs(voyage): S15 — cost-claim truth-pass + premium default-profile pins 
Resolve the S14 default-profile contradiction. Investigation overturned
the audit addendum's guess ("code is the bug → balanced"): commit 40d8742
(2026-05-13, "pin all sub-agents to Opus permanently (operator request)")
plus VOYAGE_PROFILE=premium in ~/.zshenv establish premium as the deliberate
default. Operator confirmed in-session: premium is the shipped default; fix
the stale docs, not the code. No code or behaviour changed (lib/ untouched).

Docs (default-name → premium, consistent across resolver + all three docs):
- README + docs/profiles.md + docs/operations.md: 3 lookup-order sites and
  3 profile tables now mark `premium` as the default.
- premium table row corrected to all-opus (matches premium.yaml — a third
  inconsistency the audit missed; README/profiles.md showed opus/sonnet/...).

Cost narrative made honest (premium = all-Opus reality):
- §Cost profile rewritten: uniform model per phase, no orchestrator-vs-swarm
  split; cheaper via --profile balanced/economy.
- Removed false "Sonnet exploration/review swarm" claims (README 195/223/266
  model-neutral; 804 parenthetical; the "Switch the planning model" note).
- profiles.md custom-profile prose corrected: built-in wins over same-named
  custom (findProfilePath), dropping the bogus "balanced is the locked default".

Pins (TDD red→green, doc-consistency.test.mjs):
- default-profile name invariant (resolveProfile ↔ README/profiles/operations)
- profile tables ↔ each built-in yaml phase_models (structural, catches drift)
- cost-claim regression guard (no resurrected Sonnet-swarm phrasing)

S16/S18 surface untouched: counts (23 agents, 9/10 dims, 5/6 dims), versions,
framing gates unchanged. Full suite 686 (684 pass / 2 skip / 0 fail).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LqBYc8Ltrk7LipyJmGxXiB
2026-06-18 21:06:36 +02:00

5.3 KiB
Raw Blame History

Voyage — Autonomy gates, profile system, observability

Imported from CLAUDE.md via pointer.

Autonomy mode (--gates, v3.4.0)

All four pipeline commands accept --gates {open|closed|adaptive}:

Value Behavior
open Skip optional checkpoints; trust manifests + verify gates only
closed Stop at every autonomy boundary; operator confirms each transition
adaptive (default) Stop only at meaningful boundaries (manifest-audit FAIL, plan-critic BLOCKER, main-merge gate)

Under the hood: lib/util/autonomy-gate.mjs runs the state machine idle → approved → executing → merge-pending → main-merged. lib/stats/event-emit.mjs records each transition to ${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl. The main-merge gate is the final autonomy boundary before HEAD lands on main.

Path A/B/C decision (v3.4.0; Path C closed 2026-05-05)

Three architectural options were considered for the speedup work:

  • Path A — cache-first (drop --allowedTools per child to recover cross-phase cache sharing): REJECTED. Inverts the security model; plugin hooks don't fire reliably in claude -p (research/06 GH #36071).
  • Path B — sequential --no-ff parallel waves with manifest-driven failure recovery: CHOSEN. Ships in v3.4.0. Phase 2.6 of /trekexecute runs the wave executor with hardenings for plugin-in-monorepo + gitignored-state topology.
  • Path C — hybrid (cache-warm sentinel + identical-tool parallel): CLOSED 2026-05-05. Q3 experiment measured median cache_creation_input_tokens = 163,903 across 3 fork-children at 186K parent context (CC v2.1.128, Sonnet 4.6). Master-plan thresholds: ≤ 1,500 POSITIVE / ≥ 3,500 NEGATIVE. Result is solidly NEGATIVE — CLAUDE_CODE_FORK_SUBAGENT does not preserve cache prefix across identical-tool children at our context size. Path C migration is deferred indefinitely; reassessment is appropriate when CC v2.2.xxx ships fork-cache-relevant features. Harness: scripts/q3-cache-prefix-experiment.mjs. Companion analyser: lib/stats/cache-analyzer.mjs.

A revived Path C (post-v2.2.xxx) would require: (1) re-architecting tool-list to be identical across all wave children, (2) cache-telemetry analysis confirming the new fork-cache behaviour holds, (3) prompt-level deny re-enablement to compensate for tool scoping rollback.

Profile system (--profile, v4.1.0)

Three built-in model profiles plus operator-defined <custom>.yaml. Each profile pins phase_models for the six pipeline phases (brief, research, plan, execute, review, continue). Profile is recorded in plan.md frontmatter as profile: <name> and emitted to ${CLAUDE_PLUGIN_DATA}/trek*-stats.jsonl for cost-attribution.

Profile Brief Research Plan Execute Review Continue Use case
economy sonnet sonnet sonnet sonnet sonnet sonnet Lowest cost; high-confidence small-scope tasks (operator-opt-in via --profile economy)
balanced sonnet sonnet opus sonnet opus sonnet Mixed — opus where reasoning depth pays off (operator-opt-in via --profile balanced)
premium (default) opus opus opus opus opus opus Maximum quality — Opus on every phase. Default since 2026-05-13 operator request; also the hardcoded resolver default returned by resolveProfile() in lib/profiles/resolver.mjs

Lookup order

  1. Explicit --profile <name> flag passed to the command
  2. Plan-file frontmatter profile: (when resuming via /trekexecute --resume or /trekcontinue)
  3. VOYAGE_PROFILE environment variable
  4. Default premium

Custom profiles

Create lib/profiles/<custom>.yaml to define a new tier. The validator (lib/validators/profile-validator.mjs) enforces: every phase_models[].phase must be a known phase enum; every phase_models[].model must match ^(opus|sonnet)(\b|-).* or one of the canonical short names. Custom profiles override built-ins of the same name (lookup is alphabetical with <custom> taking precedence).

Drift between plan-frontmatter profile: and step-manifest profile_used: emits a MANIFEST_PROFILE_DRIFT warning from plan-validator --strict (Step 20). Plan remains valid; the warning surfaces accidental tier-mismatch.

Observability (Stop hook, v4.1.0)

The Stop hook in hooks/hooks.json runs hooks/scripts/otel-export.mjs at session-end. The hook is opt-in — when VOYAGE_EXPORT_MODE is unset or off, no work is done.

Mode Output Endpoint env-var
off (default) (no export) —
textfile voyage.prom (Prometheus exposition format) VOYAGE_TEXTFILE_DIR
otlp OTLP/JSON POST VOYAGE_OTEL_ENDPOINT

Endpoint validation: VOYAGE_OTEL_ALLOW_PRIVATE=1 is required to send to loopback or RFC1918 destinations (CWE-918 SSRF mitigation). Allowlist lib/exporters/field-allowlist.mjs redacts records before export (CWE-212). Path validation (lib/exporters/path-validator.mjs) rejects symlink + traversal (CWE-22).

Local Docker Compose stack: examples/observability/. Operator docs: docs/observability.md. Both pin minimum versions per CVE history (prom/prometheus:v3.0.1, grafana/grafana:11.4.0, otel/opentelemetry-collector-contrib:0.115.0).