From 7e78d076f993f521da5fff6fd2f42c525c8c1708 Mon Sep 17 00:00:00 2001 From: Kjell Tore Guttormsen Date: Fri, 26 Jun 2026 14:47:24 +0200 Subject: [PATCH] docs(observability): document token-usage schema + main-context v1 scope --- docs/observability.md | 42 +++++++++++++++++++++++++++++++++ tests/fixtures/jsonl-schemas.md | 5 +++- 2 files changed, 46 insertions(+), 1 deletion(-) diff --git a/docs/observability.md b/docs/observability.md index cc0fe36..3c64102 100644 --- a/docs/observability.md +++ b/docs/observability.md @@ -60,6 +60,7 @@ operator-private data (paths, prompts, brief content). | `VOYAGE_TEXTFILE_DIR` | `${CLAUDE_PLUGIN_DATA}` | Directory for `voyage.prom` (textfile mode) | | `VOYAGE_OTEL_ENDPOINT` | _(none)_ | HTTPS URL for OTLP/HTTP POST | | `VOYAGE_OTEL_ALLOW_PRIVATE` | _(unset)_ | Set to `1` to allow loopback / RFC1918 endpoints | +| `VOYAGE_TOKEN_METER` | _(unset)_ | Set to a truthy value to capture per-session token/cost into `token-usage-stats.jsonl` on Stop (default off → zero added latency). See **Token/cost metering** below. | ## Docker Compose quickstart @@ -88,6 +89,47 @@ the allowlist explicitly. This is intentional: `${CLAUDE_PLUGIN_DATA}` is trusted local storage; OTel endpoints are operator-controlled and may be external. +## Token/cost metering + +> **SKAL-2.** Opt-in capture of per-session token usage and a cache-aware USD +> cost estimate, folded into the existing Stop hook (`hooks/scripts/otel-export.mjs`). +> No new hook is added — capture rides the same Stop event, so there is no +> second-Stop-hook ordering race. + +**Activation.** Set `VOYAGE_TOKEN_METER` to any truthy value. When unset (the +default) the capture path is skipped entirely — zero added Stop latency. When +set, the hook reads the Claude Code transcript (`transcript_path` from the Stop +payload), sums token usage, derives cost, and **upserts** one record per session +into `${CLAUDE_PLUGIN_DATA}/token-usage-stats.jsonl`. Capture is fail-open: any +error (malformed payload, unreadable transcript) is swallowed and never blocks +Stop. Once captured, the record is exported like any other stats file when +`VOYAGE_EXPORT_MODE` is `textfile` or `otlp`. + +**Schema (`token-usage`).** Flat numeric record: +`ts`, `session_id`, `scope`, `model`, `tokens_input`, `tokens_output`, +`tokens_cache_creation`, `tokens_cache_read`, `cost_usd`, `is_estimate`, +`price_table_version`. The field allowlist (`lib/exporters/field-allowlist.mjs`, +`TOKEN_USAGE_ALLOWED`) admits only the numeric + low-cardinality-label fields and +**strips `session_id` at export** (CWE-212). The exporter auto-promotes each +numeric field to a metric: `voyage_token_usage_tokens_input` (Prometheus) / +`voyage.token-usage.tokens_input` (OTLP). + +**Cost contract (honesty).** `cost_usd` is computed from a dated, in-source +`PRICE_TABLE` (per-MTok USD), cache-aware: +`input + output + cache_creation×(5m write rate) + cache_read×(read rate)`. +Each record carries `price_table_version` (the date the prices were resolved) +and `is_estimate`. When the transcript's model is **not** in the price table, the +meter **refuses to guess**: `cost_usd` is `null` and `is_estimate` is `true`. +Prices are volatile — re-resolve them against the `claude-api` reference and bump +`PRICE_TABLE_VERSION` when they change. + +**v1 limitation — MAIN-CONTEXT only.** The meter reads the main-session +transcript, which contains only `isSidechain:false` records. Sub-agent (swarm) +turns live in separate `agent-*.jsonl` sibling files and are **not** counted, so +`cost_usd` is a lower bound on total Voyage cost, not the session total. Every +record is stamped `scope:'main-context'` to make this explicit. Per-subagent +attribution is a documented v2 follow-on (it was a Non-Goal for v1). + ## Security The exporter is hardened against three CWE classes: diff --git a/tests/fixtures/jsonl-schemas.md b/tests/fixtures/jsonl-schemas.md index 0275466..1a6ab5f 100644 --- a/tests/fixtures/jsonl-schemas.md +++ b/tests/fixtures/jsonl-schemas.md @@ -27,6 +27,7 @@ | trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) | | trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | commands/trekreview.md (orchestrator-emit Phase 8) | trekreview.md:255 | profile, phase_models, profile_source | none | | trekcontinue-stats | ts, project, next_session_label, status | commands/trekcontinue.md (orchestrator-emit Phase 5) | trekcontinue.md:289 | profile, profile_source | none | +| token-usage | ts, session_id, scope, model, tokens_input, tokens_output, tokens_cache_creation, tokens_cache_read, cost_usd, is_estimate, price_table_version | lib/stats/token-usage.mjs `captureTokenUsage` (via hooks/scripts/otel-export.mjs Stop hook, opt-in `VOYAGE_TOKEN_METER`) | token-usage.mjs:buildRecord | n/a (SKAL-2 schema) | session_id (stripped at export) | ## Field-allowlist input for Step 11 @@ -46,7 +47,9 @@ critic_verdict, guardian_verdict, outcome, plan_type, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step, verdict, reviewed_files_count, duration_ms, status, next_session_label, event, known_event, success, scope, -profile, profile_source, parallel_agents, external_research_enabled +profile, profile_source, parallel_agents, external_research_enabled, +model, tokens_input, tokens_output, tokens_cache_creation, tokens_cache_read, +cost_usd, is_estimate, price_table_version ``` **EXPORT_DENYLIST** (PII or high-cardinality, never export):