docs(observability): document token-usage schema + main-context v1 scope
This commit is contained in:
parent
46d51f8088
commit
7e78d076f9
2 changed files with 46 additions and 1 deletions
|
|
@ -60,6 +60,7 @@ operator-private data (paths, prompts, brief content).
|
||||||
| `VOYAGE_TEXTFILE_DIR` | `${CLAUDE_PLUGIN_DATA}` | Directory for `voyage.prom` (textfile mode) |
|
| `VOYAGE_TEXTFILE_DIR` | `${CLAUDE_PLUGIN_DATA}` | Directory for `voyage.prom` (textfile mode) |
|
||||||
| `VOYAGE_OTEL_ENDPOINT` | _(none)_ | HTTPS URL for OTLP/HTTP POST |
|
| `VOYAGE_OTEL_ENDPOINT` | _(none)_ | HTTPS URL for OTLP/HTTP POST |
|
||||||
| `VOYAGE_OTEL_ALLOW_PRIVATE` | _(unset)_ | Set to `1` to allow loopback / RFC1918 endpoints |
|
| `VOYAGE_OTEL_ALLOW_PRIVATE` | _(unset)_ | Set to `1` to allow loopback / RFC1918 endpoints |
|
||||||
|
| `VOYAGE_TOKEN_METER` | _(unset)_ | Set to a truthy value to capture per-session token/cost into `token-usage-stats.jsonl` on Stop (default off → zero added latency). See **Token/cost metering** below. |
|
||||||
|
|
||||||
## Docker Compose quickstart
|
## Docker Compose quickstart
|
||||||
|
|
||||||
|
|
@ -88,6 +89,47 @@ the allowlist explicitly. This is intentional: `${CLAUDE_PLUGIN_DATA}` is
|
||||||
trusted local storage; OTel endpoints are operator-controlled and may be
|
trusted local storage; OTel endpoints are operator-controlled and may be
|
||||||
external.
|
external.
|
||||||
|
|
||||||
|
## Token/cost metering
|
||||||
|
|
||||||
|
> **SKAL-2.** Opt-in capture of per-session token usage and a cache-aware USD
|
||||||
|
> cost estimate, folded into the existing Stop hook (`hooks/scripts/otel-export.mjs`).
|
||||||
|
> No new hook is added — capture rides the same Stop event, so there is no
|
||||||
|
> second-Stop-hook ordering race.
|
||||||
|
|
||||||
|
**Activation.** Set `VOYAGE_TOKEN_METER` to any truthy value. When unset (the
|
||||||
|
default) the capture path is skipped entirely — zero added Stop latency. When
|
||||||
|
set, the hook reads the Claude Code transcript (`transcript_path` from the Stop
|
||||||
|
payload), sums token usage, derives cost, and **upserts** one record per session
|
||||||
|
into `${CLAUDE_PLUGIN_DATA}/token-usage-stats.jsonl`. Capture is fail-open: any
|
||||||
|
error (malformed payload, unreadable transcript) is swallowed and never blocks
|
||||||
|
Stop. Once captured, the record is exported like any other stats file when
|
||||||
|
`VOYAGE_EXPORT_MODE` is `textfile` or `otlp`.
|
||||||
|
|
||||||
|
**Schema (`token-usage`).** Flat numeric record:
|
||||||
|
`ts`, `session_id`, `scope`, `model`, `tokens_input`, `tokens_output`,
|
||||||
|
`tokens_cache_creation`, `tokens_cache_read`, `cost_usd`, `is_estimate`,
|
||||||
|
`price_table_version`. The field allowlist (`lib/exporters/field-allowlist.mjs`,
|
||||||
|
`TOKEN_USAGE_ALLOWED`) admits only the numeric + low-cardinality-label fields and
|
||||||
|
**strips `session_id` at export** (CWE-212). The exporter auto-promotes each
|
||||||
|
numeric field to a metric: `voyage_token_usage_tokens_input` (Prometheus) /
|
||||||
|
`voyage.token-usage.tokens_input` (OTLP).
|
||||||
|
|
||||||
|
**Cost contract (honesty).** `cost_usd` is computed from a dated, in-source
|
||||||
|
`PRICE_TABLE` (per-MTok USD), cache-aware:
|
||||||
|
`input + output + cache_creation×(5m write rate) + cache_read×(read rate)`.
|
||||||
|
Each record carries `price_table_version` (the date the prices were resolved)
|
||||||
|
and `is_estimate`. When the transcript's model is **not** in the price table, the
|
||||||
|
meter **refuses to guess**: `cost_usd` is `null` and `is_estimate` is `true`.
|
||||||
|
Prices are volatile — re-resolve them against the `claude-api` reference and bump
|
||||||
|
`PRICE_TABLE_VERSION` when they change.
|
||||||
|
|
||||||
|
**v1 limitation — MAIN-CONTEXT only.** The meter reads the main-session
|
||||||
|
transcript, which contains only `isSidechain:false` records. Sub-agent (swarm)
|
||||||
|
turns live in separate `agent-*.jsonl` sibling files and are **not** counted, so
|
||||||
|
`cost_usd` is a lower bound on total Voyage cost, not the session total. Every
|
||||||
|
record is stamped `scope:'main-context'` to make this explicit. Per-subagent
|
||||||
|
attribution is a documented v2 follow-on (it was a Non-Goal for v1).
|
||||||
|
|
||||||
## Security
|
## Security
|
||||||
|
|
||||||
The exporter is hardened against three CWE classes:
|
The exporter is hardened against three CWE classes:
|
||||||
|
|
|
||||||
5
tests/fixtures/jsonl-schemas.md
vendored
5
tests/fixtures/jsonl-schemas.md
vendored
|
|
@ -27,6 +27,7 @@
|
||||||
| trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
|
| trekexecute-stats (PostToolUse Bash) | ts, session_id, command_excerpt, duration_ms, success | hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) | post-bash-stats.mjs:42-54 | none (hook is plugin-level, not profile-aware) | command_excerpt (CWE-212) |
|
||||||
| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | commands/trekreview.md (orchestrator-emit Phase 8) | trekreview.md:255 | profile, phase_models, profile_source | none |
|
| trekreview-stats | ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms | commands/trekreview.md (orchestrator-emit Phase 8) | trekreview.md:255 | profile, phase_models, profile_source | none |
|
||||||
| trekcontinue-stats | ts, project, next_session_label, status | commands/trekcontinue.md (orchestrator-emit Phase 5) | trekcontinue.md:289 | profile, profile_source | none |
|
| trekcontinue-stats | ts, project, next_session_label, status | commands/trekcontinue.md (orchestrator-emit Phase 5) | trekcontinue.md:289 | profile, profile_source | none |
|
||||||
|
| token-usage | ts, session_id, scope, model, tokens_input, tokens_output, tokens_cache_creation, tokens_cache_read, cost_usd, is_estimate, price_table_version | lib/stats/token-usage.mjs `captureTokenUsage` (via hooks/scripts/otel-export.mjs Stop hook, opt-in `VOYAGE_TOKEN_METER`) | token-usage.mjs:buildRecord | n/a (SKAL-2 schema) | session_id (stripped at export) |
|
||||||
|
|
||||||
## Field-allowlist input for Step 11
|
## Field-allowlist input for Step 11
|
||||||
|
|
||||||
|
|
@ -46,7 +47,9 @@ critic_verdict, guardian_verdict, outcome, plan_type, result,
|
||||||
steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step,
|
steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step,
|
||||||
verdict, reviewed_files_count, duration_ms, status, next_session_label,
|
verdict, reviewed_files_count, duration_ms, status, next_session_label,
|
||||||
event, known_event, success, scope,
|
event, known_event, success, scope,
|
||||||
profile, profile_source, parallel_agents, external_research_enabled
|
profile, profile_source, parallel_agents, external_research_enabled,
|
||||||
|
model, tokens_input, tokens_output, tokens_cache_creation, tokens_cache_read,
|
||||||
|
cost_usd, is_estimate, price_table_version
|
||||||
```
|
```
|
||||||
|
|
||||||
**EXPORT_DENYLIST** (PII or high-cardinality, never export):
|
**EXPORT_DENYLIST** (PII or high-cardinality, never export):
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue