open/voyage
1
0
Fork 0
Code Issues Releases 13 Activity
voyage/tests/fixtures/jsonl-schemas.md
Kjell Tore Guttormsen f6e18a1d09
 feat(skills): interviewing - one source for the brief's question round (order 7e) 
/trekbrief Phase 3 carried the interview form inline. It now asks for the
new skill voyage:interviewing by name (allowed-tools gains Skill) and keeps
only its own contract: the size class and budget, the three fixed questions
(framing first, class, storage), where each answer lands in the brief, and
the answers that decide nothing. Step 3a and the form points of Step 3c move
into the skill with their credits (Pocock, Osmani, Spec Kit; licences and
line ranges checked at the pinned commits).

Red baseline first (3 helper interviews against a fixed answer sheet,
Opus 5.5): 10-11 questions in round 1 (3 of 3), 15 of 31 without a
recommendation, a hollow answer written in as a decision (2 of 3), unasked
scope as a requirement (2 of 3). With the skill: 4, 0 of 12, 0 of 3, 0 of 3.
Fact questions and stopping did not fail without it.

Also: pointers to voyage:writing-criteria (S3) where criteria are asked and
written, both review skill names (S3 + S4) in the brief-reviewer prompt; the
intent pins that read moved text now read the skill, same assertions; the
Phase 7 line_ref moves with the shorter Phase 3; 7 eval cases (3/2/2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 07:44:42 +02:00

6.2 KiB
Raw Blame History

Voyage JSONL stats — schema audit (v4.1 input)

Purpose: Field-allowlist input for v4.1 OTel exporter (Step 11). Lists every field every voyage stats JSONL writer emits today, plus the additive fields v4.1 introduces. Load-bearing for Step 11 (field-allowlist) and Step 8 (stats plumbing).

PII-flag: command_excerpt from hooks/scripts/post-bash-stats.mjs slices the first 120 chars of an arbitrary Bash command — may contain operator paths, branch names, or fragments of secrets that survived the secrets-hook. CWE-212 (Improper Cross-boundary Removal of Sensitive Data). The OTel exporter MUST NOT export this field unless the operator explicitly opts in via VOYAGE_EXPORT_INCLUDE_COMMAND_EXCERPT=1 (deferred to v4.2 — v4.1 hard-excludes).

Additive v4.1 fields: profile, phase_models, parallel_agents, external_research_enabled, profile_source. All are forward-compat: existing v4.0 consumers ignore unknown keys, v4.1 consumers get richer signal.

Field table per JSONL writer

schema_id fields writer_path line_ref v4.1 additive PII
trekbrief-stats ts, task, slug, mode, interview_turns, review_iterations, brief_quality, research_topics, auto_research, auto_result, intent_approved, project_dir, order_id, order_claimed_at, order_reason commands/trekbrief.md (orchestrator-emit Phase 7; order_* added by lib/stats/stats-append.mjs) trekbrief.md:1127-1151 profile, phase_models, profile_source none
trekresearch-stats ts, question, mode, scope, engine, slug, project_dir, brief_path, dimensions, dimensions_baseline, dimensions_baseline_preserved, effort, conv_turns, empty_turns, unique_sources, agents_local, agents_external, gemini_used, confidence, contradictions, open_questions lib/stats/stats-append.mjs (trekresearch.md Stats tracking) trekresearch.md:782-808 profile, phase_models, parallel_agents, external_research_enabled, profile_source none
trekplan-stats ts, task, mode, slug, brief_path, project_dir, codebase_size, codebase_files, agents_deployed, deep_dives, research_briefs_used, research_scout_used, critic_verdict, guardian_verdict, outcome lib/stats/stats-append.mjs (trekplan.md Phase 12) trekplan.md:1271-1299 profile, phase_models, parallel_agents, profile_source none
trekexecute-stats (Phase 9 record) ts, plan, plan_type, mode, result, steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step lib/stats/stats-append.mjs (trekexecute.md Phase 9) trekexecute.md:2892-2914 profile, phase_models, profile_source none
trekexecute-stats (autonomy events) ts, event, known_event, payload lib/stats/event-emit.mjs emit() event-emit.mjs:78-99 payload.profile, payload.phase_models, payload.profile_source none
trekexecute-stats (PostToolUse Bash) ts, session_id, command_excerpt, duration_ms, success hooks/scripts/post-bash-stats.mjs (Bash PostToolUse) post-bash-stats.mjs:42-54 none (hook is plugin-level, not profile-aware) command_excerpt (CWE-212)
trekreview-stats ts, slug, verdict, counts (BLOCKER/MAJOR/MINOR/SUGGESTION), reviewed_files_count, mode, duration_ms lib/stats/stats-append.mjs (trekreview.md Phase 8) trekreview.md:623-673 profile, phase_models, profile_source none
trekcontinue-stats ts, project, next_session_label, status lib/stats/stats-append.mjs (trekcontinue.md Phase 5) trekcontinue.md:293-297 profile, profile_source none
token-usage ts, session_id, scope, model, tokens_input, tokens_output, tokens_cache_creation, tokens_cache_read, cost_usd, is_estimate, price_table_version lib/stats/token-usage.mjs captureTokenUsage (via hooks/scripts/otel-export.mjs Stop hook, opt-in VOYAGE_TOKEN_METER) token-usage.mjs:buildRecord n/a (SKAL-2 schema) session_id (stripped at export)

Field-allowlist input for Step 11

The OTel exporter (Step 11 lib/exporters/field-allowlist.mjs) MUST inline the following static const arrays (NOT load from this file at runtime — Step 11 explicit constraint: INLINE static const, IKKE runtime fra tests/fixtures):

EXPORT_ALLOWLIST (numeric/bool/short-string fields safe for OTel metric labels):

ts, slug, mode, brief_quality, auto_research, auto_result,
codebase_size, codebase_files, agents_deployed, deep_dives,
agents_local, agents_external, gemini_used, dimensions, confidence,
contradictions, open_questions, interview_turns, review_iterations,
research_topics, research_briefs_used, research_scout_used,
critic_verdict, guardian_verdict, outcome, plan_type, result,
steps_total, steps_passed, steps_failed, steps_skipped, failed_at_step,
verdict, reviewed_files_count, duration_ms, status, next_session_label,
event, known_event, success, scope,
profile, profile_source, parallel_agents, external_research_enabled,
model, tokens_input, tokens_output, tokens_cache_creation, tokens_cache_read,
cost_usd, is_estimate, price_table_version

EXPORT_DENYLIST (PII or high-cardinality, never export):

task, question, project_dir, project, plan, brief_path, command_excerpt, payload, counts, phase_models, session_id,
order_id, order_claimed_at, order_reason

Notes:

  • task and question may contain user-content prose → high-cardinality + PII risk.
  • project_dir and paths leak filesystem layout.
  • command_excerpt per CWE-212 above.
  • phase_models is a structured object (6 keys) — too high-cardinality for label; profile name (profile) is the safe summary. v4.2 may revisit if operators ask.
  • counts (review BLOCKER/MAJOR/MINOR/SUGGESTION) is a nested object — Step 11 exporter flattens to voyage_review_counts_blocker/_major/_minor/_suggestion metrics rather than a label.
  • session_id is a UUID — high-cardinality, not useful as a label, log-only.

Cross-reference

  • Step 8 (stats plumbing) — adds profile + phase_models + profile_source to all 6 orchestrator-emit sites listed above.
  • Step 11 (field-allowlist) — codifies the EXPORT_ALLOWLIST/DENYLIST arrays above as inline static consts in lib/exporters/field-allowlist.mjs.
  • Step 9 (Prometheus textfile) — emits one metric line per allowlist-numeric field per JSONL writer; PII-flagged fields are dropped at format-layer, not export-layer.