test(trekresearch): pin deep-research adapter output contract

This commit is contained in:
Kjell Tore Guttormsen 2026-06-30 13:38:09 +02:00
commit 581489a513
2 changed files with 84 additions and 0 deletions

View file

@ -0,0 +1,35 @@
// tests/commands/trekresearch-engine.test.mjs
// Step 1 (deep-research-engine): pin the contract the `--engine deep-research`
// adapter must hit. The adapted in-context `/deep-research` report, reduced into
// the research-brief schema, must pass research-validator under the strict
// default; and a brief missing a required section must fail. This is the one
// genuinely automatable slice of SC2 (schema, not provenance).
import { test } from 'node:test';
import { strict as assert } from 'node:assert';
import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { validateResearchContent } from '../../lib/validators/research-validator.mjs';
const HERE = dirname(fileURLToPath(import.meta.url));
const ROOT = join(HERE, '..', '..');
const FIXTURE = join(ROOT, 'tests', 'fixtures', 'research-deep-research-adapted.md');
test('deep-research adapter output contract — valid brief passes, missing section fails', () => {
const text = readFileSync(FIXTURE, 'utf-8');
// (a) positive: the adapter's target output passes the validator (default = strict).
const okResult = validateResearchContent(text);
assert.equal(okResult.valid, true, JSON.stringify(okResult.errors));
// (b) negative: stripping a required section makes it fail with RESEARCH_MISSING_SECTION,
// giving the contract teeth (a fixture that always passes proves nothing).
const mutated = text.replace('## Dimensions', '## Removed');
const badResult = validateResearchContent(mutated);
assert.equal(badResult.valid, false);
assert.ok(
badResult.errors.find(e => e.code === 'RESEARCH_MISSING_SECTION'),
'expected RESEARCH_MISSING_SECTION; got ' + JSON.stringify(badResult.errors),
);
});

View file

@ -0,0 +1,49 @@
---
type: trekresearch-brief
created: 2026-06-30
question: "Should /trekresearch delegate its external phase to the built-in /deep-research workflow?"
confidence: 0.8
dimensions: 2
mcp_servers_used: []
local_agents_used: []
external_agents_used:
- deep-research
---
# Deep-research engine adapter output
> Fixture: a `/deep-research` in-context report reduced into the research-brief
> schema by the `--engine deep-research` adapter (Step 4). Models the target the
> adapter must hit; not real engine output.
## Executive Summary
Delegating the external phase to the built-in `/deep-research` workflow is a
viable opt-in engine that supplies fan-out and cited claim-verification for free.
Confidence is medium-high on the mechanism but lower on availability, because the
workflow exposes no positive "is-enabled" probe. The load-bearing caveat is
provenance: structural validity does not certify that the cited URLs are real, so
a swarm fallback plus a human URL spot-check stay mandatory.
## Dimensions
### Engine mechanism -- Confidence: high
**External findings:**
- `/deep-research` is a built-in dynamic workflow reachable only by prose instruction, with no programmatic API (https://code.claude.com/docs/workflows).
- Its report lands in-context with no on-disk artifact, so the adapter must transform what is already in the turn (https://code.claude.com/docs/commands).
### Fallback ergonomics -- Confidence: high
**External findings:**
- There is no positive availability probe; only `disableWorkflows` / `CLAUDE_CODE_DISABLE_WORKFLOWS` off-switches and a 2.1.154 version floor are documented (https://code.claude.com/docs/skills).
- Disabled-workflow behavior under `claude -p` is undocumented, so the post-hoc presence check must be robust to every failure manifestation (https://github.com/anthropics/claude-code/issues/52272).
## Sources
| # | Source | Type | Quality | Used in |
|---|--------|------|---------|---------|
| 1 | https://code.claude.com/docs/workflows | official | high | Engine mechanism |
| 2 | https://code.claude.com/docs/commands | official | high | Engine mechanism |
| 3 | https://code.claude.com/docs/skills | official | high | Fallback ergonomics |
| 4 | https://github.com/anthropics/claude-code/issues/52272 | community | medium | Fallback ergonomics |