Closes the gap STATE has been carrying since e1511f9. Section 1.1's runtime
behaviour has been correct since then -- the third verdict is real, the
declared set is one constant with two uses -- but what we PUBLISHED was a
console.log summary: the right facts in a format only its author could
parse, which is most of what section 1.1 exists to prevent. Commons
shipped a shape for it in v0.3.0
(schema/conformance-declaration.schema.json 0.1.0), so the artifact can
now exist.
Every field is counted from the run rather than restated. The cases record
their own verdict as they execute, keeping `failed` (ran and disagreed)
apart from `error` (could not run) on exactly the distinction section 1
turns on. `commons_commit` is read out of the subtree-pull subject in our
own history rather than transcribed into a constant that would drift at
the next pull, and it refuses to publish a coordinate it cannot determine
-- a fabricated commit is worse than no declaration. The artifact is
gitignored: a committed declaration keeps asserting what was true once,
and nothing makes it wrong out loud when it stops being.
Validated once against the vendored schema with a real 2020-12
implementation: VALID, and the validator proven discriminating by six
negative controls it rejected (dropped zero-count, unknown key,
out-of-enum source, non-integer count, missing enumeration, malformed
case id). Continuous validation would mean a Python dependency in a suite
that has none, so what stays is the cheap half that actually drifts -- the
two key sets, asserted exactly.
TWO DEFECTS FOUND BY MUTATING THIS GATE, both in its own first draft:
1. It lived in `after()`. Measured on Node 25.8.2: an assertion that fails
in an after hook prints under "failing tests" and marks the suite red,
but leaves `fail 0` and exit code ZERO. `npm test` and CI would have
read a falsified declaration as green. The gate against "reports
success without running" was itself reporting success without running.
It is now a test, declared last, and the verdict-count assertion is
what guards the ordering that makes "last" meaningful.
2. Nothing tied the PUBLISHED `declared_tables` to the runner's constant.
Substituting a literal list left every other assertion green, because
they all read the constant rather than what was published -- so
`declaration_source: derived-from-runner` could be a lie with no code
change to point at. Now asserted identical.
Seven mutations, all exiting non-zero: dropped zero-count, falsified
not_applicable, unpublished field, hand-maintained tables, lied-about
source, and a hardcoded `passed` combined with a genuine case failure.
Published this run: 90 total, 84 passed, 0 failed, 6 not-applicable,
0 error, at commons 4641a7b (v0.3.0). Full suite 2192 pass / 0 fail /
6 skipped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017XDdiKC9ZXmcSUQ2m84s6y
org-ops flagged that STATE.md is tracked in this public repo (open/llm-security),
contradicting the current global convention (public remote => STATE.md gitignored).
The .gitignore comment claiming the opposite was stale. Stops tracking going
forward; existing history is left untouched (already public).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019GQCcuMneSfXp6xHUvYbQq
Brief + 8-step trekplan for 3 new dep-free scanners: TRG (trigger-abuse),
SIG (signature engine over the decode pipeline), AST (Python AST taint).
Adversarial review fixed 3 blockers + 7 majors on draft 1 (59/D -> 86/B).
STATE.md prepped as a self-contained cold-start for /trekexecute next session.
Un-gitignore STATE.md (global ~/.claude rule overrides old polyrepo convention);
ignore generated plan annotation HTML. No scanner code yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3s6WnubSSrFjAQTLQdVbG
Add .local/ and HANDOFF-FINDINGS.local.md to .gitignore so session
handoff artifacts (NEXT-SESSION-PROMPT.local.md, scratch findings)
do not leak into commits.
Pre-flight for Batch C v7.3.0.
Add /ultraresearch-local for structured research combining local codebase
analysis with external knowledge via parallel agent swarms. Produces research
briefs with triangulation, confidence ratings, and source quality assessment.
New command: /ultraresearch-local with modes --quick, --local, --external, --fg.
New agents: research-orchestrator (opus), docs-researcher, community-researcher,
security-researcher, contrarian-researcher, gemini-bridge (all sonnet).
New template: research-brief-template.md.
Integration: --research flag in /ultraplan-local accepts pre-built research
briefs (up to 3), enriches the interview and exploration phases. Planning
orchestrator cross-references brief findings during synthesis.
Design principle: Context Engineering — right information to right agent at
right time. Research briefs are structured artifacts in the pipeline:
ultraresearch → brief → ultraplan --research → plan → ultraexecute.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
