Kjell Tore Guttormsen
585f972a05
The engine can now RETIRE a fact via an operator-gated, explicitly-signalled
temporal-update supersede — completing the consolidation motor's one S2 TODO
("no supersede in S2 — that's S3"). A `supersedes` signal on a candidate retires
the stale fact (re-minted to an archival id, status: superseded, REPLACED IN
PLACE, retained as audit) and installs the new winner under the canonical key-id,
so mintEntityId(key) always points at the live fact.
- consolidate.ts: Candidate.supersedes? + ProfileDiff.supersedes (SupersedeOp
carries the full winner fact, so applyDiff stays a pure projector). proposeDiff
value-guarded routing fork (routes only when the target holds a DIFFERENT value —
else a re-sent signal would self-supersede every run) + intra-batch
first-supersede-wins guard. applyDiff replace-in-place + value-matched state-check
(oldId present, active, value===oldValue) → idempotent re-apply + stale-diff safe;
superseded facts never bumped/promoted (supersede wins). Decay excludes superseded.
archivalId seeded with the pre-archival id (collision-free).
- cli.ts: renderDiffMd `## Supersessions (old → new)` (rendered last, only when
present → zero-supersession diffs stay byte-identical); validateCandidates optional
single-line `supersedes`; `--gather` profileFacts filtered to active (superseded
archival facts never re-presented as live context).
- Tests: +12 brain (consolidate 10 + consolidate-cli 2). TDD: RED (6 fail) → GREEN
(94/94). BRAIN_TESTS_FLOOR 82->94; ASSERT_BASELINE_FLOOR unchanged at 80 (no new
test-runner.sh section). Gate 95/0/0.
- Docs: consolidation-loop.md rule table + honest-limit reconciled (the operator
gate is the only classification net); engine docstring updated.
All 13 success criteria deterministically tested (unlike S3a, no behavioural-only
SC). READ-only gate unchanged — brain consolidate --apply --confirm stays the sole
profile.md writer. Scope held: scripts/brain/ only; temporal-update only
(condition-dependent/distractor deferred).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RigJBiRFNtFZKCz21qNbQ4
82 lines
5.5 KiB
Markdown
82 lines
5.5 KiB
Markdown
# Consolidation loop — the compounding mechanism (SB-S2)
|
|
|
|
> How the second brain turns the published gold signal into an ever-improving, drift-resistant
|
|
> `brain/profile.md` — operator-invoked, operator-gated, deterministic. Part of the second-brain arc
|
|
> (`architecture.md`); landed in SB-S2.
|
|
|
|
## The shape
|
|
|
|
The loop is **operator-invoked** ("sleep-time" = when you run it), not automatic — the session-start
|
|
hook is zero-dep and cannot run AI, so it only **nudges** when consolidation is due. The pass itself is:
|
|
|
|
```
|
|
brain consolidate --gather # 1. dump new published deltas + the current profile
|
|
→ (the session reads them and extracts a Candidate[] JSON)
|
|
brain consolidate --propose --candidates cand.json # 2. deterministic diff → brain/pending-diff.{md,json}
|
|
→ (you review brain/pending-diff.md — the [OPERATØR] gate)
|
|
brain consolidate --apply --diff brain/pending-diff.json --confirm # 3. the ONLY write to profile.md
|
|
```
|
|
|
|
`--apply` records `brain/consolidation-state.json` `{last_run}`; the session-start nudge reads it +
|
|
counts `ingest/published/*.md` to know when to nudge again. Roll back any apply via git.
|
|
|
|
## The deterministic engine (`scripts/brain/src/consolidate.ts`)
|
|
|
|
`proposeDiff({current, candidates, today, opts})` classifies each candidate (matched to existing facts
|
|
by the candidate's `key`):
|
|
|
|
| Rule | Condition | Effect |
|
|
|------|-----------|--------|
|
|
| **reject** | `provenance: ai-draft` | dropped — never learns from the engine's own drafts (model-collapse guard) |
|
|
| **add** | no matching fact, provenance `published`/`human` | new dynamic fact, `evidence_count: 1` |
|
|
| **evidence-bump** | matching fact, same value | `evidence_count++`, `last_seen = today` |
|
|
| **promote** | a dynamic fact reaches `N = 3` observations | dynamic → static |
|
|
| **conflict** | matching key, different value, NO supersede signal | **keep both**, timestamped, with DISTINCT ids; the old fact is untouched (a genuine contradiction — both views coexist) |
|
|
| **supersede** (SB-S3b) | matching key, different value, explicit `supersedes` signal on the candidate | retire the old fact (re-minted to an archival id, `status: superseded`, retained as audit) + install the new winner at the canonical key-id — a temporal update, not a contradiction |
|
|
| **decay-flag** | a dynamic **active** fact's `last_seen` > `90` days | listed in `staleFlags` (informational; never auto-removed; superseded facts are decay-exempt) |
|
|
|
|
**Id model (no duplicate ids):** a concept's primary fact id is `mintEntityId({kind:'observed', key})`;
|
|
a conflict alt fact id is `mintContentId('observed-alt:'+key+'::'+value+'::'+date)` — byte-distinct, so
|
|
two facts never share an id. The SB-S0 folded `profile-field` static seeds use a different kind, so
|
|
consolidation never mutates them (immutable in S2). `applyDiff` produces a `ProfileDoc` that round-trips
|
|
exactly through the SB-S0 grammar; re-running is idempotent (bump, not duplicate).
|
|
|
|
Defaults: `promoteThreshold = 3`, `decayDays = 90` (operator-confirmed).
|
|
|
|
## The candidate file — the session↔engine contract
|
|
|
|
`--propose --candidates <file.json>` takes a JSON **array** of candidates; each is validated
|
|
(malformed → non-zero exit, nothing written):
|
|
|
|
```json
|
|
[
|
|
{ "key": "primary-expertise", "value": "AI governance in the public sector",
|
|
"provenance": "published", "source": "published:1a2b3c4d5e6f", "observed_date": "2026-05-26" }
|
|
]
|
|
```
|
|
|
|
- `key`, `value`, `source`, `observed_date` — non-empty strings; `provenance ∈ {human, published, ai-draft}`.
|
|
- `key` and `value` must be **single-line** (no newline/CR — the profile grammar is one fact per line).
|
|
- The session produces this from `--gather`'s output. The engine guarantees the *mechanics*; the *quality*
|
|
of the candidates is the session's job (see limits).
|
|
|
|
## Honest limits
|
|
|
|
- **The loop's value depends on the session's extraction.** The engine only guarantees threshold/conflict/
|
|
decay/provenance mechanics. Garbage candidates → a garbage diff. The operator gate + candidate-shape
|
|
validation catch shape errors, not insight quality.
|
|
- **`brain/profile.md` has one reader as of SB-S3a.** S2 evolved the profile motor-only; SB-S3a wired the
|
|
first consumer — `strategy-advisor` reads it as *evidence-to-test* (anti-sycophancy: counter-pressured, never
|
|
parroted). Broader consumption (more content agents, a hook-level prompt digest) remains later S3 work, and
|
|
the profile is still mutated ONLY via `brain consolidate --apply --confirm`.
|
|
- **Supersede is explicit + operator-gated; no AUTO-demotion (SB-S3b).** A temporal update retires the old
|
|
fact ONLY when the candidate carries an explicit `supersedes` signal (the session's judgement), and only
|
|
after the operator confirms the rendered `## Supersessions` in the diff. The engine guarantees the supersede
|
|
*mechanics* (replace-in-place, archival re-mint, audit-retain); whether a contradiction is really a temporal
|
|
update vs a genuine contradiction is the session's + operator's call — **the operator gate is the only
|
|
classification net** (a mis-classified supersede silently retires a fact that should have been kept-both).
|
|
Genuine contradictions (no signal) still **keep both**; stale facts are still only *flagged*, never auto-removed.
|
|
- **No AI at session-start.** The nudge is a deterministic file-count + sidecar read; the consolidation
|
|
pass is always operator-invoked.
|
|
- **The session-start nudge is consolidation-due only** — it counts published records + days since last run;
|
|
it does not parse `profile.md` for per-fact staleness (that cost/parser is deferred).
|