Kjell Tore Guttormsen
ac532f9218
voice-trainer's Gather step now reads ingest/published/ (provenance=published) as the primary gold source, keeps voice-samples as a human tributary (not reshaped), forbids learning from provenance=ai-draft, and fences the auto-append trap. New gate Section 16c (Brain Published-Only Invariant) enforces the wiring with exact-literal greps (grep -F 'ingest/published' + 'provenance=ai-draft') + a non-vacuity self-test (rejects an 'AI-generated'-only probe), against voice-trainer.md + the new contract doc docs/second-brain/ingest-manual-import.md. BRAIN_TESTS_FLOOR 34→63; assertion floor 75→78 (+3 unconditional checks; not pinned to deps-present TOTAL to preserve the warn-skip margin). Gate 90→93/0/0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RigJBiRFNtFZKCz21qNbQ4
87 lines
4.3 KiB
Markdown
87 lines
4.3 KiB
Markdown
# Manual-import contract — ingest + the published gold signal (SB-S1)
|
|
|
|
> The contract for getting your **actual published posts** into the second brain as the
|
|
> **gold signal** (`provenance=published`) that voice/profile learning is allowed to learn from.
|
|
> Part of the second-brain arc (`architecture.md`); landed in SB-S1.
|
|
|
|
## Why this exists
|
|
|
|
Profile and voice must learn from **human-published content only** — never from what the engine
|
|
itself drafted. A content engine that learns its own voice from its own drafts collapses toward its
|
|
own priors (model collapse). So the brain has one provenance vocabulary — `human | published | ai-draft`
|
|
— and the learning surfaces are bound to `published` and **forbidden from `provenance=ai-draft`**.
|
|
|
|
A published post already lands in three non-referencing places today (the state file's `## Recent Posts`,
|
|
the dead `analytics/content-history.md`, and `analytics/posts/*.json` from the CSV) — but **none of them
|
|
holds the full post text**. Voice learning needs the verbatim text, so `ingest/published/` storing it is
|
|
additive, not a re-count. (Threading a shared id across those silos is a later slice, SB-S3.)
|
|
|
|
## The drop-zone model
|
|
|
|
```
|
|
${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/ingest/
|
|
inbox/ # drop-zone: paste a published post here as a .md file (manual, or a future connector)
|
|
published/ # processed gold records, provenance=published — one file per post, <id>.md
|
|
```
|
|
|
|
- **`inbox/`** — you drop raw post files here (top-level `*.md`, one post per file).
|
|
- **`published/`** — the `ingest` CLI writes processed, provenance-tagged records here. `<id>.md` where
|
|
`<id> = sha256(verbatim body)[:12]` — byte-identity, so the same post never duplicates and two
|
|
genuinely different posts never collide.
|
|
|
|
## CLI usage
|
|
|
|
Run from `scripts/brain` (`node --import tsx src/cli.ts <cmd>`), or via the package's `start` script.
|
|
|
|
```bash
|
|
# one post from a file → ingest/published/<id>.md, tagged provenance=published
|
|
brain ingest --file path/to/post.md [--source manual] [--date 2026-05-26]
|
|
|
|
# process everything dropped in ingest/inbox/ (top-level *.md, skips dotfiles), non-destructive
|
|
brain ingest --scan-inbox
|
|
|
|
# inspect the gold corpus — id · provenance · date · first line (eyeball that nothing ai-draft leaked)
|
|
brain published list [--json]
|
|
```
|
|
|
|
- `--date` is the real publish date; it defaults to the ingest date when omitted.
|
|
- Re-ingesting the same body is a **no-op** (idempotent). An id-collision with a *differing* body is
|
|
disambiguated to `<id>-2.md` — a differing gold record is **never silently dropped**.
|
|
- `ingest` **creates `ingest/published/` on demand** — it does not require a prior `brain init`.
|
|
|
|
## Record file grammar (no YAML)
|
|
|
|
One file per post: a fixed five-line header, a `---` sentinel, then the **verbatim body**.
|
|
|
|
```
|
|
id: 1a2b3c4d5e6f
|
|
provenance: published
|
|
published_date: 2026-05-26
|
|
captured_at: 2026-06-23
|
|
source: manual
|
|
---
|
|
The verbatim post body goes here.
|
|
|
|
It may contain anything — blank lines, ] | characters, even a line that looks
|
|
like --- — because the parser splits on the FIRST sentinel only.
|
|
```
|
|
|
|
`parse ∘ serialize` is an exact identity over all six fields including the body. A record in
|
|
`published/` whose `provenance` is anything other than `published` is treated as a **corruption signal**
|
|
and rejected on parse.
|
|
|
|
## The published-only learning rule (and its honest limit)
|
|
|
|
- `agents/voice-trainer.md` is wired to read `ingest/published/` as its primary gold source and to
|
|
**exclude `provenance=ai-draft` unconditionally**. This wiring is **gate-enforced** by a structure-lint
|
|
(`scripts/test-runner.sh`, the Brain Published-Only Invariant section): the lint asserts the agent
|
|
carries both the `ingest/published` source and the `provenance=ai-draft` exclusion.
|
|
- **Honest limit:** the *wiring* is gate-enforced, but an agent instruction can still be ignored at
|
|
runtime — the gate proves the rule is present, not that the model obeyed it on a given run. The
|
|
strongest enforcement short of executing the agent.
|
|
|
|
## What SB-S1 does NOT do
|
|
|
|
No consolidation loop and **no automatic `brain/profile.md` mutation** (SB-S2 owns the sleep-time
|
|
profile-diff loop) · no session-start wiring · no cross-silo id threading (SB-S3) · no connector
|
|
(SB-S4). `source` carries a forward-compatible field for a future connector, but no connector code ships.
|