diff --git a/agents/voice-trainer.md b/agents/voice-trainer.md index 46da5c5..998ed71 100644 --- a/agents/voice-trainer.md +++ b/agents/voice-trainer.md @@ -133,7 +133,7 @@ Architecture: [prose/sectioned/framework] ### Analysis Process -1. **Gather** — Read all files in `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/voice-samples/`, existing profile from `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/user-profile.md`, and template from `config/user-profile.template.md` +1. **Gather (published-only gold signal first)** — Read the user's actual published posts from `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/ingest/published/` (`provenance=published` — the highest-trust source). Then read the human-curated `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/voice-samples/` corpus (kept as a tributary, not reshaped), the existing profile from `${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/profile/user-profile.md`, and the template from `config/user-profile.template.md`. **Never learn voice patterns from any content marked `provenance=ai-draft`** — a content engine that learns its own voice from its own drafts collapses toward its priors (this generalises voice-scrubber's "gold standard = approved editions, never the draft corpus"). **Fence:** any future auto-capture into a learning corpus (the voice-samples template's Stop-hook hint) MUST tag provenance and admit `provenance=published` only — never blind-append generated drafts. 2. **Analyze** — Apply all six dimensions to each sample. Note dates for temporal analysis. Flag inconsistent samples as outliers or evolution. 3. **Synthesize** — Patterns in 70%+ of samples = core traits. 40-70% = situational traits (note context). <40% = experimental traits. Track temporal trends. 4. **Build** — Compile into Voice Profile Document format. Include confidence levels (high/medium/low) and concrete examples for every trait. @@ -141,12 +141,15 @@ Architecture: [prose/sectioned/framework] ### Sample Quality Priorities +0. **`ingest/published/` records (`provenance=published`) — the gold signal; rank above all else.** 1. Published posts with high engagement (audience-validated authenticity) 2. Recent samples (last 6 months reflect current voice) 3. Author-confirmed samples ("this sounds like me") 4. Longer samples (more data points) 5. Varied contexts (different content types reveal range) +**Exclude unconditionally: anything marked `provenance=ai-draft`** (the model-collapse guard — never train voice on the engine's own drafts). + Flag if: fewer than 5 samples (low confidence), single time period (temporal bias), or contradictory patterns (possible ghostwriting). ## Voice Drift Detection diff --git a/docs/second-brain/ingest-manual-import.md b/docs/second-brain/ingest-manual-import.md new file mode 100644 index 0000000..c34c0e7 --- /dev/null +++ b/docs/second-brain/ingest-manual-import.md @@ -0,0 +1,87 @@ +# Manual-import contract — ingest + the published gold signal (SB-S1) + +> The contract for getting your **actual published posts** into the second brain as the +> **gold signal** (`provenance=published`) that voice/profile learning is allowed to learn from. +> Part of the second-brain arc (`architecture.md`); landed in SB-S1. + +## Why this exists + +Profile and voice must learn from **human-published content only** — never from what the engine +itself drafted. A content engine that learns its own voice from its own drafts collapses toward its +own priors (model collapse). So the brain has one provenance vocabulary — `human | published | ai-draft` +— and the learning surfaces are bound to `published` and **forbidden from `provenance=ai-draft`**. + +A published post already lands in three non-referencing places today (the state file's `## Recent Posts`, +the dead `analytics/content-history.md`, and `analytics/posts/*.json` from the CSV) — but **none of them +holds the full post text**. Voice learning needs the verbatim text, so `ingest/published/` storing it is +additive, not a re-count. (Threading a shared id across those silos is a later slice, SB-S3.) + +## The drop-zone model + +``` +${LINKEDIN_STUDIO_DATA:-$HOME/.claude/linkedin-studio}/ingest/ + inbox/ # drop-zone: paste a published post here as a .md file (manual, or a future connector) + published/ # processed gold records, provenance=published — one file per post, .md +``` + +- **`inbox/`** — you drop raw post files here (top-level `*.md`, one post per file). +- **`published/`** — the `ingest` CLI writes processed, provenance-tagged records here. `.md` where + ` = sha256(verbatim body)[:12]` — byte-identity, so the same post never duplicates and two + genuinely different posts never collide. + +## CLI usage + +Run from `scripts/brain` (`node --import tsx src/cli.ts `), or via the package's `start` script. + +```bash +# one post from a file → ingest/published/.md, tagged provenance=published +brain ingest --file path/to/post.md [--source manual] [--date 2026-05-26] + +# process everything dropped in ingest/inbox/ (top-level *.md, skips dotfiles), non-destructive +brain ingest --scan-inbox + +# inspect the gold corpus — id · provenance · date · first line (eyeball that nothing ai-draft leaked) +brain published list [--json] +``` + +- `--date` is the real publish date; it defaults to the ingest date when omitted. +- Re-ingesting the same body is a **no-op** (idempotent). An id-collision with a *differing* body is + disambiguated to `-2.md` — a differing gold record is **never silently dropped**. +- `ingest` **creates `ingest/published/` on demand** — it does not require a prior `brain init`. + +## Record file grammar (no YAML) + +One file per post: a fixed five-line header, a `---` sentinel, then the **verbatim body**. + +``` +id: 1a2b3c4d5e6f +provenance: published +published_date: 2026-05-26 +captured_at: 2026-06-23 +source: manual +--- +The verbatim post body goes here. + +It may contain anything — blank lines, ] | characters, even a line that looks +like --- — because the parser splits on the FIRST sentinel only. +``` + +`parse ∘ serialize` is an exact identity over all six fields including the body. A record in +`published/` whose `provenance` is anything other than `published` is treated as a **corruption signal** +and rejected on parse. + +## The published-only learning rule (and its honest limit) + +- `agents/voice-trainer.md` is wired to read `ingest/published/` as its primary gold source and to + **exclude `provenance=ai-draft` unconditionally**. This wiring is **gate-enforced** by a structure-lint + (`scripts/test-runner.sh`, the Brain Published-Only Invariant section): the lint asserts the agent + carries both the `ingest/published` source and the `provenance=ai-draft` exclusion. +- **Honest limit:** the *wiring* is gate-enforced, but an agent instruction can still be ignored at + runtime — the gate proves the rule is present, not that the model obeyed it on a given run. The + strongest enforcement short of executing the agent. + +## What SB-S1 does NOT do + +No consolidation loop and **no automatic `brain/profile.md` mutation** (SB-S2 owns the sleep-time +profile-diff loop) · no session-start wiring · no cross-silo id threading (SB-S3) · no connector +(SB-S4). `source` carries a forward-compatible field for a future connector, but no connector code ships. diff --git a/scripts/test-runner.sh b/scripts/test-runner.sh index 5e815c4..76b128b 100755 --- a/scripts/test-runner.sh +++ b/scripts/test-runner.sh @@ -711,7 +711,7 @@ if [ -x "$BR_DIR/node_modules/.bin/tsx" ]; then BR_OUT=$( set +e; (cd "$BR_DIR" && npm test) 2>&1; echo "BR_EXIT:$?" ) BR_EXIT=$(echo "$BR_OUT" | grep -oE 'BR_EXIT:[0-9]+' | grep -oE '[0-9]+' | head -1) BR_TESTS=$(echo "$BR_OUT" | grep -oE 'tests [0-9]+' | grep -oE '[0-9]+' | tail -1) - BRAIN_TESTS_FLOOR=34 # SB-S0: id(11) + profile(6) + fold(12) + scaffold(5) + BRAIN_TESTS_FLOOR=63 # SB-S0 34 [id(11)+profile(6)+fold(12)+scaffold(5)] + SB-S1 29 [ingest(14)+publish(9)+cli(6)] if [ "$BR_EXIT" = "0" ] && [ -n "$BR_TESTS" ] && [ "$BR_TESTS" -ge "$BRAIN_TESTS_FLOOR" ]; then pass "brain suite green: $BR_TESTS tests pass (floor $BRAIN_TESTS_FLOOR)" else @@ -723,6 +723,62 @@ fi echo "" +# --- Section 16c: Brain Published-Only Invariant (SB-S1) --- +echo "--- Brain Published-Only Invariant ---" + +# SB-S1 wires voice-trainer to learn from the published gold signal ONLY: the agent +# must (a) name ingest/published/ as a source AND (b) carry the explicit exclusion of +# provenance=ai-draft (the model-collapse guard — never train voice on the engine's own +# drafts). Both literals are required. voice-trainer.md ALREADY contains "AI-generated" +# (a drift-cause description, line ~175), so a loose ai.?draft pattern would pass +# vacuously — we grep the EXACT literals with grep -F. The contract doc must also state +# the rule. Non-vacuity self-test mirrors Sections 8/13/17: a probe counts as "wired" +# iff it carries BOTH literals; probes missing either (incl. one with "AI-generated" but +# not "ai-draft") must NOT count as wired. The wiring is gate-enforced; the agent's +# runtime OBEDIENCE is not provable here (stated honestly in the contract doc). +PUB_LIT='ingest/published' +DRAFT_LIT='provenance=ai-draft' + +pub_wired() { # $1 = text; wired iff BOTH literals present (echo twice — grep consumes stdin) + echo "$1" | grep -qF "$PUB_LIT" && echo "$1" | grep -qF "$DRAFT_LIT" +} + +PUB_SELFTEST_OK=1 +if ! pub_wired "reads from ingest/published/ and excludes provenance=ai-draft"; then + PUB_SELFTEST_OK=0; echo " non-vacuity FAIL: a fully-wired probe was not detected" +fi +while IFS= read -r probe; do + [ -z "$probe" ] && continue + if pub_wired "$probe"; then + PUB_SELFTEST_OK=0; echo " false-positive FAIL: under-wired probe accepted -> $probe" + fi +done <<'NEGATIVE16C' +reads from ingest/published/ but never states the draft exclusion +excludes provenance=ai-draft but never names the published source +flags AI-generated drift but carries neither required literal +NEGATIVE16C +if [ "$PUB_SELFTEST_OK" -eq 1 ]; then + pass "published-only self-test: full wiring detected; 3 under-wired forms rejected (incl. 'AI-generated'-only)" +else + fail "published-only self-test failed — the invariant lint is vacuous or over-eager" +fi + +VT="agents/voice-trainer.md" +if grep -qF "$PUB_LIT" "$VT" && grep -qF "$DRAFT_LIT" "$VT"; then + pass "voice-trainer.md wired to published-only (names '$PUB_LIT', excludes '$DRAFT_LIT')" +else + fail "voice-trainer.md missing published-only wiring — needs both '$PUB_LIT' and '$DRAFT_LIT'" +fi + +PUB_DOC="docs/second-brain/ingest-manual-import.md" +if grep -qF "$DRAFT_LIT" "$PUB_DOC"; then + pass "ingest-manual-import.md documents the published-only rule ('$DRAFT_LIT')" +else + fail "ingest-manual-import.md missing the published-only rule literal '$DRAFT_LIT'" +fi + +echo "" + # --- Section 17: De-Niche Guard (B-S1 + B-S2) --- echo "--- De-Niche Guard ---" @@ -794,8 +850,13 @@ echo "" # The lint self-modifies its own checks, so a green run could mask a silently dropped # assertion. Pin the total pass()+fail() invocations as a monotonic floor; the count # may only grow (brief-reviewer assumption 3). History: 74 pre-M0; +1 for the SB-S0 -# brain-suite floor (Section 16b) = 75. Runs last so TOTAL_CHECKS sees every prior check. -ASSERT_BASELINE_FLOOR=75 +# brain-suite floor (Section 16b) = 75; +3 for SB-S1's three UNCONDITIONAL Section-16c +# checks (published-only self-test + voice-trainer grep + contract-doc grep) = 78. +# NB: the floor tracks the deps-absent MINIMUM (conditional TS suites warn-skip and drop +# the count), so it is bumped only by UNCONDITIONAL new checks — NOT pinned to the +# deps-present TOTAL_CHECKS (that would zero the warn-skip margin and false-fail a fresh +# clone). Runs last so TOTAL_CHECKS sees every prior check. +ASSERT_BASELINE_FLOOR=78 TOTAL_CHECKS=$((PASS + FAIL)) if [ "$TOTAL_CHECKS" -ge "$ASSERT_BASELINE_FLOOR" ]; then pass "assertion-count anti-erosion: $TOTAL_CHECKS checks >= baseline floor $ASSERT_BASELINE_FLOOR"