docs: the normalisation door, the refused column and the role map

README gains the partial-refusal paragraph and the soft-hyphen door
with its measured reach; CHANGELOG gets an Unreleased section over
this round; CLAUDE.md's open commitment "R761 is refused whole by
guard 1.4.0 ... that is the security repo's question" is replaced by
what was decided and built, with the alternatives that were felled and
the numbers behind each rule.

No version bump and no tag: `v0.10.1` is not tagged yet and releasing
is the operator's.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -5,6 +5,56 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- **One normalisation door in front of the persist gate: U+00AD is removed and
+  COUNTED** (operator decision 2026-09-18). `llm-ingestion-guard` 1.4.0 keeps
+  the soft hyphen in `_ZERO_WIDTH_CPS`, and `output:zero-width-present` is an
+  any-tier carrier — `fail_secure` at every trust level, no sanitisation, no
+  exception. R761 Prosesskoden:2025 carries 71 of them and 0 of the four real
+  zero-width characters; all 71 are Norwegian hyphenation points inside words,
+  so a 701-page process code was unreadable for the whole chain over
+  typography. `extract.normalise_extracted` removes that one character from
+  every extracted text; `ExtractedDocument.soft_hyphens`,
+  `InboxResult.normalised` and the accounting's `normalised_soft_hyphen` carry
+  the number per document and per run, and `log.md` gains a `**Normalisation**`
+  bullet. The guard is untouched, the other four characters and U+00A0 NBSP are
+  untouched, and a real zero-width character is still `fail_secure`. Reach,
+  measured: **0 of the 78** readable documents of the reference corpus carry
+  any of the six, so no bundle measured here moves.
+- **`refused` in the accounting: a partial refusal is never silent.** The
+  report and `log.md` now say how many of M documents the run persisted nothing
+  of. The exit code is unchanged — it belongs to the whole run.
+
+### Fixed
+
+- **The content-accounting gate: a document refused whole is never clean.**
+  Its elements are all booked as coded rejections, so u = 0 and d = 0, and
+  `refused_whole` asks its question only for a corpus that persisted NOTHING —
+  one refused source beside an accepted one read as clean with the content
+  gone. `Unit.refused` is that loss with its own column and the document's code
+  in the note.
+- **The STS JSON role map reads the publisher's own tags.** `count_sts_json`
+  compared the raw tag string where the XML witness has always used `_local`,
+  so `mml:math` reached `tag == "math"` on nothing — 74 formulas in N200
+  Vegbygging:2024 counted as 0. And the publisher's JSON writes a figure's
+  caption as `figcaption` under the `graphic`, not as the `fig/caption`
+  NISO-STS writes — 430 of them over the eight deliveries measured. No other
+  count moves, measured role by role over those eight and the committed twins.
+- **The mutation harness is a gate.** A surviving mutant now exits 1; the run
+  ended `2 if errors else 0`, so `killed 0 of 1` exited 0. PM's X2 mutant — a
+  report may declare a document rejected while the bundle holds it — is in the
+  set and is killed by a new test driven from both sides.
+- **The skipped-row guard measures the machine, not the argument.** Row 6 is
+  SKIPPED exactly when the corpora the arguments name are absent, so asking the
+  arguments made the branch unreachable.
+- **Row 6 says when a corpus measures no element class at all.** On N200 the
+  build proposes 0 plans and exits 2 before the accounting door, so 16 549
+  elements land as unaccounted with no declared fate — a finding about the run
+  that read as a finding about the build.
+
 ## [0.10.1] — 2026-09-18
 
 ### Added

CLAUDE.md

@@ -1034,8 +1034,70 @@ and fixtures, never code.
       column, never also `extractor_unknown`. The files come from
       `ExtractedDocument.files` (what the resolver actually resolved), never
       from byte equality, which would credit R761's 7 unpointed duplicates.
-  - **R761 is refused whole by guard 1.4.0 because of its 71 U+00AD soft
-    hyphens.** That is the security repo's question, and it has been asked.
+  - **R761 WAS refused whole by guard 1.4.0 because of its 71 U+00AD soft
+    hyphens, and since 2026-09-19 okf removes them itself.** Operator decision
+    2026-09-18, the chosen alternative verbatim: **one documented
+    normalisation door BEFORE the guard**, `extract.normalise_extracted`,
+    removing U+00AD and BOOKING the count as `normalised_soft_hyphen` per
+    document and per run, in the accounting JSON and in a `**Normalisation**`
+    bullet in `log.md`. The guard is untouched -- the two alternatives
+    (weaken the rule, ship a cleaned corpus) were felled as a weaker screen
+    for every consumer and a fix for one document. `output:zero-width-present`
+    is an ANY-TIER carrier, so the verdict was formally right at every trust
+    level and materially a false positive: the 71 are Norwegian hyphenation
+    points inside words (`ar[SHY]beider`, `bitu[SHY]men`), and the four real
+    zero-width characters (U+200B, U+200C, U+200D, U+FEFF) plus U+2060 are
+    **0 of 0** in both deliveries. Those four are NOT removed and U+00A0 NBSP
+    (6 633 in R761) is not either -- it is not in the guard's set at all.
+    Applied ONCE, at the end of `extract_document`, so `propose` and Door B
+    cannot disagree about the text a plan's `text_sha256` indexes; `_pdf_units`
+    measures its page offsets through the same door, because that table is
+    CHARACTER offsets rebuilt from the pages while the text it indexes comes
+    back normalised. Removing a character removes no newline, so every
+    line-indexed rule downstream is unmoved by construction. **Reach measured
+    rather than argued: 0 of the 78** readable documents of the K2 reference
+    corpus carry any of the six characters, 0 in the pinned bundle's concept
+    bodies and 0 across `tests/fixtures`, `examples`, `skills`, `docs`, `src`,
+    README and CHANGELOG -- the door cannot have moved a byte measured here.
+  - **A DOCUMENT REFUSED WHOLE IS NEVER CLEAN, AND A PARTIAL REFUSAL IS NEVER
+    SILENT (H1, 2026-09-19).** `refused_whole` asked its question only when a
+    corpus persisted NOTHING, and a refused document's own unit balances by
+    construction -- every element booked as a coded rejection is u = 0 and
+    d = 0. Measured by PM: one refused source beside one accepted one gave
+    `clean = 4 of 4`, `refused_whole = None`, `okf build` exit 0 and three
+    elements gone unseen, which is the ORDINARY case on a heterogeneous
+    corpus. `Unit.refused` is a fifth column and the only one that is not a
+    defect in the report; the build writes `refused` into the accounting and
+    `R of D document(s) refused whole` into `log.md`. **The exit code does not
+    move**: it belongs to the whole run, and a corpus holding one unreadable
+    file among many is ordinary -- so the order's other half, state it in the
+    accounting, is the one taken.
+  - **THE JSON ROLE MAP READS THE PUBLISHER'S OWN TAGS (H3, 2026-09-19).**
+    `count_sts_json` compared the RAW tag string where `count_sts_xml` has
+    always used `_local`, so N200 Vegbygging:2024's **74 `mml:math`** (2 811
+    characters) counted as 0; and the publisher's JSON writes a figure's
+    caption as `figcaption` under the `graphic`, not as the `fig/caption`
+    NISO-STS writes, so **49 `figcaption`** counted as 0. The `figcaption`
+    clause is in the JSON map ALONE -- the two maps are written twice on
+    purpose so row 5 can see a hole in either. NOT ONE OTHER COUNT MOVES,
+    measured role by role over the eight JSON deliveries on this machine and
+    the three committed twins: `figure_caption` 0 -> 123/127/62/49/43/27 and
+    `math` 0 -> 74/3/88, R761 and N601 unchanged, the two existing twins
+    unchanged. The known-negative for stripping a prefix is N101's **574
+    `tbx:`** tags, not one of whose local names is a role.
+  - **The mutation harness is a GATE since 2026-09-19 (H4)**: a surviving
+    mutant exits 1, where `2 if errors else 0` made `killed 0 of 1` an exit 0.
+    PM's **X2** -- "a report may declare a document rejected while the bundle
+    holds it" -- is in the set and killed by a test driven from both sides;
+    it survived 98 green tests when it was written. The skipped-row guard
+    measures the DEFAULT sources and not the arguments (H5: row 6 is SKIPPED
+    exactly when the arguments' corpora are absent, so the branch was
+    unreachable), and row 6 now SAYS when a corpus measures no element class
+    at all (H6: on N200 the build proposes 0 plans and exits 2 before the
+    accounting door, so 16 549 elements land with no declared fate -- a
+    finding about the run that read as a finding about the build; what it
+    would take is an STS reader for the JSON delivery, and `standardContent`
+    occurs 0 times in `src/`).
   - **Operator 2026-09-17:** the door stays OPT-IN until the reference
     corpus's losses are fixed, and of the three proposed exceptions only the
     PDF one (heading, paragraph, table) is APPROVED. It moves no number --

README.md

@@ -434,6 +434,32 @@ declares no heading, paragraph or table, so no witness can count them. An
 image in a workbook, or in md, txt, csv, json, odt or rtf, stays unaccounted
 and therefore stays red.
 
+A run that refused a document whole says so in both places: the accounting
+carries `refused` and each document's own `status`, and `log.md` carries
+`R of D document(s) refused whole`. The exit code does not move for it — it
+belongs to the whole run, and a corpus holding one unreadable file among many
+is ordinary — so the count is what keeps a partial refusal from being silent.
+The judge treats such a document as never clean, with its elements in their own
+`refused` column: every one of them is booked honestly as a coded rejection, so
+u and d both stay 0 and nothing else could see the loss.
+
+**The soft hyphen is removed before the persist gate, and counted** (operator
+decision 2026-09-18). U+00AD is in `llm-ingestion-guard`'s zero-width set, and
+`output:zero-width-present` is an any-tier carrier: a document carrying one is
+`fail_secure` at every trust level. Measured on R761 Prosesskoden:2025 — 71
+U+00AD, and 0 of U+200B, U+200C, U+200D, U+FEFF and U+2060 — those 71 are
+Norwegian hyphenation points inside words (`ar[SHY]beider`, `bitu[SHY]men`),
+so a 701-page process code was unreadable for the whole chain over typography.
+`extract.normalise_extracted` removes that one character from every extracted
+text and reports the count as `normalised_soft_hyphen`, per document and for
+the run, in the accounting JSON and in a `**Normalisation**` bullet in
+`log.md`. The guard is not touched and the other four characters are not
+touched: they carry no typographic job in running text, so removing one would
+be a decision about what the guard screens for, taken in the wrong repository.
+U+00A0 NBSP is not in the guard's set and is not touched either. Reach,
+measured 2026-09-19: **0 of the 78** readable documents of the reference
+corpus carry any of the six characters, so no bundle measured here moves.
+
 Two things hold with or without the flag:
 
 - `okf build` exits 1 when it extracted at least one document and persisted