feat(consume): the door's link line is out of the default reading

`link_in_signal` defaults to `False` on all three entry points --
`searchable_text`, `concept_scores` and `build_payload` -- through one named
constant, `DEFAULT_LINK_IN_SIGNAL`. This carries out K3-23's recommendation
(alternative c), and its numbers are the whole warrant: of the newcomers the
door's `Enclosing section:` line ever added a question token to, 39 of 39
gained it from the bundle-absolute PATH and 0 of 39 from the link's title, and
every token the path contributed is a segment of the document's own directory
-- the saturation `shared_id_prefix` takes out of the id signal, back in
through the body.

A CONSTANT rather than three naked literals, chosen against the module: the
other eight defaults here are constants (`DEFAULT_PROFILE`, `DEFAULT_LIMIT`,
`DEFAULT_TIE_SHARED_RANK`, `DEFAULT_STEM_PREFIX`, `DEFAULT_TITLE_COVERED`,
`DEFAULT_SOURCE_QUOTA`, `DEFAULT_FOLLOW_PARENT`, `DEFAULT_K`) and this was the
one default spelled out three times. It also lets a test fall on the rule from
the module that owns it.

NO CLI flag is added and none is changed. `grep -n '"--.*link'` over `src/`
gives 0 hits before and after, and `consume.main` never passed the parameter,
so the CLI inherits the function default. A flag was considered and NOT added,
because no caller could be named that needs the older reading: the three call
sites outside `consume.py` (`skill.py:360`, `tools/okf_consume_measure.py:155`
and `:247`) all inherit it deliberately.

Measured before landing: 0 of 5 shipped bundles carry the door's line
(denominators 446, 1 133, 270, 2 756, 2 761 concept files) and 5 of 5 payloads
are byte-identical across the move. The consumer repository was read only and
its tree is unchanged, listing identical including mtimes.

The docstrings move with the code rather than after it. `concept_scores` said
this was "an INSTRUMENT, not a flag and not a default" and `build_payload` that
it "carries no CLI flag"; the first is now false and the second still true, and
each says so. `cli.DEFAULT_SHELL_PARENT` keeps its value and loses its reason:
the cost it cited -- delivered sets moving through that path -- is what this
commit removes, so the comment now states why it stays off NOW, which is N = 1
document and a consumption half nobody has judged. README and CLAUDE.md carry
the same sentence where they already describe the line, measured rather than
written on reflex: neither said anything about the ranking before.

One test changed with the code. The characterisation of what the LINE does
called `concept_scores` with no parameter, because the reading it characterises
used to be the default; it now names that reading. Same fixture, same number.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-12 18:56:53 +02:00
commit 38320cd509
6 changed files with 130 additions and 47 deletions

View file

@ -28,9 +28,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
form -- so a human line opening with the same two words, and the door's exact form -- so a human line opening with the same two words, and the door's exact
form anywhere but last, both keep every character they have. form anywhere but last, both keep every character they have.
`--shell-parent` therefore stays OFF at the current link form, and reading `--shell-parent` therefore stays OFF at the current link form, and reading
the body without the line is recommended as `consume`'s default in a later the body without the line was recommended as `consume`'s default -- carried
round: it is byte-identical on **5 of 5** bundles anyone ships today, none of out in the same unreleased block below: it is byte-identical on **5 of 5**
which carries the line. bundles anyone ships today, none of which carries the line.
### Changed
- **`consume` no longer scores the door's `Enclosing section:` line (K3-25).**
`link_in_signal` defaults to `False` (`consume.DEFAULT_LINK_IN_SIGNAL`) on
all three entry points -- `searchable_text`, `concept_scores` and
`build_payload` -- so the ranking and the stem vocabulary read a heading-only
body without the line the door appends under `--shell-parent`. The excerpt
still carries it and no CLI flag changed, so what moves is ORDER and never an
excerpt's bytes. Carrying out K3-23's recommendation with its numbers: of the
newcomers that line ever added a question token to, **39 of 39** gained it
from the bundle-absolute PATH and **0 of 39** from the link's title, every
such token being a segment of the document's own directory; under this
reading a flagged bundle delivers what the unflagged build delivers on **16
of 16** rows (list, order and `spent`), with hit@1/8/50 **6/6** at both `k`
and the known-positive at rank **1**. The cost to anyone shipping a bundle
today is zero bytes: **0 of 5** bundles carry the line, so **5 of 5** payloads
are byte-identical across the move. A bundle that does carry it -- only
`okf build --shell-parent` writes it, and that default is unchanged and still
OFF -- gets a different delivered order; `link_in_signal=True` is still
reachable for the older reading.
### Fixed ### Fixed

View file

@ -638,6 +638,20 @@ and fixtures, never code.
absolute form), appended AFTER structure derivation -- read as body text it absolute form), appended AFTER structure derivation -- read as body text it
became a second, unresolved `references` edge -- and screened on its own. became a second, unresolved `references` edge -- and screened on its own.
`okf check`'s seventeenth rule, `parent_unfollowable`, holds the form. `okf check`'s seventeenth rule, `parent_unfollowable`, holds the form.
**Since K3-25 the RANKING does not score that line**
(`consume.DEFAULT_LINK_IN_SIGNAL = False` on `searchable_text`,
`concept_scores` and `build_payload`; no CLI flag, `True` still reachable).
The line reaches the file and the excerpt exactly as before, so what moved is
ORDER and never an excerpt's bytes. Measured: of the newcomers it ever added
a question token to, **39 of 39** gained it from the bundle-absolute PATH and
**0 of 39** from the link's TITLE, every such token being a segment of the
document's own directory -- the saturation `shared_id_prefix` took OUT of the
id signal, back in through the body. Under this reading a `--shell-parent`
bundle delivers what the unflagged build delivers on **16 of 16** rows (list,
order and `spent`), hit@1/8/50 **6/6** at both `k` with the known-positive at
rank 1. Exposure today is zero: **0 of 5** shipped bundles carry the line, so
**5 of 5** payloads are byte-identical across the move, and the day a bundle
carries it is the day the path would have started costing rank instead.
`okf consume --follow-parent` (K3-21 B) is the second form: `parent` also `okf consume --follow-parent` (K3-21 B) is the second form: `parent` also
carries the enclosing concept's `text` with that concept's own `sha256`, carries the enclosing concept's `text` with that concept's own `sha256`,
placed AFTER the cut from the room it left, in rank order, so the delivered placed AFTER the cut from the room it left, in rank order, so the delivered

View file

@ -189,7 +189,16 @@ heading-only body gains one line, `Enclosing section: [title](/path)`, in the
bundle-relative form SPEC § 6.1 recommends. `okf consume --follow-parent` bundle-relative form SPEC § 6.1 recommends. `okf consume --follow-parent`
carries the enclosing section's text inside `parent` as well, with that carries the enclosing section's text inside `parent` as well, with that
concept's own `sha256`, and only from the room the cut left — so it never concept's own `sha256`, and only from the room the cut left — so it never
displaces an excerpt. Whether either default moves is measured separately. The index resolves the pointer too: a segment id is looked up among the displaces an excerpt. Whether either default moves is measured separately.
**The ranking does not score that line (K3-25).** It reaches the excerpt and
the file exactly as before; `consume.DEFAULT_LINK_IN_SIGNAL` is `False`, so the
body signal and the stem vocabulary read a heading-only body without it. The
measurement: of the newcomers the line ever added a question token to, **39 of
39** gained it from the bundle-absolute PATH and **0 of 39** from the link's
title, every such token being a segment of the document's own directory — so a
bundle built with `--shell-parent` now delivers what the unflagged build
delivers, on **16 of 16** rows. It cost no shipped bundle a byte: **0 of 5**
carry the line, so **5 of 5** payloads are byte-identical across the move. The index resolves the pointer too: a segment id is looked up among the
concepts of its own document, so it renders `parent: p1` when the concept it concepts of its own document, so it renders `parent: p1` when the concept it
names is in the bundle and keeps the `?` only when nothing answers to it. names is in the bundle and keeps the `?` only when nothing answers to it.

View file

@ -176,17 +176,25 @@ DEFAULT_CLOSE_SPAN_GAPS = True
#: naming the nearest ancestor that holds text (`propose._link_shells`) and, #: naming the nearest ancestor that holds text (`propose._link_shells`) and,
#: since K3-21, one `Enclosing section:` link line in its body. NOT a #: since K3-21, one `Enclosing section:` link line in its body. NOT a
#: segmentation rule: it adds no boundary and copies no text, and the concept #: segmentation rule: it adds no boundary and copies no text, and the concept
#: count is unchanged. OFF, re-measured in K3-21 once `okf consume` read the #: count is unchanged. It moves the bytes of a bundle holding a heading-only
#: key -- the reason it was off before. On the one standard with shells hit@1, #: section and of nothing else: 0 of 453 K2 bodies are heading-only, and the
#: hit@8 and hit@50 stay 6/6 at both k with the known-positive at rank 1, but #: five-document folder builds byte-identical with it.
#: the link's bundle-absolute path repeats the document's directory in 675 #:
#: bodies, and the delivered set moved on 2 of 8 questions at the default k #: OFF -- and the reason it was off has been REMOVED rather than restated.
#: and 3 of 8 at k 50, newly delivered shells matching the question ONLY #: K3-21 measured hit@1/8/50 at 6/6 at both k with the known-positive at rank
#: through that path on 1 of 3, 4 of 7 and 6 of 23. That is the saturation #: 1, but the delivered SET moved, because the link's bundle-absolute path
#: K3-20 took out of the id signal, re-entering through the body. It moves the #: repeats the document's own directory in 675 bodies -- the saturation K3-20
#: bytes of a bundle holding a heading-only section and of nothing else: 0 of #: took out of the id signal, re-entering through the body. K3-25 made
#: 453 K2 bodies are heading-only, and the five-document folder builds #: `consume.DEFAULT_LINK_IN_SIGNAL` `False`, so that path is no longer scored,
#: byte-identical with it. #: and K3-23 measured what the acceptance then reads on the one standard with
#: shells: newcomers matching through the path 0, delivered sets moved 0 of 8
#: at both k, hit@k unchanged at 6/6.
#:
#: It stays OFF for what that measurement does NOT cover: **N = 1 document**.
#: Three of nothing is still one bundle from one publisher, the consumption
#: half is a judged reading nobody has run, and a build default is a promise to
#: every consumer and not to this one. Turning it on is its own decision with
#: its own denominator, and this line is not it.
DEFAULT_SHELL_PARENT = False DEFAULT_SHELL_PARENT = False
#: Round 9. Clause 1 asked whether a title ENDED in a number, which is a #: Round 9. Clause 1 asked whether a title ENDED in a number, which is a
#: question about the number rather than about the title: a drawing's dimension #: question about the number rather than about the title: a drawing's dimension

View file

@ -945,15 +945,18 @@ _LINK_LINE = re.compile(rf"^{re.escape(ENCLOSING_SECTION)}: \[[^\]]*\]\(/[^)]*\)
def body_without_link_line(body: str) -> str: def body_without_link_line(body: str) -> str:
"""`body` minus the one line the door appends to a heading-only body. """`body` minus the one line the door appends to a heading-only body.
THE FILE KEEPS THE LINE AND SO DOES THE READER. This is what a caller uses THE FILE KEEPS THE LINE AND SO DOES THE READER. Since K3-25 this is the
to score a body the way it read before `--shell-parent` existed, on the same DEFAULT path through the ranker (`DEFAULT_LINK_IN_SIGNAL`), on the same
bytes, in the same process -- never a second build compared against a first. bytes and in the same process -- never a second build compared against a
first. `link_in_signal=True` is how a caller asks for the older reading.
Three conditions, all the door's own, and each one is a known-negative that Three conditions, all the door's own, and each one is a known-negative that
would otherwise be stripped: the line is LAST, a blank line stands before would otherwise be stripped: the line is LAST, a blank line stands before
it, and it matches the door's form (`_LINK_LINE`). A human sentence opening it, and it matches the door's form (`_LINK_LINE`). A human sentence opening
with the same two words has no link and survives; the door's exact form with with the same two words has no link and survives; the door's exact form with
prose under it is not last and survives. What cannot be told apart is a prose under it is not last and survives. Those two are now on the default
path rather than behind a parameter, which is what makes them load-bearing
rather than illustrative. What cannot be told apart is a
human line that is byte-for-byte the door's own, last in a body -- an human line that is byte-for-byte the door's own, last in a body -- an
ambiguity of the form itself, stated rather than hidden. ambiguity of the form itself, stated rather than hidden.
""" """
@ -963,7 +966,28 @@ def body_without_link_line(body: str) -> str:
return "\n".join(lines[:-2]) + "\n" return "\n".join(lines[:-2]) + "\n"
def searchable_text(concepts: Sequence["Concept"], *, link_in_signal: bool = True) -> list[str]: #: Round 25 (K3-25). Whether the DOOR's `Enclosing section:` line is scored.
#: `False` since K3-25, on K3-23's decomposition: of the newcomers that line
#: ever added a question token to, **39 of 39** gained it from the
#: bundle-absolute PATH and **0 of 39** from the link's title, and every token
#: the path contributed is a segment of the document's own directory -- the
#: saturation `shared_id_prefix` takes OUT of the id signal, arriving back
#: through the body. Under this reading a `--shell-parent` bundle delivers
#: exactly what the unflagged build delivers: the same list, the same order and
#: the same `spent` on **16 of 16** rows, with hit@1/8/50 **6/6** and the
#: known-positive at rank 1 under every reading. It moves no byte of any bundle
#: shipped today -- **0 of 5** carry the line, so **5 of 5** payloads are
#: byte-identical -- and the day one does, it is the day the path would have
#: started costing rank instead. A constant rather than three naked literals
#: because the other eight defaults in this module are constants, and because a
#: test can then fall on the rule from the module that owns it.
#: `docs/2026-09-12-k3-runde23-stien-i-kroppssignalet.md` SS 4.
DEFAULT_LINK_IN_SIGNAL = False
def searchable_text(
concepts: Sequence["Concept"], *, link_in_signal: bool = DEFAULT_LINK_IN_SIGNAL
) -> list[str]:
"""The text a concept is scored against, one string per concept. """The text a concept is scored against, one string per concept.
The same two fields the ranker's two lexical signals read -- title plus The same two fields the ranker's two lexical signals read -- title plus
@ -971,10 +995,11 @@ def searchable_text(concepts: Sequence["Concept"], *, link_in_signal: bool = Tru
a hit can be scored on. Counting rarity over one field and matching on a hit can be scored on. Counting rarity over one field and matching on
another would weight a token by how rare it is somewhere it is not read. another would weight a token by how rare it is somewhere it is not read.
`link_in_signal=False` reads the bodies without the door's link line, for `link_in_signal` reads the bodies with or without the door's link line, and
the same reason: the stem vocabulary and the rarity `df` are counted over the same reason binds it here: the stem vocabulary and the rarity `df` are
what a hit is scored on, so an instrument that took the line out of the counted over what a hit is scored on, so a reading that took the line out of
signal and left it in the vocabulary would measure two different texts. the signal and left it in the vocabulary would measure two different texts.
Since K3-25 the default is `False` (`DEFAULT_LINK_IN_SIGNAL`).
""" """
return [ return [
f"{concept.title} {concept.concept_id.replace('/', ' ')} " f"{concept.title} {concept.concept_id.replace('/', ' ')} "
@ -1403,7 +1428,7 @@ def concept_scores(
tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK, tie_shared_rank: bool = DEFAULT_TIE_SHARED_RANK,
title_covered: bool = DEFAULT_TITLE_COVERED, title_covered: bool = DEFAULT_TITLE_COVERED,
stems: frozenset[str] | None = None, stems: frozenset[str] | None = None,
link_in_signal: bool = True, link_in_signal: bool = DEFAULT_LINK_IN_SIGNAL,
) -> list[tuple[Concept, float, int]]: ) -> list[tuple[Concept, float, int]]:
"""Every concept, ordered best first, fused from three signals by RRF. """Every concept, ordered best first, fused from three signals by RRF.
@ -1448,16 +1473,18 @@ def concept_scores(
only `tunnel` and `vann` whose ids sorted earlier. With shared ranks it only `tunnel` and `vann` whose ids sorted earlier. With shared ranks it
fuses to rank 3. fuses to rank 3.
**`link_in_signal=False` is an INSTRUMENT, not a flag and not a default.** **`link_in_signal=False` is the DEFAULT reading since K3-25**
The body this reads is the file's, and `inbox._link_enclosing` puts one line (`DEFAULT_LINK_IN_SIGNAL`), and it was K3-23's instrument before that. The
in it -- `Enclosing section: [<title>](/<bundle-absolute path>)` -- whose body this reads is the file's, and `inbox._link_enclosing` puts one line in
path repeats the document's own directory in every linked body. SS 6.1's it -- `Enclosing section: [<title>](/<bundle-absolute path>)` -- whose path
argument for the absolute form is about the FILE; nothing decided it should repeats the document's own directory in every linked body. SS 6.1's argument
be SCORED. Passing `False` scores the body without that line, on the same for the absolute form is about the FILE; nothing ever decided it should be
bytes and in the same process, so the two readings can be measured against SCORED, and K3-23 measured what happens when it is: **39 of 39** newcomers
each other rather than two builds. The excerpt a reader gets is untouched the line added a question token to gained it through the PATH and **0 of
either way; no caller in the run path passes it and the CLI does not expose 39** through the link's title. The excerpt a reader gets is untouched either
it. way, so what moved here is ORDER and never bytes. `True` is still reachable
and still does the older thing; the CLI exposes neither value, because no
caller has been named that needs the older one.
The rule takes the FIRST position of a score group rather than its middle. The rule takes the FIRST position of a score group rather than its middle.
Both were measured on the same four cases; the middle put the same concept Both were measured on the same four cases; the middle put the same concept
@ -2024,7 +2051,7 @@ def build_payload(
stem_prefix: bool = DEFAULT_STEM_PREFIX, stem_prefix: bool = DEFAULT_STEM_PREFIX,
source_quota: int | None = DEFAULT_SOURCE_QUOTA, source_quota: int | None = DEFAULT_SOURCE_QUOTA,
follow_parent: bool = DEFAULT_FOLLOW_PARENT, follow_parent: bool = DEFAULT_FOLLOW_PARENT,
link_in_signal: bool = True, link_in_signal: bool = DEFAULT_LINK_IN_SIGNAL,
) -> dict[str, object]: ) -> dict[str, object]:
"""One bundle plus one question, cut to one contract-conformant payload. """One bundle plus one question, cut to one contract-conformant payload.
@ -2049,12 +2076,14 @@ def build_payload(
Whether the naming is worth the bookkeeping is the caller's call, and the Whether the naming is worth the bookkeeping is the caller's call, and the
flag is how it stays one. flag is how it stays one.
**`link_in_signal` is an instrument and carries no CLI flag** -- see **`link_in_signal` defaults to `False` since K3-25 and carries no CLI
:func:`concept_scores`. `False` keeps the door's `Enclosing section:` line flag** -- see :func:`concept_scores`. It keeps the door's `Enclosing
out of the ranking and out of the stem vocabulary; the excerpt still carries section:` line out of the ranking and out of the stem vocabulary; the
it, so a payload built either way holds the same bytes per excerpt and only excerpt still carries it, so a payload built either way holds the same bytes
the ORDER can differ. That is what separates a ranking movement from a per excerpt and only the ORDER can differ. That is what separates a ranking
budget displacement on one bundle. movement from a budget displacement, and it is why the default moved without
a bundle changing: **0 of 5** bundles anyone ships carries the line, so **5
of 5** payloads are byte-identical across the move.
""" """
case, expected, measured = known_positive() case, expected, measured = known_positive()
if expected != measured: if expected != measured:

View file

@ -107,12 +107,14 @@ def _lexical(root: Path, *, link_in_signal: bool) -> dict[str, int]:
# --- What the line does to the signal today ----------------------------------- # --- What the line does to the signal today -----------------------------------
def test_the_doors_line_is_the_shells_only_lexical_hit_today(tmp_path: Path) -> None: def test_the_doors_line_is_the_shells_only_lexical_hit(tmp_path: Path) -> None:
"""CHARACTERISATION, green before the instrument exists. """CHARACTERISATION of what the LINE does, not of what the default is.
The shell's own text answers nothing in the question, and it still scores -- The shell's own text answers nothing in the question, and it still scores --
on the line the door wrote. It takes no new parameter to show, which is why on the line the door wrote. Until K3-25 this was asked with no parameter,
it is asked without one: the mechanism is HEAD's behaviour, not the fix's. because the reading it characterises was the default; it now names that
reading, because the mechanism is the line's and the default has moved away
from it. The same fixture, the same number, one word of spelling.
""" """
root = _bundle(tmp_path / "bundle") root = _bundle(tmp_path / "bundle")
concepts = [ concepts = [
@ -122,7 +124,7 @@ def test_the_doors_line_is_the_shells_only_lexical_hit_today(tmp_path: Path) ->
for concept_id in okf_consume.enumerate_concepts(root) for concept_id in okf_consume.enumerate_concepts(root)
] ]
ranked = okf_consume.concept_scores( ranked = okf_consume.concept_scores(
concepts, QUESTION, okf_consume.document_scores(root, QUESTION) concepts, QUESTION, okf_consume.document_scores(root, QUESTION), link_in_signal=True
) )
scored = {concept.concept_id: lexical for concept, _, lexical in ranked} scored = {concept.concept_id: lexical for concept, _, lexical in ranked}
assert scored["r761/skall"] > 0 assert scored["r761/skall"] > 0