open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

fix(gate,propose): okf build runs a real guard; a code fence declares no structure

Browse source 
Two defects reported from outside by claude-code-llm-wiki (order
20260915T202332Z-228694739), both reproduced against this tree before
anything moved.

F1 -- the packaged CLI never ran the guard. corpus.measure wired an
unconditional approve-everything stub into process_inbox and 0 of 90
add_argument calls named a gate, so the one path most people use screened
nothing while pyproject.toml made the guard a mandatory runtime dependency
and the README recommended a composition the command line could not reach.

  --gate takes guard-trusted-source (default), guard-user-upload or none.
  corpus.resolve_gate is the one name->callable map, with the guard imported
  lazily so importing the package still does not pull it in; an unknown name
  RAISES rather than falling back, because a fallback reproduces the defect
  with an extra step. The gate's NAME goes into the section 9 log.md -- a
  stub is only dangerous when nothing downstream can see it -- and --gate
  none renders NOTHING WAS SCREENED.

  The default was chosen on a measurement: over the 453 concept bodies of
  the pinned reference bundle, PRESET_TRUSTED_SOURCE persists 453 of 453 and
  PRESET_USER_UPLOAD holds 1, costing that concept's whole source document.
  Neither tier waves anything through -- an invisible carrier and a CRITICAL
  finding fail secure at both. Door B's library default is UNCHANGED at
  PRESET_USER_UPLOAD: an inbox drop is an untrusted upload, an operator
  pointing this command at their own folder is not. The second tier ships as
  guard_adapter.inbox_gate_trusted_source, the three-line adapter that
  module's docstring already described, never a preset parameter.

  process_inbox(segmentations=..., gate=inbox_gate) now has a test. Before
  this, `grep -rl inbox_gate tests/` gave 1 file with 0 occurrences of
  `segment` -- the recommended composition was untested, which is how the
  defect survived.

F2 -- a fenced code block declared structure. `# Use the opus[1m] alias`
inside a ```bash fence became a level-1 ATX heading: the document was
refused entirely where the line carried [ or ] (5 of 191 pages of the
reporter's corpus), and the concept TITLE came from somebody's shell session
on 62 of 191 (32.5 %). The fix is in the proposer and never in Door B's
title rule -- that rule is right, and a heading that was never a heading is
what has to stop being proposed. propose.fenced_lines is computed once per
text and no rule reads a fenced line, including Arm D's outline RUN, which
selects from the whole line list. Backtick and tilde fences, three leading
spaces, a closing fence at least as long as its opener, and no backtick in a
backtick fence's info string -- that last one keeps a line holding only
`okf build` from silencing a document.

MEASURED ON THE BYTES, and this is the number that decides: the 43-document
reference corpus built at b6da09c (from git archive, never the editable
tree) and rebuilt at the shipped defaults differ in log.md alone, by the one
added bullet. 865 concept files on both sides, every concept byte-identical.

Found by that control and NOT caused by this work: the pinned artifact
K2-bundle-default-20260912 was written 2026-09-09, two days before ed0418f
changed title: quoting, so it differs from what HEAD produces on 42 concept
files. test_default_bundle_pin stays green because it pins the count and the
hit@8 ranks, not the bytes. Re-pinning is the operator's call.

Suite 1896 passed / 1 skipped (+27 from 1869). ruff, ruff format and mypy
--strict clean. No version bump, no tag, no push.

Report: docs/2026-09-15-f1-f2-gaten-og-kodefencen.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-16 00:19:50 +02:00
commit 332961a19c
11 changed files with 1194 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

241
tests/test_code_fence.py Normal file
View file

@ -0,0 +1,241 @@
"""A fenced code block declares no structure (F2).
Reported from outside on 2026-09-15 by `claude-code-llm-wiki` and reproduced
here on `b6da09c` before a line moved. The proposer read every line of the
extracted text with the same grammars, so a shell comment inside a
```` ```bash ```` fence -- `# Use the opus[1m] alias` -- became a level-1 ATX
heading. Two effects, and the SMALLER one is the visible one:
- **the document is refused entirely** when the fenced line carries `[` or `]`,
because Door B validates a title fail-fast (`inbox.py`, `inbox_title_invalid`)
and never repairs one. Measured on the reporter's corpus: 5 of 191 pages.
- **the TITLE is poisoned** on every document where such a line survives
validation. Measured: 62 of 191 pages (32.5 %) carry `#` lines inside fences.
That is the larger effect and it is silent -- the bundle builds, and the
concept is filed under a line of somebody's shell session.
The fix is in the PROPOSER and not in Door B's title rule: the title rule is
right, and a heading that was never a heading is what has to stop being
proposed. A fenced block is the one construct in markdown that says "the lines
inside me are not markdown", so no rule may read them -- not ATX, not the
numbered grammar, not a table row, not a bold title, not an outline run.
**Exposure, measured on the bytes before the change:** 0 of 865 concept files
in the pinned default bundle (`K2-bundle-default-20260912`) and 0 of the
shipped fixtures and goldens that reach the proposer contain a fence of either
kind. A rule that only ever fires INSIDE a fence therefore cannot move a byte
in anything this repository has measured, which is why it lands unconditionally
rather than behind an eleventh flag: it is not a default move, it is a defect.
"""
from __future__ import annotations
from pathlib import Path
from llm_ingestion_okf import cli
from llm_ingestion_okf.propose import RULE_HEADING, find_candidates
BUNDLE_ID = "code-fence-fixture"
OKF_VERSION = "0.2"
# The reporter's minimal repro, verbatim in shape: a real heading, a real
# subheading, and one fenced shell comment carrying brackets.
REPRO = (
"# Tittel\n"
"Source: https://example.invalid/x\n"
"\n"
"Broedtekst.\n"
"\n"
"## Seksjon\n"
"\n"
"```bash\n"
"# Use the opus[1m] alias\n"
"claude --model opus\n"
"```\n"
"\n"
"Mer tekst.\n"
)
def _titles(text: str, **flags: object) -> list[str]:
return [candidate.title for candidate in find_candidates(text, **flags)] # type: ignore[arg-type]
# --- the defect ------------------------------------------------------------
def test_a_fenced_hash_line_is_not_a_heading() -> None:
"""The defect itself: the fenced comment proposed a boundary."""
titles = _titles(REPRO)
assert "Use the opus[1m] alias" not in titles
assert titles == ["Tittel", "Seksjon"]
def test_the_document_still_builds_a_bundle(tmp_path: Path) -> None:
"""End to end, on the reporter's own numbers: 0/1 substantive -> 1/1.
The proposer test above would pass against a fix that dropped the fenced
line and also broke the door; this is the one that says the document
arrives.
"""
inbox = tmp_path / "inbox"
inbox.mkdir()
(inbox / "doc.md").write_text(REPRO, encoding="utf-8", newline="")
bundle = tmp_path / "bundle"
assert (
cli.main(
[
"build",
str(inbox),
"--bundle",
str(bundle),
"--bundle-id",
BUNDLE_ID,
"--okf-version",
OKF_VERSION,
]
)
== 0
)
concepts = [
path for path in sorted(bundle.rglob("*.md")) if path.name not in {"index.md", "log.md"}
]
assert concepts, "the document was refused entirely before the fix"
def test_the_discriminating_control_still_holds() -> None:
"""The reporter's own control: brackets fell the door, the fence proposed it.
With `1m` instead of `[1m]` the document built before the fix, because the
title validated -- and the concept was still filed under a shell comment.
Both readings must end at the same place now: the line is not a heading.
"""
control = REPRO.replace("[1m]", "1m")
assert "Use the opus 1m alias" not in _titles(control)
assert _titles(control) == ["Tittel", "Seksjon"]
# --- the fence grammar -----------------------------------------------------
def test_a_tilde_fence_closes_the_same_way() -> None:
"""`~~~` is a fence in CommonMark and reaches this library the same way.
Reading only backticks would leave the same defect behind a second spelling
nothing here measures.
"""
text = "# Tittel\n\nTekst.\n\n~~~\n# ikke en overskrift\n~~~\n\nMer tekst.\n"
assert _titles(text) == ["Tittel"]
def test_a_longer_fence_is_not_closed_by_a_shorter_one() -> None:
"""A closing fence must be at least as long as the opening one.
Otherwise a four-backtick block quoting a three-backtick example closes on
the quoted line, and every line after it is read as markdown again.
"""
text = "# Tittel\n\n````\n```\n# ikke en overskrift\n```\n````\n\nTekst under.\n"
assert _titles(text) == ["Tittel"]
def test_an_unclosed_fence_runs_to_the_end_of_the_document() -> None:
"""An unterminated fence swallows the rest, which is CommonMark's own rule.
The alternative -- treating the opener as ordinary text -- would read a
truncated code listing as a document full of headings, which is the defect
in its worst form rather than a repair of it.
"""
text = "# Tittel\n\nTekst.\n\n```\n# ikke en overskrift\n## heller ikke denne\n"
assert _titles(text) == ["Tittel"]
def test_an_indented_fence_marker_still_opens_a_fence() -> None:
"""Up to three leading spaces still open a fence in CommonMark.
A nested list holding a code block is the ordinary way this appears in
technical documentation, so a column-0-only rule would miss the common case.
"""
text = "# Tittel\n\nTekst.\n\n ```\n # ikke en overskrift\n ```\n\nSlutt.\n"
assert _titles(text) == ["Tittel"]
def test_a_fence_marker_inside_prose_does_not_open_a_fence() -> None:
"""The known-negative: a line must BEGIN with the marker to be a fence.
Without this, a sentence mentioning ``` would silence every heading after
it -- a rule that removes real boundaries instead of false ones.
"""
text = "# Tittel\n\nSkriv ``` for en kodeblokk.\n\n## Seksjon\n\nTekst.\n"
assert _titles(text) == ["Tittel", "Seksjon"]
def test_the_info_string_may_not_contain_a_backtick() -> None:
"""CommonMark forbids a backtick in a backtick fence's info string.
A line of inline code alone on a line (`` `okf build` ``) otherwise reads as
an opening fence and silences the rest of the document.
"""
text = "# Tittel\n\n`okf build`\n\n## Seksjon\n\nTekst.\n"
assert _titles(text) == ["Tittel", "Seksjon"]
# --- every other rule is fenced off too ------------------------------------
def test_a_fenced_numbered_line_is_not_a_boundary() -> None:
"""The numbered grammar reads the same lines and must stop at the fence."""
text = "# Tittel\n\nTekst her.\n\n```\n1.2 Installer pakken foerst\n```\n\nSlutt.\n"
assert _titles(text) == ["Tittel"]
def test_a_fenced_numbered_line_does_not_feed_an_outline_run() -> None:
"""Arm D selects its run from the whole text, so the fence binds there too.
Filtering only at admission would leave a fenced listing deciding WHICH run
wins -- a boundary moved by lines that declare nothing.
"""
text = (
"# Tittel\n"
"\n"
"Innledende tekst om emnet.\n"
"\n"
"```\n"
"1 Installer\n"
"2 Konfigurer\n"
"3 Kjoer\n"
"```\n"
"\n"
"Avsluttende tekst.\n"
)
assert _titles(text, outline_run=3) == ["Tittel"]
def test_a_fenced_pipe_line_does_not_open_a_table_block() -> None:
"""A piped line inside a fence is shell syntax, not a table row."""
text = "# Tittel\n\nTekst her.\n\n```\nokf build | tee log\nokf check | wc -l\n```\n\nSlutt.\n"
assert _titles(text) == ["Tittel"]
def test_a_fenced_bold_line_is_not_a_bold_title() -> None:
"""Round 10's rule reads the same lines and stops at the fence too."""
text = "```\n**ikke en tittel**\n```\n\nTekst som staar under.\n"
assert _titles(text, bold_title=True) == []
# --- the exposure control --------------------------------------------------
def test_a_document_without_a_fence_is_untouched() -> None:
"""The control that says the rule fires only inside a fence.
This is the property the exposure measurement rests on: 0 of 865 pinned
concept files carry a fence, so a rule that cannot fire outside one cannot
have moved them.
"""
text = (
"# 1 Innledning\n\nDette dokumentet beskriver krav til seksjonering.\n\n"
"## 1.1 Omfang\n\nOmfanget er hele anlegget og alle systemer.\n"
)
candidates = find_candidates(text)
assert [candidate.title for candidate in candidates] == ["1 Innledning", "1.1 Omfang"]
assert all(candidate.rule == RULE_HEADING for candidate in candidates)