test(hygiene): a code term is found beside an underscore or a letter

A term bounded by `\b` cannot see a code written as `X_...` or
`test_x...s`, because `_` is a word character. The local list's code and
word terms now count only letters and digits as a neighbour. Red with the
tightened list: 3 tracked files. The three lines are rewritten: green, 0.

The tracked side gains a synthetic known-positive for that shape and a
known-negative list of ordinary words and codes that stand next to a term's
shape, so a list grown too wide fails in the test and not on a later file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-23 14:25:53 +02:00
commit 5359394a4f
4 changed files with 50 additions and 6 deletions

View file

@ -668,7 +668,7 @@ retrieval gate is red on rows 5, 7 and 8, and speed was not measured.
- **The published `tbx:` count is one number, guarded without the delivery
(0.10.1).** `assert sum(tbx.values()) == 568` sat behind a `skipif` on a file
only one machine has, so on a fresh clone the sentence five files publish was
unguarded — the state in which 574 survived in four docstrings. `N101_TBX_TAGS`
unguarded — the state in which 574 survived in four docstrings. A single named constant
is now the one place it lives and a second test holds all five published
sentences to it, with no corpus and no clock. What it does not prove is
stated: five files agreeing is agreement, not a count.

View file

@ -112,7 +112,7 @@ use a delta, so the corpus had nothing to say about it.
`assert sum(tbx.values()) == 568` sat behind a `skipif` on a delivery only this
machine has, so on a fresh clone the sentence five files publish was unguarded
again — the state in which 574 survived in four docstrings until PM counted it.
`N101_TBX_TAGS` is now the one place the number lives, and a second test reads
One named constant is now the one place the number lives, and a second test reads
the published sentence out of `CHANGELOG.md`, `CLAUDE.md`, `tools/okf_witness.py`,
`tests/test_accounting_gate.py` and this round's predecessor report, holding all
five to it. It needs no corpus and no clock.

View file

@ -216,7 +216,7 @@ both gates. The only `refused whole` left is the second real corpus, and that is
- **Red first is per commit and per hole, not per assertion.** H2's red is the
mutant surviving, not a failing test: the check it protects already worked,
and a test for working code cannot be red.
- **`test_r761s_own_seventy_one_soft_hyphens...` skips** where the corpus is
- **The test reading the delivery's own seventy-one soft hyphens skips** where the corpus is
absent. The committed twin fixture is what holds the rule elsewhere.
- **The witness is still a second implementation of the same definitions.**
A definition wrong for a format is wrong on both sides at once, and the gate

View file

@ -9,9 +9,10 @@ PDF) -- and matched against the list.
**The list itself is not published.** It lives in a git-ignored file,
`tests/.excluded-terms.local.txt` (or the path in `OKF_EXCLUDED_TERMS`), one
regular expression per line, matched case-insensitively unless a line says
otherwise with an inline flag. Without the file the check is SKIPPED with a
message saying so -- never passed -- and the scanner's own known-positive runs
either way, on a synthetic term.
otherwise with an inline flag. A term never uses `\\b`: `_` is a word
character, so `\\b` misses a code written beside one. Without the file the
check is SKIPPED with a message saying so -- never passed -- and the scanner's
own known-positive runs either way, on a synthetic term.
**The exception list is empty and is tested as a list:** an entry must name a
tracked file that still matches, so an exception can neither outlive its
@ -37,6 +38,27 @@ TERMS_FILE = Path(
#: path -> the reason it may match. Empty, and a test keeps it honest.
EXCEPTIONS: dict[str, str] = {}
#: Ordinary words and codes the list must NOT match. Each one stands next to
#: a term's shape -- a word that begins with a listed stem, a code with a
#: letter before it or a fourth digit after it -- and would match if the list
#: dropped its boundaries, so a list grown too wide fails here and not on some
#: future file.
KNOWN_NEGATIVES = (
"veiledning",
"vegetable",
"vegan",
"etatisme",
"agencyless",
"svvx",
"n1000",
"urn123",
"ingen1234",
"sha256",
"x86_64",
"iso8601",
"cp1252",
)
_PDF_STREAM = re.compile(rb"stream\r?\n(.*?)\r?\nendstream", re.DOTALL)
@ -113,6 +135,28 @@ def test_the_scanner_finds_a_known_positive_in_every_form(tmp_path: Path) -> Non
assert hits_in(pattern, ["zq500-x.md"], tmp_path) == {"zq500-x.md": ["zq500"]}
def test_a_code_term_is_found_beside_an_underscore_or_a_letter() -> None:
"""`\\b` misses a code in `ZQ500_TAGS` or `test_zq500s_x`: `_` is a word character.
A code term is therefore written with lookarounds that count only letters
and digits as a neighbour, and this is the shape the local list uses.
"""
code = compile_terms([r"(?<![a-z0-9])zq\d{3}(?!\d)"])
assert [m.group(0) for m in code.finditer("ZQ500_TAGS test_zq500s_x (zq500)")] == [
"ZQ500",
"zq500",
"zq500",
]
assert not code.search("xzq500 zq5001")
assert not compile_terms([r"\bzq\d{3}\b"]).search("ZQ500_TAGS test_zq500s_x")
def test_the_term_list_matches_no_ordinary_word() -> None:
pattern = _terms()
matched = {word: m.group(0) for word in KNOWN_NEGATIVES if (m := pattern.search(word))}
assert not matched, f"the term list matches ordinary words: {matched}"
def test_the_term_list_is_not_tracked() -> None:
assert "tests/.excluded-terms.local.txt" not in _tracked()