feat(propose): a grid rule line does not close an open table block
Arm E, off by default. `find_candidates` gains a keyword-only `table_grid` whose branch is not even evaluated when it is False, so the flag-off path is byte-identical by construction rather than by test. The defect it addresses is measured. The converter emits pandoc GRID tables, whose rows are separated by `+---+---+` rule lines that `_TABLE_ROW` cannot match, so `in_table` resets between every pair of rows and ONE table becomes one concept per row group. On the K2 corpus that is 33 of Arm D's 709 entries, on exactly 3 of the 33 documents that produce a plan -- and those three are the K3 sample positions 5, 10 and 11, all three rated `too fine`. Three points where this could have gone silently wrong, and what each cost: - `Candidate` is frozen and `dataclasses.replace` is not imported, so the join is recorded as a `set[int]` over `marked` and applied at the orphan-check pass that already rebuilds every candidate. `subdivide` rebuilds them again from an explicit keyword list, so `grid` is copied there too -- exactly the trap `split` already has. - `rule_pending` is cleared on the fall-through together with `in_table` and `open_block`. A grid table ends with a bottom rule, which sets it; without the clear, the NEXT table's first row would be recorded as a join although nothing was joined. `test_two_tables_separated_by_a_blank_line_stay_two_concepts` is built to catch precisely that: its second table has ONE row group, so it cannot be joined, and the test asserts `grid is False` on it. A test that checked only candidate counts would stay green through the defect. - A rule line can never OPEN a block: it is reached only with `in_table` true. So no surviving candidate's `start` moves, bodies only grow, and the orphan check -- which is monotone in the line set -- cannot drop a candidate it previously kept. Asserted as an offset, not a length. Measured on the real corpus with this code, reproducing a prediction written down before it was built: 21 -> 6, 15 -> 3, 2 -> 1 entries, and identical at --outline-run 0 and 3, so Arm D and Arm E do not interact here. Tests first: 8 red, then green. 1235 -> 1243. ruff check: exit 0. ruff format --check: exit 0. mypy --strict src/ tools/: 27 files, Success. pytest -q: exit 0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
parent
dc67f86511
commit
7dae0d1a3a
2 changed files with 158 additions and 1 deletions
|
|
@ -1278,3 +1278,119 @@ def test_the_grid_rule_grammar_is_the_class_the_corpus_declared() -> None:
|
|||
assert grid.match(line) is not None, line
|
||||
for line in ("+\n", "++\n", "|---|---|\n", "+--- +---+\n", "---+---\n", "+abc+\n"):
|
||||
assert grid.match(line) is None, line
|
||||
|
||||
|
||||
def test_a_grid_rule_does_not_close_an_open_table_block() -> None:
|
||||
"""Both halves in ONE test, because either alone is worthless.
|
||||
|
||||
A flag-on assertion by itself stays green if the DEFAULT also moved; a
|
||||
flag-off assertion by itself is true by absence before the rule exists.
|
||||
Asserting `3` and `1` over the same text is what makes each half depend on
|
||||
the rule actually running.
|
||||
"""
|
||||
off = okf_propose_segments.find_candidates(GRID_TABLE)
|
||||
assert len(off) == 3
|
||||
on = okf_propose_segments.find_candidates(GRID_TABLE, table_grid=True)
|
||||
assert len(on) == 1
|
||||
assert on[0].title == "Tabell linje 2"
|
||||
assert on[0].start == 18
|
||||
assert on[0].end == len(GRID_TABLE)
|
||||
assert on[0].grid is True
|
||||
|
||||
|
||||
def test_a_leading_grid_rule_does_not_open_a_block() -> None:
|
||||
"""The rule line suppresses a CLOSE; it never opens.
|
||||
|
||||
Asserted as an offset rather than as a length: a span of the right size
|
||||
starting at 0 would swallow the table's top border and would still have
|
||||
"the right number of candidates". `18` is the offset of the first `|` row,
|
||||
unchanged from Arm D, which is what keeps every surviving candidate's
|
||||
`start` fixed and the orphan check monotone.
|
||||
"""
|
||||
on = okf_propose_segments.find_candidates(GRID_TABLE, table_grid=True)
|
||||
assert on[0].start == 18
|
||||
assert GRID_TABLE[on[0].start :].startswith("| Navn")
|
||||
|
||||
|
||||
def test_two_tables_separated_by_a_blank_line_stay_two_concepts() -> None:
|
||||
"""The test that catches a join whose pending state is never cleared.
|
||||
|
||||
The second table has exactly ONE row group, so it cannot be joined. If the
|
||||
bottom rule of the FIRST table leaves "a rule line is pending" set, and the
|
||||
blank line after it does not clear the flag, the second table's first row
|
||||
is marked joined although nothing was joined -- and a test that checked
|
||||
only the candidate COUNT would stay green through it.
|
||||
"""
|
||||
off = okf_propose_segments.find_candidates(TWO_TABLES_BLANK_SEPARATED)
|
||||
assert len(off) == 3
|
||||
on = okf_propose_segments.find_candidates(TWO_TABLES_BLANK_SEPARATED, table_grid=True)
|
||||
assert len(on) == 2
|
||||
assert on[0].grid is True
|
||||
assert on[1].grid is False
|
||||
|
||||
|
||||
def test_two_tables_separated_by_a_rule_line_alone_are_merged() -> None:
|
||||
"""A declared limit, measured rather than argued away.
|
||||
|
||||
Pandoc puts a blank line between two adjacent tables, so it does not emit
|
||||
this shape -- but that is a property of the WRITER, not of this code, and
|
||||
`in_table` survives an arbitrary run of rule lines. The corpus diff is what
|
||||
tests whether the shape occurs; this test states what happens if it does.
|
||||
"""
|
||||
off = okf_propose_segments.find_candidates(TWO_TABLES_RULE_SEPARATED)
|
||||
assert len(off) == 2
|
||||
on = okf_propose_segments.find_candidates(TWO_TABLES_RULE_SEPARATED, table_grid=True)
|
||||
assert len(on) == 1
|
||||
assert on[0].grid is True
|
||||
|
||||
|
||||
def test_a_pipe_table_is_unchanged_by_the_flag() -> None:
|
||||
"""A markdown pipe table has no rule line, so Arm E has nothing to say."""
|
||||
off = okf_propose_segments.find_candidates(PIPE_TABLE)
|
||||
on = okf_propose_segments.find_candidates(PIPE_TABLE, table_grid=True)
|
||||
assert len(off) == 1
|
||||
assert off == on
|
||||
assert on[0].grid is False
|
||||
|
||||
|
||||
def test_a_rule_line_outside_a_table_changes_nothing() -> None:
|
||||
"""A `+---+` in prose is not inside a table, so no block is open to keep.
|
||||
|
||||
The `len == 1` is the denominator: an equality over two empty lists would
|
||||
be true and would prove nothing.
|
||||
"""
|
||||
off = okf_propose_segments.find_candidates(PROSE_WITH_A_STRAY_RULE)
|
||||
on = okf_propose_segments.find_candidates(PROSE_WITH_A_STRAY_RULE, table_grid=True)
|
||||
assert len(off) == 1
|
||||
assert off == on
|
||||
|
||||
|
||||
def test_the_grid_rule_is_off_at_the_function_level_by_default() -> None:
|
||||
"""Default and explicit-off agree, AND the count is the pre-change one.
|
||||
|
||||
The equality alone would stay green if both had moved together; the
|
||||
`len == 3` is what makes it a guard. The known-positive control proves the
|
||||
fixture is not inert.
|
||||
"""
|
||||
default = okf_propose_segments.find_candidates(GRID_TABLE)
|
||||
explicit = okf_propose_segments.find_candidates(GRID_TABLE, table_grid=False)
|
||||
assert default == explicit
|
||||
assert len(default) == 3
|
||||
assert len(okf_propose_segments.find_candidates(GRID_TABLE, table_grid=True)) == 1
|
||||
|
||||
|
||||
def test_every_candidate_present_in_both_arms_keeps_its_start() -> None:
|
||||
"""Monotonicity: Arm E only ever DELETES marks, so bodies only grow.
|
||||
|
||||
This is what makes the orphan check unable to drop a candidate it
|
||||
previously kept -- the check is monotone in the line set, so a growing body
|
||||
can only ever become non-empty. Asserted over all three table fixtures,
|
||||
with the number of common starts printed as the denominator.
|
||||
"""
|
||||
for text in (GRID_TABLE, TWO_TABLES_BLANK_SEPARATED, TWO_TABLES_RULE_SEPARATED, PIPE_TABLE):
|
||||
off = {c.start: c for c in okf_propose_segments.find_candidates(text)}
|
||||
on = {c.start: c for c in okf_propose_segments.find_candidates(text, table_grid=True)}
|
||||
common = sorted(set(off) & set(on))
|
||||
assert common, "no common start offsets -- the comparison would be vacuous"
|
||||
for start in common:
|
||||
assert on[start].end >= off[start].end
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue