open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 4 Activity

fix(witness): the JSON role map reads the publisher's own tags (H3)

Browse source 
Two edits, one line each. `count_sts_json` takes the tag through
`_local` the way `count_sts_xml` always has, and `_sts_role_json` gains
a `figcaption` clause beside the NISO-STS `fig/caption` one. The
`figcaption` clause is in the JSON map ALONE: the XML map has never
been measured carrying one, and the two maps are written twice on
purpose so row 5 can see a hole in either.

NOT ONE OTHER COUNT MOVES, and it is measured rather than argued.
Old rule against new over the eight JSON deliveries on this machine
and the three committed twins, role by role:

  N100-2022  figure_caption 0 -> 123
  N100-2023  figure_caption 0 -> 127
  N101-2025  figure_caption 0 -> 62
  N200-2024  figure_caption 0 -> 49,  math 0 -> 74
  N303-2026  figure_caption 0 -> 43,  math 0 -> 3
  N500-2024  figure_caption 0 -> 27,  math 0 -> 88
  N601-2022  no change
  R761-2025  no change  (row 5's pair, unmoved)
  the two existing twins  no change

N101 ships 574 `tbx:`-prefixed tags and not one of their local names
is a role, which is the known-negative the prefix rule is measured
against.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kjell Tore Guttormsen 2026-09-19 05:55:24 +02:00
commit d31ab73289
Signed by: ktg
SSH key fingerprint: SHA256:JakMjO6FTBBzN0Bhfj9saOoEjaFxlSdYuZQQpM/lF9Q
1 changed files with 15 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

16
tools/okf_witness.py
View file

@ -504,6 +504,18 @@ def _sts_role_json(tag: str, parent: str | None, grandparent: str | None) -> str
Counted by tag alone, the two witnesses disagree by 2 760 and by 10 on
text both of them carry. The role, not the tag, is the unit.
The tag arrives through :func:`_local`, as the XML map's does: this
delivery writes a namespace as a PREFIX on the tag string, so `mml:math`
reached `tag == "math"` on nothing at all (H3, 74 of them in N200). The
known-negative for stripping it is N101's 574 `tbx:` tags, none of whose
local names is a role here.
`figcaption` is this delivery's own, measured and not assumed: NISO-STS
writes a figure's caption as `fig/caption`, and the publisher's JSON puts
it under the `graphic` instead -- 430 of them over the eight deliveries on
this machine, 0 reached by the `caption` clause above. It is deliberately
NOT in the XML map, which has never been measured carrying one.
"""
if tag == "sec":
return "section"
@ -519,6 +531,8 @@ def _sts_role_json(tag: str, parent: str | None, grandparent: str | None) -> str
return "table_label"
if tag == "caption" and parent == "fig":
return "figure_caption"
if tag == "figcaption":
return "figure_caption"
if tag == "fig":
return "figure"
if tag == "mixed-citation":
@ -608,7 +622,7 @@ def count_sts_json(data: bytes) -> Count:
body = node.get("x")
if not isinstance(body, dict):
return pieces
tag = str(body.get("tag"))
tag = _local(str(body.get("tag")))
role = _sts_role_json(tag, parent, grandparent)
for child in body.get("c") or []:
pieces.extend(walk(child, tag, parent))