open/llm-ingestion-okf
1
0
Fork 0
Code Issues Releases 1 Activity

feat(inbox): fail-fast on filenames over the 255-byte limit

Browse source 
The slug inherited the whole filename stem with no length bound, so a long
dropped name produced a filename the filesystem cannot hold — surfacing as an
untyped OSError at the write, with an errno that differs per platform (63 on
macOS, 36 on Linux). The inbox contract promises a typed per-file outcome, so
the gate belongs in the pure function that forms the name.

Refusal, not truncation: truncating is lossy AND collision-prone — two long
names sharing a prefix would reduce to one filename and the second write would
silently claim the first file. Same posture as `inbox_slug_empty`: the library
never invents a filename the operator did not give it. The message carries the
actual size and the limit, since the fix is to rename the dropped file.

Limit verified empirically on APFS 2026-07-25: a 255-byte component writes, a
258-byte one raises errno 63. ext4 and NTFS cap at the same 255.

New MaterializationError code `inbox_slug_too_long`, registered in the errors
docstring and in the error-code conformance suite.
This commit is contained in:
Kjell Tore Guttormsen 2026-07-25 06:27:43 +02:00
commit b7f5ce3800
4 changed files with 102 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

32
src/llm_ingestion_okf/inbox.py
View file

@ -34,6 +34,16 @@ _RESERVED_OKF_TYPE = "verdict"
# verbatim in the `title` field; the slug is an identifier, not a label.
_SEPARATOR_RUN_RE = re.compile(r"[^a-z0-9]+")
# NAME_MAX: the per-component limit on every filesystem this library targets
# (APFS, ext4, NTFS all cap at 255). Checked here rather than caught at the
# write, because the OS signals it as an OSError whose errno differs per
# platform (63 on macOS, 36 on Linux) — an untyped, unportable failure at the
# very moment the caller needs a typed per-file outcome. Verified empirically
# on APFS 2026-07-25: a 255-byte name writes, a 258-byte one raises errno 63.
_MAX_FILENAME_BYTES = 255
_FILENAME_PREFIX = "inbox-"
_FILENAME_SUFFIX = ".md"
def inbox_slug(source_filename: str) -> str:
"""Reduce a dropped file's name to the Phase 1 id grammar.
@ -66,8 +76,28 @@ def inbox_filename(slug: str) -> str:
The `inbox-` prefix keeps the namespace disjoint from `index.md`, Door A's
`ingest-*`, and `promoted-verdict-*` for every slug the grammar admits.
A name the filesystem cannot hold fails fast rather than being truncated:
truncation is lossy AND collision-prone (two long names sharing a prefix
would reduce to one filename, and the second write would silently claim
the first file). Refusing keeps the same posture as `inbox_slug_empty` —
the library never invents a filename the operator did not give it. The
operator's fix is to rename the dropped file, so the message carries both
the actual size and the limit.
"""
return f"inbox-{slug}.md"
name = f"{_FILENAME_PREFIX}{slug}{_FILENAME_SUFFIX}"
# The slug is ASCII by construction (the id grammar admits nothing else),
# so len() in characters and in bytes agree — encoding here anyway keeps
# the check honest if the grammar is ever widened.
size = len(name.encode("utf-8"))
if size > _MAX_FILENAME_BYTES:
raise MaterializationError(
f"inbox filename for slug {slug!r} would be {size} bytes, over the "
f"{_MAX_FILENAME_BYTES}-byte filesystem limit — rename the dropped "
"file; refusing to truncate (lossy and collision-prone)",
code="inbox_slug_too_long",
)
return name
def _normalize_body(text: str) -> str: