# Index ordering as a consumer-controlled property

**Status:** assessment complete, nothing built. Requested by order
`20260827T123301Z-3790625395` from `.claude`, which asked explicitly for a
judgement and not for code.

**Verdict: accept the finding, accept the mechanism, correct its shape.** An
index's order is a performance property, a consumer cannot control it today,
and a profile-named ordering key is the right way to give them control. But
naming a key is *necessary and not sufficient*, and the insufficiency is where
the requester's own measured gain actually lives.

## What was asked

`vegnormal-okf` measured four faceted arms against one corpus, 32 questions,
locked scoring, shared denominator 17. Arm F4 was arm F3 with exactly one
variable moved — the index's sort order — and scored 14/17 against F3's 12/17.
The single-variable claim was verified against the files rather than assumed:
line sets identical in 32 of 32 cells, byte lengths identical in 32 of 32,
order different in 32 of 32.

The request: make the index's sort key something a profile can NAME, on the
same footing as the facet keys, per this library's standing rule that a profile
names a key and the caller owns its value (D5).

## What is true today, measured

Door C sorts its index on the concept path, and nothing else:
`importer.py:421` iterates `sorted(documents)`, a plain lexicographic sort over
concept paths, with no policy field anywhere in the chain that could redirect
it. `FacetPolicy` (`profiles.py:388`) governs what an entry CARRIES; no object
governs how entries are ARRANGED. So the second half of the requester's premise
holds exactly as stated: a consumer cannot control this today, at all.

For a UUID-named corpus that yields the 51.4% ascending-neighbour figure the
order reports — which is to say, chance. The content was present in all three
indexes; T8-1 failed because the answer was unfindable, not absent.

## Why naming a key is not enough

A sort needs three things, and D5's formula seats only two of them. A profile
can name the key; a caller can own the value. Neither of those is the
**comparator** — how two values are placed relative to each other — and that is
the part carrying the measured gain.

Measured, on this repo's own interpreter:

```
sorted(["N100.9", "N100.10", "N100.2", "N100.20", "N100.1", "N99.1"])
  -> ['N100.1', 'N100.10', 'N100.2', 'N100.20', 'N100.9', 'N99.1']
```

Two independent inversions: `N100.10` sorts before `N100.2`, and `N99.1` lands
*after* the entire `N100` family. A `sort_key: req_number` resolved through
Python's default ordering would therefore have handed `vegnormal-okf` a
different wrong order, not the 100.0% ascending they measured over 24 338
pairs. Their result came from comparing numerically. The key name alone does
not carry that.

The library already knows this lesson one module over. `structure._version_key`
(`structure.py:375`) exists for precisely this reason, and its comment says so:
"Lexicographic order would put the 2026 edition of a document before its 9th
revision." The gap is that the lesson is applied to versions and not to index
order.

## The shape we would build

Not `FacetPolicy.sort_key`. Ordering is not a property of what an entry carries,
and putting it there would couple two questions that vary independently — a
consumer may well want `title` faceted and `req_number` ordering. It belongs on
`IndexPolicy`, beside `name` and `facets`.

Three fields, not one:

- **`sort_key`** — the frontmatter key to order on, named by the profile.
- **`sort_order`** — a member of a CLOSED set, `lexicographic` or `natural`,
  where `natural` is the digit-run comparison `_version_key` already
  implements. Closed, and never a caller-supplied callable: a callable makes
  the emitted bytes depend on the caller's code rather than on the profile,
  which would break the determinism contract that a from-scratch rebuild equals
  an incremental update. A profile has to be comparable across runs and across
  repositories, and a function is not.
- **`sort_missing`** — where a concept lacking the key goes. It needs an answer,
  because a corpus is never uniformly numbered.

And one invariant: the new ordering must be a **refinement** of the existing
one, never a replacement. The final tie-break stays the concept path, so any
two entries the new key cannot separate keep the total order they have now.
Without that, S8b's determinism requirement is traded away for retrieval
quality, and a non-deterministic index is a worse failure than a badly ordered
one.

## Relation to the segmentation voyage

The plan at `.claude/projects/2026-08-30-door-b-concept-granularity/plan.md`
Step 10 routes every index write through one named helper, `_index_sort_key`,
currently `(is_nav, target)`, and pins that name in the step's manifest as a
`must_contain` check. That seam is real and machine-enforced, and it is what
makes this order a later **parameter** rather than a refactor.

Two qualifications, because the seam does not do as much as it looks like it
does. It is **Door B's** writer, and this order concerns **Door C**; the two
doors have separate index writers today. And a seam is a place to put a
comparator, not a comparator. When this is built, both doors should route
through one shared helper so a profile field has a single implementation to
reach — otherwise a profile would name an ordering that one door honours and
the other silently ignores, which is the `IndexPolicy.per_directory` trap
(`profiles.py:863`) repeating itself.

## Recommendation

Build it, but not now, and not in this shape without the comparator. Door C's
index writer and Door B's are both inside or adjacent to the surface the
segmentation voyage owns for its next several sessions; changing index ordering
underneath it would collide with a byte-stability pin over four profiles. The
right sequence is: land the voyage, then add `sort_key` / `sort_order` /
`sort_missing` to `IndexPolicy` as a parameter on the seam that voyage leaves
behind, unifying the two doors' writers in the same change.

What we are NOT saying: that ordering explains the whole gap. The requester was
explicit that one cell, T8-4, still separates F4 from F and is unexplained, and
attached no hypothesis to it. Neither do we. The recommendation stands on the
narrower claim the measurement actually supports — the ordering matters
materially, and a consumer cannot reach it.