feat(ingest): I3 — D7-speil av ingest (filkatalog/CSV), bygget fra commons-spec alene
Speiler MAF I2 fra shared/ingest-spec.md alene: manifest → CSV-konnektor → materialisert OKF-bundle, byte-identisk med den delte golden-fasiten. - ingest.py: ManifestContract (pydantic, fail-fast, file-kilde, verdict-reservasjon §3, id-grammatikk, max_rows), CSV-konnektor (boundary-checked fail-closed), materialisering (§5-frontmatter eksakt rekkefølge, markdown-tabell m/ escaping, LF-only, SHA-256 manifest-stamp), index-generering (§6), replacement §3/§5. - okf.py: _parse_index_entry — tolererer frontmatterløs index (method-spec §3: index rendres via body = summary, ikke som typet concept-fil). Golden var spec-konform; D7-okf var strengere enn standarden. Scoped: non-index concept- filer krever fortsatt type (honesty-test). - examples/ingest-golden-file/: repo-lokal golden (byte-frossen kopi av I2s fasit). - Speiltester (I2s load-bearing-sett, alle detach-bevist røde): golden byte-fasit + mutasjonskontroller · provenance/navigability/verdict-reservasjon/re-ingest-safety · kontrakt fail-fast/max_rows/boundary/kollisjon · spec-integritet §11. - docs/2026-07-04-I3-brief.md: brief + de to operatør-avgjorte beslutningene. Suite 239 passed uten nøkkel/nettverk (189 + 50 nye) · ruff + mypy --strict rene. [skip-docs] README + docs/extending.md er bevisst utsatt til I7 per sesjonsplan (programmet batcher ingest-doc der, avgrenset til det D7 faktisk har — CSV nå, SQL/HTTP senere). Endringen er dokumentert i docs/2026-07-04-I3-brief.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017MM6BWb1hWmJZuXFZ7rjxT
This commit is contained in:
parent
dc2345e134
commit
e03bd79876
15 changed files with 958 additions and 2 deletions
3
examples/ingest-golden-file/expected-bundle/index.md
Normal file
3
examples/ingest-golden-file/expected-bundle/index.md
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
Golden extraction case: two CSV extracts from a local project archive (file source type).
|
||||
- [Project costs](ingest-costs.md)
|
||||
- [Escaping edge cases](ingest-edge.md)
|
||||
14
examples/ingest-golden-file/expected-bundle/ingest-costs.md
Normal file
14
examples/ingest-golden-file/expected-bundle/ingest-costs.md
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
---
|
||||
type: dataset
|
||||
title: Project costs
|
||||
source_system: prosjekt-arkiv
|
||||
source_query: costs.csv
|
||||
ingested_at: 2026-07-03T12:00:00Z
|
||||
ingest_manifest: manifest@24109f486b9ef708
|
||||
generated: true
|
||||
---
|
||||
|
||||
| item | cost_nok | note |
|
||||
| --- | --- | --- |
|
||||
| led-retrofit | 120000 | ROI under 2 years |
|
||||
| hvac-tuning | 80000 | requires vendor quote |
|
||||
18
examples/ingest-golden-file/expected-bundle/ingest-edge.md
Normal file
18
examples/ingest-golden-file/expected-bundle/ingest-edge.md
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
---
|
||||
type: reference
|
||||
title: Escaping edge cases
|
||||
source_system: prosjekt-arkiv
|
||||
source_query: edge.csv
|
||||
ingested_at: 2026-07-03T12:00:00Z
|
||||
ingest_manifest: manifest@24109f486b9ef708
|
||||
generated: true
|
||||
---
|
||||
|
||||
| id | value |
|
||||
| --- | --- |
|
||||
| backslash | a\\b |
|
||||
| pipe | a\|b |
|
||||
| newline | x y |
|
||||
| both | \\\| |
|
||||
| zeros | 007 |
|
||||
| decimal | 1.50 |
|
||||
3
examples/ingest-golden-file/fixture/costs.csv
Normal file
3
examples/ingest-golden-file/fixture/costs.csv
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
item,cost_nok,note
|
||||
led-retrofit,120000,ROI under 2 years
|
||||
hvac-tuning,80000,requires vendor quote
|
||||
|
8
examples/ingest-golden-file/fixture/edge.csv
Normal file
8
examples/ingest-golden-file/fixture/edge.csv
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
id,value
|
||||
backslash,a\b
|
||||
pipe,a|b
|
||||
newline,"x
|
||||
y"
|
||||
both,\|
|
||||
zeros,007
|
||||
decimal,1.50
|
||||
|
1
examples/ingest-golden-file/ingested-at.txt
Normal file
1
examples/ingest-golden-file/ingested-at.txt
Normal file
|
|
@ -0,0 +1 @@
|
|||
2026-07-03T12:00:00Z
|
||||
21
examples/ingest-golden-file/manifest.json
Normal file
21
examples/ingest-golden-file/manifest.json
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
{
|
||||
"manifest_version": 1,
|
||||
"source": {"type": "file", "id": "prosjekt-arkiv", "root": "fixture"},
|
||||
"bundle_summary": "Golden extraction case: two CSV extracts from a local project archive (file source type).",
|
||||
"extractions": [
|
||||
{
|
||||
"id": "costs",
|
||||
"title": "Project costs",
|
||||
"query": "costs.csv",
|
||||
"okf_type": "dataset",
|
||||
"max_rows": 10
|
||||
},
|
||||
{
|
||||
"id": "edge",
|
||||
"title": "Escaping edge cases",
|
||||
"query": "edge.csv",
|
||||
"okf_type": "reference",
|
||||
"max_rows": 10
|
||||
}
|
||||
]
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue