feat(gate): pakke-gaten leser INNHOLDET, ikke bare filnavn + vurdering av Azure-omdøping
ORDRE 20260818T103716Z-251212929. To deler. DEL 2 - innholds-gapet (TDD, red-first MÅLT): Hver eksisterende gate i test_handover_package_loadbearing.py leser arkiv-MEDLEMSNAVN. Ingen leste hva medlemmene SIER - som er nøyaktig hvorfor ressursgruppe, ressurs, prosjekt og vertsnavn nådde en ekstern organisasjon i 14:24-bygget uten at én av 869 tester merket det. RED FIRST, mot ekte data: gate-kroppen kjørt mot den LEVERTE zip-en gir 7 funn (vertsnavnet + seks /Users-stier), mot `git archive77076b9` gir den 8. Den finner altså det som faktisk lakk, før den brukes til å påstå at HEAD er ren. Gaten er en NEKTELSE, aldri et filter: den fjerner ingenting fra arkivet, den sier at treet ikke er leveringsklart. Pakka forblir `git archive HEAD` - kø-(p) intakt. Mønstrene bor i ÉN liste, hver rad med sin egen kjent-positive prøve, og hver prøve er BYGGET VED KONKATENERING så fila ikke matcher seg selv (verifisert: 0 funn i egen kilde - ellers hadde eneste fiks vært et hull i gaten der en hemmelighet kan gjemme seg). Aksept-lista er selv gatet: en oppføring som ikke lenger nås er drift og felles. MÅLT, seks mutasjoner - fem røde, én uten diskriminerende kraft: - detach scanneren -> 1 rød (leaked-kontrollen) - aksept-lista sluker ekte vert -> 2 røde - ødelegg vertsnavn-regexen -> 2 røde - foreldet aksept-oppføring -> 1 rød (minimalitets-kontrollen) - koordinat tilbake i HEAD -> 1 rød, gaten ALENE - fjern nevner-asserten -> 0 røde (kontroll, ikke søm - uttalt) Nevner: 325 medlemmer lest, 1 hoppet over (sqlite-binær). 873 passed / 5 skipped. ÆRLIGHETS-GRENSE, uttalt i koden: gaten fanger STRUKTUR. Vertsnavnet har en form; ressursgruppe og prosjekt er fri tekst uten form, og ble i august bare oppdaget fordi de sto i samme tabell som verten. Å lukke det gapet krever en navneliste - den andre kopien av eksponeringsregelen, som er dét pakkas `git archive HEAD`-form finnes for å forby. DEL 1 - vurdering av omdøping (ingenting rørt i Azure): docs/2026-08-18-vurdering-azure-omdoeping.md. Anbefaling: IKKE døp om. Lekkasjen ga MÅLRETTING, ikke tilgang, og målrettingen kan ikke trekkes tilbake - navnene ligger i publisert historikk og i en zip hos en tredjepart. Omdøping finnes dessuten ikke som operasjon: et custom subdomain KAN IKKE endres (Learn), så det er riving + gjenoppbygging i sju steg. Det ene tiltaket som faktisk fjerner en autorisasjonsvei Entra ikke dekker er `disableLocalAuth` + nøkkelregenerering. Beslutningen er operatørens; valgene står med konsekvenser, ikke som konklusjon. PREMISS KORRIGERT (Verifiseringsloven ansikt 3): ordren sa koordinatene sto i repoet og at tre /Users/ktg-stier lå på open/main. Målt på6d2837f: 0 og 0 -241b50dog6d2837flukket begge. Premisset var sant da ordren ble skrevet (10:37Z) og sluttet å være det 13:03/13:20. Ingen begrunnet aksept-oppføring var derfor nødvendig for sti-klassen. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01964PUr46mfnxWtMw23AnVD
This commit is contained in:
parent
6d2837fdca
commit
4cf8c4f6ba
2 changed files with 404 additions and 0 deletions
|
|
@ -36,6 +36,7 @@ than in ours:
|
|||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
|
@ -221,3 +222,207 @@ def test_receiver_documents_start_the_service_as_a_python_process(package: zipfi
|
|||
f"{label} instructs a container/azd build step that this package no longer ships: "
|
||||
f"{offenders}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The content gap (ordre 20260818T103716Z). Every gate above reads archive MEMBER
|
||||
# NAMES. None reads what the members SAY -- which is exactly how a Foundry resource
|
||||
# group, resource, project and host name reached an external organisation in the
|
||||
# 14:24 build on 14.08 without one of 869 tests noticing. The archive is
|
||||
# ``git archive HEAD`` by construction, so a content gate here is a REFUSAL, never a
|
||||
# filter: it never removes anything from the archive, it says the tree is not
|
||||
# deliverable. That keeps the kø-(p) rule intact -- there is still exactly one copy
|
||||
# of "what a receiver gets", and it is HEAD.
|
||||
#
|
||||
# One list, never patterns spread through the code. Each row carries its own
|
||||
# known-positive SAMPLE, so a pattern that can no longer find anything cannot be
|
||||
# added: ``test_every_content_pattern_can_find`` runs the whole list against its own
|
||||
# samples. Every sample is BUILT BY CONCATENATION so this file's own source bytes
|
||||
# contain no literal match -- otherwise the gate would be red on the file that
|
||||
# defines it, and the only sane fix would be a hole in the gate at exactly the place
|
||||
# a secret could hide.
|
||||
#
|
||||
# HONESTY LIMIT, stated because it decided the design: this gate catches STRUCTURE, so it
|
||||
# catches the host (`<label>.services.ai.azure.com`) and misses the resource group and the
|
||||
# project name, which are free-form strings with no shape to match. In the 14.08 leak those
|
||||
# two were only spotted because they sat in the same table as the host. Closing that gap
|
||||
# would take a name list -- the second copy of the exposure rule, free to drift, which this
|
||||
# package's `git archive HEAD` shape exists to forbid. A structural gate that catches the one
|
||||
# recoverable-by-anyone coordinate is worth more than a name list nobody prunes.
|
||||
#
|
||||
# And it is DELAYED BY ONE COMMIT: `git archive HEAD` reads HEAD, not the working tree, so a
|
||||
# coordinate added in an uncommitted edit is invisible here until it is committed (funn 35,
|
||||
# already true of every other assertion in this file).
|
||||
_GUID_ZERO = "-".join(("0" * 8, "0" * 4, "0" * 4, "0" * 4, "0" * 12))
|
||||
|
||||
_SECRET_CONTENT_PATTERNS: tuple[tuple[str, re.Pattern[str], str], ...] = (
|
||||
(
|
||||
# A tenant's own Foundry/OpenAI host. The label must START with an alphanumeric,
|
||||
# which is what keeps the repo's placeholder (`<resource>.`) and wildcard
|
||||
# (`*.services.ai.azure.com`) forms out: neither `>` nor `*` is a label character.
|
||||
"azure-ai-host",
|
||||
re.compile(
|
||||
r"[A-Za-z0-9][A-Za-z0-9-]*\.(?:services\.ai|openai|cognitiveservices)\.azure\.com"
|
||||
),
|
||||
"https://" + "sample" + ".services.ai.azure.com/api/projects/p",
|
||||
),
|
||||
(
|
||||
# Subscription id in its structural context. A bare GUID is NOT a coordinate --
|
||||
# `53ca6127-db72-4b80-b1b0-d745d6d5456d` is Azure's PUBLIC built-in role definition
|
||||
# id for Foundry User, identical in every tenant, and it is quoted in DEPLOY.md.
|
||||
"arm-subscription-scope",
|
||||
re.compile(r"/subscriptions/[0-9a-fA-F]{8}-[0-9a-fA-F-]{27}"),
|
||||
"/subscriptions/" + _GUID_ZERO + "/resourceGroups/rg",
|
||||
),
|
||||
(
|
||||
# An absolute path into somebody's home directory: a machine layout, and usually a
|
||||
# username with it. Zero on HEAD since 6d2837f; six lines in the delivered 14:24 zip.
|
||||
"absolute-home-path",
|
||||
re.compile(r"/(?:Users|home)/[A-Za-z0-9._-]+/"),
|
||||
"/Users/" + "someone" + "/repos/thing",
|
||||
),
|
||||
(
|
||||
"bearer-jwt",
|
||||
re.compile(r"ey" + r"J[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}\.[A-Za-z0-9_-]{8,}"),
|
||||
"ey" + "J" + "abcdefghij.klmnopqrst.uvwxyz0123",
|
||||
),
|
||||
(
|
||||
"vendor-api-key",
|
||||
re.compile(r"\b(?:sk-[A-Za-z0-9]{20,}|gh[pousr]_[A-Za-z0-9]{20,}|AKIA[0-9A-Z]{16})\b"),
|
||||
"sk-" + "A" * 24,
|
||||
),
|
||||
(
|
||||
# Foundry/Cognitive Services keys are 32 hex characters. Git hashes are 7 or 40, and
|
||||
# a 32-run inside a 40-char hash is excluded by the boundary look-arounds.
|
||||
"cognitive-services-key",
|
||||
re.compile(r"(?<![0-9a-fA-F])[0-9a-fA-F]{32}(?![0-9a-fA-F])"),
|
||||
"a" * 32,
|
||||
),
|
||||
(
|
||||
"private-key-block",
|
||||
re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH )?" + "PRIVATE KEY-----"),
|
||||
"-----BEGIN " + "PRIVATE KEY-----",
|
||||
),
|
||||
)
|
||||
|
||||
# Matched text that is deliberately NOT a coordinate. Every entry needs a reason, and every
|
||||
# entry is checked for being STILL REACHED (``test_accepted_content_literals_stay_minimal``)
|
||||
# -- an allowlist nobody prunes is the second copy of the exposure rule, free to drift, which
|
||||
# is the very thing this package's "git archive HEAD" shape exists to forbid.
|
||||
_ACCEPTED_CONTENT_LITERALS: tuple[tuple[str, str], ...] = (
|
||||
("x.services.ai.azure.com", "preflight/backends test dummy: one-letter label, not a resource"),
|
||||
("platform.services.ai.azure.com", "hosted-backend test dummy for the platform-injected value"),
|
||||
("x.openai.azure.com", "preflight test dummy for the WRONG Azure surface"),
|
||||
("wrong.openai.azure.com", "preflight test dummy for the WRONG Azure surface"),
|
||||
)
|
||||
|
||||
# The commit the external organisation actually received (the 14:24 build of 14.08). It is the
|
||||
# known-positive control on REAL data: the scanner has to find what did leak, not only what a
|
||||
# synthetic sample can be made to contain.
|
||||
_LEAKED_COMMIT = "77076b9"
|
||||
|
||||
|
||||
def _scan_text(text: str) -> list[tuple[str, str]]:
|
||||
"""Return ``(pattern label, matched text)`` for everything in ``text`` that is not on the
|
||||
accepted list. Detection only -- nothing is rewritten, nothing is removed."""
|
||||
findings: list[tuple[str, str]] = []
|
||||
for label, pattern, _sample in _SECRET_CONTENT_PATTERNS:
|
||||
for match in pattern.finditer(text):
|
||||
hit = match.group(0)
|
||||
if any(hit in accepted for accepted, _why in _ACCEPTED_CONTENT_LITERALS):
|
||||
continue
|
||||
findings.append((label, hit))
|
||||
return findings
|
||||
|
||||
|
||||
def _scan_archive(archive: zipfile.ZipFile) -> tuple[list[str], int, int]:
|
||||
"""Scan every member of ``archive``. Returns ``(findings, files read, files skipped)``.
|
||||
|
||||
The denominator is returned, not discarded: "nothing found" over an unknown number of files
|
||||
is not zero, it is unmeasured."""
|
||||
findings: list[str] = []
|
||||
read = skipped = 0
|
||||
for name in archive.namelist():
|
||||
if name.endswith("/"):
|
||||
continue
|
||||
try:
|
||||
text = archive.read(name).decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
skipped += 1
|
||||
continue
|
||||
read += 1
|
||||
findings.extend(f"{name}: {label}: {hit}" for label, hit in _scan_text(text))
|
||||
return findings, read, skipped
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def leaked_package(tmp_path_factory: pytest.TempPathFactory) -> zipfile.ZipFile:
|
||||
"""The archive as it was at the commit that reached an external organisation."""
|
||||
dest = tmp_path_factory.mktemp("leaked") / "leaked.zip"
|
||||
subprocess.run(
|
||||
["git", "archive", "--format=zip", "--output", str(dest), _LEAKED_COMMIT],
|
||||
cwd=_REPO_ROOT,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
return zipfile.ZipFile(dest)
|
||||
|
||||
|
||||
def test_every_content_pattern_can_find(package: zipfile.ZipFile) -> None:
|
||||
"""Control: each row of the pattern list is proven able to match before any of them is
|
||||
trusted to report nothing. A pattern that silently matches nothing makes the gate below
|
||||
green forever -- this repo's recurring vacuity class."""
|
||||
for label, pattern, sample in _SECRET_CONTENT_PATTERNS:
|
||||
assert pattern.search(sample), f"pattern {label!r} does not match its own sample"
|
||||
|
||||
# And the samples must not be literal in this file, or the gate would be red on itself and
|
||||
# the only fix would be excluding the file that defines the patterns.
|
||||
own_source = package.read("tests/test_handover_package_loadbearing.py").decode("utf-8")
|
||||
assert not _scan_text(own_source), "the pattern samples are literal in this file's source"
|
||||
|
||||
|
||||
def test_content_scan_finds_what_actually_leaked(leaked_package: zipfile.ZipFile) -> None:
|
||||
"""Control on REAL data: point the scanner at the tree the external organisation received
|
||||
and it must report the Foundry coordinates and the absolute home paths. Without this the
|
||||
gate below would only prove that HEAD is clean, never that the scanner can see."""
|
||||
findings, read, _skipped = _scan_archive(leaked_package)
|
||||
assert read > 300, f"only {read} members read from the leaked archive -- control is vacuous"
|
||||
|
||||
labels = {finding.split(": ")[1] for finding in findings}
|
||||
assert "azure-ai-host" in labels, f"the leaked Foundry host was not found; got {findings}"
|
||||
assert "absolute-home-path" in labels, f"the leaked home paths were not found; got {findings}"
|
||||
|
||||
|
||||
def test_package_leaks_no_secret_content(package: zipfile.ZipFile) -> None:
|
||||
"""The gate. Detach point: put a tenant coordinate, an ARM subscription scope, an absolute
|
||||
home path or a credential back into any tracked file -> RED.
|
||||
|
||||
Refusal, not filtering: what fails here is the TREE, and the fix is to change HEAD."""
|
||||
findings, read, skipped = _scan_archive(package)
|
||||
assert read > 300, f"only {read} members read -- this gate would be reporting on nothing"
|
||||
assert skipped <= 1, f"unexpectedly many undecodable members ({skipped}) went unscanned"
|
||||
assert not findings, (
|
||||
f"the handover package leaks secret content ({read} members read, {skipped} skipped): "
|
||||
f"{findings}"
|
||||
)
|
||||
|
||||
|
||||
def test_accepted_content_literals_stay_minimal(package: zipfile.ZipFile) -> None:
|
||||
"""Every accepted literal must still be reachable in the archive. An entry that matches
|
||||
nothing is a standing exemption for a string the repo no longer has -- the second copy of
|
||||
the exposure rule, quietly drifting away from what is actually shipped."""
|
||||
corpus = "\n".join(
|
||||
archive_text
|
||||
for name in package.namelist()
|
||||
if not name.endswith("/")
|
||||
for archive_text in _decoded(package, name)
|
||||
)
|
||||
stale = [literal for literal, _why in _ACCEPTED_CONTENT_LITERALS if literal not in corpus]
|
||||
assert not stale, f"accepted content literals no longer occur in the package: {stale}"
|
||||
|
||||
|
||||
def _decoded(archive: zipfile.ZipFile, name: str) -> list[str]:
|
||||
try:
|
||||
return [archive.read(name).decode("utf-8")]
|
||||
except UnicodeDecodeError:
|
||||
return []
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue