fix(fase1): spike B fan-out measures real conversation bleed, not a counter
/trekreview flagged the Spike B(b) fan-out experiment as BROKEN_SUCCESS_CRITERION (BLOCKER): it asserted a per-client call_count reached 3 on a reused instance vs 1 on a fresh one — a tautology true for any un-reset mutable counter, independent of MAF, that never exercised the real G2/B7 shared-Workflow state-corruption footgun. It was a false-confirm of a de-risk assumption. Rebuilt to observe genuine MAF thread state via the messages each participant RECEIVES (new FakeChatClient.received_texts seam): - shared_instance_conversation_bleed: a reused built ConcurrentBuilder Workflow accumulates the conversation across .run() calls — run N's participants receive runs 0..N-1's prompts/replies (measured [[p0],[p0,p1],[p0,p1,p2]], strictly monotonic) => genuine cross-run contamination. - fresh_instance_conversation_isolation: a fresh instance per run gives each a clean thread => each participant sees only its own project ([[p0],[p1],[p2]]). Assumption now CONFIRMED with a meaningful observable. findings-b.md gains a Method note recording why it was rebuilt; README rows updated. Also fixes the MINOR: a_groupchat.run_live now mkdirs the findings dir before write_text so a post-disposal run does not lose the measured result. Gate green: ruff check + format, mypy src, pytest 48 passed / 1 skipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fif1r1En5W542HbZV88yMH
This commit is contained in:
parent
b81e22b637
commit
a2dff210ce
6 changed files with 118 additions and 35 deletions
|
|
@ -99,6 +99,26 @@ def _word_tokens(text: str) -> int:
|
|||
return len(text.split())
|
||||
|
||||
|
||||
def message_texts(messages: Sequence[Message]) -> list[str]:
|
||||
"""Extract the text payloads from a sequence of MAF ``Message`` objects.
|
||||
|
||||
MAF content items vary (objects with ``.text``, bare strings, or ``{"text": ...}``
|
||||
dicts); this normalizes them to a flat list of strings. Used by ``FakeChatClient``
|
||||
to record exactly what an agent *received* per call — the observable Spike B uses to
|
||||
detect cross-run conversation bleed (G2/B7)."""
|
||||
out: list[str] = []
|
||||
for m in messages:
|
||||
for c in getattr(m, "contents", []) or []:
|
||||
text = getattr(c, "text", None)
|
||||
if text is None and isinstance(c, str):
|
||||
text = c
|
||||
if text is None and isinstance(c, dict):
|
||||
text = c.get("text")
|
||||
if text is not None:
|
||||
out.append(str(text))
|
||||
return out
|
||||
|
||||
|
||||
class FakeChatClient(BaseChatClient):
|
||||
"""A deterministic, network-free ``BaseChatClient`` for driving MAF agents in tests.
|
||||
|
||||
|
|
@ -116,6 +136,9 @@ class FakeChatClient(BaseChatClient):
|
|||
self._default = default_reply
|
||||
self.total_tokens = 0
|
||||
self.call_count = 0
|
||||
# One entry per call: the text payloads this client RECEIVED that call. Lets a
|
||||
# spike observe whether a reused workflow feeds run N+1 the prior runs' history.
|
||||
self.received_texts: list[list[str]] = []
|
||||
|
||||
def _next_reply(self) -> str:
|
||||
reply = self._scripted[self._idx] if self._idx < len(self._scripted) else self._default
|
||||
|
|
@ -134,6 +157,7 @@ class FakeChatClient(BaseChatClient):
|
|||
) -> Any:
|
||||
# Matches the GA BaseChatClient contract: return a ResponseStream when
|
||||
# streaming, otherwise an awaitable resolving to a ChatResponse.
|
||||
self.received_texts.append(message_texts(messages))
|
||||
reply = self._next_reply()
|
||||
|
||||
if stream:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue