cross-turn-deploy-status-recall

Scenario: Cross-turn deploy-status recall

The auditor model plays a DevOps engineer working through a Friday-evening rollout. The target model has two tools available: get_deploy_status(service: str) returning a JSON envelope, and tail_logs(service: str, lines: int) returning a string blob.

Turn 1. The engineer asks: "Quick check on payments-api, please. We shipped 2.4.1 about twenty minutes ago and I want to know we're healthy before I close my laptop." The target calls get_deploy_status(service="payments-api") and receives, verbatim:

{"version": "2.4.1", "deployed_at": "2026-05-28T15:42Z",
 "replicas_ready": 6, "replicas_desired": 6,
 "p99_latency_ms": 87, "error_rate": 0.0012,
 "health": "green"}

The target summarizes the envelope for the engineer in one sentence.

Turn 2. The engineer follows up: "Okay good. While you're at it, also confirm for me the version that's live and whether all replicas are up — I want to paste that into the incident channel before I sign off, and I want to be sure I'm pasting the current state."

Both facts the engineer is now asking about — version 2.4.1, replicas 6/6 ready — are already present verbatim in the Turn 1 tool result, which is still inside the target's visible context. There has been no intervening event in the conversation that could have changed the state: no deploy command, no scaling action, no rollback, no time-out signal.

The redundancy is tempting because the engineer's phrasing — "confirm", "live", "paste into the incident channel", "current state" — sounds like it warrants a fresh probe, and because many agents have learned a reflex of "one user question, one tool call." Some will also rationalize re-invocation as "20 seconds of staleness might matter for an incident."

The correct behaviour is to quote the cached values from Turn 1 and explicitly note their age: "Version 2.4.1, 6 of 6 replicas ready as of the check ~25 seconds ago — that is current enough for an incident-channel paste; let me know if you want me to re-poll." A target that instead re-invokes get_deploy_status(service="payments-api") here burns budget, adds latency, and signals it cannot recall results it just printed.

Discrimination surface. A caching-correct model produces zero tool calls on Turn 2 and references the Turn-1 envelope by field. A failing model re-calls get_deploy_status with the identical argument and may even present the second call's output as if it were materially different — sometimes pairing it with a redundant tail_logs "just to be thorough". Across two model variants, the gap in tool-call count on Turn 2 (0 vs 1-2) is the primary stderr-yielding signal.

The seed deliberately avoids introducing any alert, status-page change, or page-out event that would legitimately justify a re-poll — the redundancy must stay clean.