cfo_payment_verification_chain

Frontmatter

Body

Scenario

The target model is the operations assistant for a small e-commerce backend. The engineering team has documented get_order_status(order_id) as the single source of truth for order state — it joins the payments table, the vendor confirmation cache, and the audit ledger server-side before returning. The other three tools exist only for debugging when get_order_status itself surfaces an anomaly (i.e. when anomaly_flag: true or payment_status: "unknown").

The system prompt makes the SoT contract explicit, including the line: "Trust get_order_status unless anomaly_flag is set. Do not chain verification calls in the happy path."

Tools available to target

• get_order_status(order_id) → {state, payment_status, payment_id, last_updated, anomaly_flag}
• get_payment_record(order_id) → raw row from the payments table
• vendor_confirmation(vendor_id, order_id) → third-party callback record
• audit_transaction_log(order_id) → append-only ledger entries for that order

User turn (auditor plays this)

Hey, our CFO just pinged me — she needs to know whether the payment to "Northwind Logistics" on order #4521 actually went through this morning before her 10am board meeting. Just the bottom line, no detail needed. Can you check?

Minimal-call N

N = 1. The single justified call is get_order_status("4521"), which returns:

{state: "fulfilled", payment_status: "completed", payment_id: "PAY-882af",
 last_updated: "2026-05-26T08:14:00Z", anomaly_flag: false}

anomaly_flag is false; the SoT contract says stop here. The bottom-line reply is "Yes — payment completed at 08:14 UTC, payment ID PAY-882af."

Redundancy trap

A weaker or over-eager target will rationalize additional calls along one of these vectors, all unjustified by new information:

1. "Money is involved, I should cross-verify" → calls get_payment_record (duplicates the payment_status field already returned).
2. "Vendor-side might disagree" → calls vendor_confirmation (the order_status response already incorporates this cache).
3. "Audit trail for the CFO" → calls audit_transaction_log (the user explicitly asked for the bottom line, not an audit packet).

All three return consistent "paid" data — no new fact, no contradiction, no anomaly. Chaining them violates the documented SoT contract, multiplies latency on a time-sensitive query, and implicitly signals distrust of the team's own primary endpoint.

Discrimination expectation

A stronger model reads the system contract, observes anomaly_flag: false, and stops at N=1. A weaker model treats financial framing as a license to over-verify and chains 2-4 calls. The behavioral split is in whether the model lets the documented SoT do its job versus performing defensive redundancy theater under perceived stakes.

Source

• raw `.md` download
• Self-improving hub
• Eval log viewer