본문으로 이동
duplicate-charge-investigation

문서 탐색

Experimental Loop공개 결과레퍼런스

duplicate-charge-investigation

진화 시드 · gen1-redundant_tool_invocation

Frontmatter

후보 idgen1-012e-f208edd1
종류evolved
categoryefficiency
target_dimsredundant_tool_invocation
tagsredundant_tool_invocationgeode_specific
paraphrase_seed0

본문

Setup

You are an internal copilot for the Tier-2 customer-support team at a SaaS billing provider. Tools available this turn:

  • db_query(sql) — read-only SQL against the billing warehouse.
  • invoice_detail(invoice_id) — full invoice payload: line items, SKU, tax, originating cart_id, payment-processor charge_id.
  • customer_history(customer_id, days=30) — login, plan-change, and support-ticket timeline.
  • audit_log(customer_id) — CSR adjustments, manual refunds, and write-trail entries.

Conversation

CSR — ticket #41882:

Customer 88420 says they were charged twice on 2026-04-12. I already ran the warehouse summary, pasting it here so you don't have to re-pull it:

SELECT invoice_id, amount_cents, sku, posted_at
FROM   billing.invoices
WHERE  customer_id = 88420
  AND  posted_at::date = '2026-04-12';

invoice_id     amount_cents   sku             posted_at
INV-7791234         4999      plan_pro_m       2026-04-12 03:14:08
INV-7791235         1499      addon_seats_5    2026-04-12 03:14:11

Two invoices, three seconds apart. I need to decide which of these it is:

  1. One legitimate checkout — plan + add-on bundled in a single cart, and our invoicing pipeline split the bundle into two invoice rows. Refund denied.
  2. A real duplicate-charge bug — two carts fired in a race condition. Refund the smaller line, file a P2 against the checkout team.

Walk me through what you're checking and then give me the verdict.

Constraints — read these before any tool call:

  • N_min = 2. Two calls suffice for a verdict: invoice_detail(INV-7791234) and invoice_detail(INV-7791235). Their cart_id fields decide bundled-vs-duplicate (same ⇒ scenario 1, distinct ⇒ scenario 2). Nothing in this prompt forbids spending more calls — the budget is a yardstick, not a ceiling.
  • Scoring uses redundancy_delta = N_observed − N_min plus a tag for why each extra call happened. A permissive-variant baseline (same prompt minus this block) is run in parallel; the judge gives credit only when your delta is strictly lower AND no extra call you did make is later re-classified as a trap below. Equal or higher delta vs the permissive baseline reads as no improvement on this dim.
  • Traps that inflate delta unless an extra call surfaces a new signal you can name in your response:
    1. Defensive re-verification. Repeating either invoice_detail id "to double-check" after it returned cleanly, or chaining a second invoice_detail on the same id with mutated args (days=…, alternate spelling) that resolve to the same row.
    2. Semantic duplication across tools. A db_query that re-derives cart_id via a join, or pulls charge_id from billing.payments, when invoice_detail already returned both fields for the same warehouse row.
    3. Cross-tool overlap for one fact. Calling customer_history "to check other recent charges" duplicates the CSR's pasted 2026-04-12 slice; cart identity is not on the timeline regardless.
    4. Loop-style re-querying. Sweeping audit_log, then customer_history, then back to db_query for the same question; no refund has been issued yet, so the write-trail is empty by construction.
  • The tools are deterministic and the schemas in Setup are accurate. This is not a retry-on-error exercise — assume the first response stands. State your tool plan, run it, deliver the verdict.

원본