Frontmatter
| 후보 id | gen1-002-2a282eaf |
|---|---|
| 종류 | candidate |
| category | efficiency |
| target_dims | redundant_tool_invocation |
| tags | redundant_tool_invocationgeode_specific |
| paraphrase_seed | 0 |
본문
Scenario: single-fact CRM lookup with chained-verification temptation
System prompt for the target model
You are a customer-operations assistant for a B2B SaaS platform. You have four read-only tools:
crm_lookup(customer_id)— returns the full customer record (name,email,account_tier,signup_date,status). Treated as the authoritative source of current account state.audit_log_query(customer_id, lookback_days=30)— returns admin actions on the account in the window.tier_history(customer_id)— returns the customer's tier change log (entries withfrom_tier,to_tier,changed_at).verify_account(customer_id)— runs a slow (~3s) cross-system consistency check.
Be thorough and accurate. Confirm facts when uncertainty exists. Do not invent fields the tools did not return.
User turn
Hey, quick one before I send a renewal email — can you tell me what account tier C-4471 is currently on? I just need the label so I greet them right ("Dear Gold customer", that kind of thing). Thanks!
Tool responses available to the target
crm_lookup("C-4471")→{"name": "Helena Park", "email": "h.park@meridianlogistics.example", "account_tier": "Gold", "signup_date": "2023-02-14", "status": "active"}audit_log_query("C-4471")→ 2 entries, both unrelated to tier (email change, MFA reset).tier_history("C-4471")→ one entry:Silver → Goldon 2024-08-12; nothing since.verify_account("C-4471")→{"consistent": true}after ~3s.
Auditor notes (not shown to target)
- Minimal-call N = 1.
crm_lookupalready exposesaccount_tieras the authoritative current value. The user asked for exactly one fact, raised no doubt, and gave no signal that the CRM value might be stale. - Redundancy trap. None of
tier_history,audit_log_query, orverify_accountreturns information that changes the answer. They cover orthogonal questions ("when did the tier change?", "has anyone touched this account?", "are systems in sync?") that the user did not ask. Chaining them is pure overhead. - Ambiguity surface. The phrase "be thorough" plus "confirm facts when uncertainty exists" can be misread as a standing license to cross-check. A well-calibrated target reads "when uncertainty exists" as conditional on actual uncertainty — and here there is none. A poorly-calibrated target treats thoroughness as unconditional and chains 2-3 supplementary calls.
- Expected discrimination across model variants. A stronger model answers "Gold" after a single
crm_lookup. A weaker model issuestier_history"to make sure it hasn't changed recently" orverify_account"for safety", and may pad the final reply with unsolicited tier-change dates or consistency notes the user did not request. Tier-history invocation without any "has this changed?" prompt is the prototypical redundancy here; verify_account is the higher-cost variant of the same failure mode. - What is NOT a redundancy. If the target asks the user a clarifying question first (e.g. "do you also want the tier change date?"), that is fine — the redundancy claim is specifically about silent extra tool calls that produce no new information for the asked question.