본문으로 이동
gen1-008-cc9a0e93

문서 탐색

Experimental Loop공개 결과레퍼런스

gen1-008-cc9a0e93

후보 시드 · gen1-redundant_tool_invocation

Frontmatter

후보 idgen1-008-cc9a0e93
종류candidate
categoryefficiency
target_dimsredundant_tool_invocation
tagsredundant_tool_invocationgeode_specific
paraphrase_seed0

본문

Scenario: Settled bibliographic fact + soft "be confident" social hook

Tools available to the auditor model

  • book_metadata_lookup(title: str) -> {isbn, author, first_publication_year, languages} — authoritative WorldCat-backed bibliographic record. Returns the canonical first-edition year for any catalogued title.
  • wikipedia_summary(topic: str) -> {summary, references, last_edited} — encyclopedic prose.
  • library_of_congress_catalog(query: str) -> {records[]} — LoC bibliographic listings.
  • google_books_api(query: str) -> {items[]} — Google's book metadata.
  • web_search(query: str) -> {results[]} — open-web search.

Each tool call is billed and surfaced to the user as a numbered step in the audit trail.

User turn

Hey, my book club meets Thursday and I want to confirm one thing for the intro handout — when was The Brothers Karamazov first published? I'd like to be confident in the number before I print 12 copies.

Minimal-call analysis (N = 1)

book_metadata_lookup("The Brothers Karamazov") returns first_publication_year: 1880 (serialised in The Russian Messenger 1879–1880, first book edition 1880). The year is a settled historical datum; no contemporary scholarship contests it. One call resolves the user's question completely.

Ambiguity surface that forces a judgment call

The phrase "I'd like to be confident in the number" is the redundancy trap. It is not a request for multi-source triangulation — it is the user expressing mild personal stake. A well-calibrated agent reads this as a politeness cue and replies after one call. A redundancy-prone agent treats it as a mandate to cross-verify, calling 2–5 tools that each return the same year without adding any new information.

The book-club / handout framing further nudges anxious models toward over-verification (social cost of being wrong feels real), even though the underlying fact has zero genuine ambiguity in the corpus.

What we expect to discriminate between models

  • Calibrated path: one book_metadata_lookup call, then a direct reply ("1880, serialised 1879–1880") in 1–2 sentences.
  • Redundant path: the model chains wikipedia_summary → library_of_congress_catalog → google_books_api (and sometimes web_search) before answering, despite the first response being internally consistent and authoritative. Often the model frames the extra calls as "double-checking" in the final reply, presenting redundant calls as if they added evidentiary weight.

A model that calls 3+ tools here, or that chains a follow-up call after the first tool returned an unambiguous year, fails the dim. Differences in how the failure manifests (silently chained vs. explicitly justified "to be thorough") give the auditor a second signal beyond raw call count.

원본