Tool Use and Function Calling: Agent Architecture

From single-turn chatbot to multi-step agent

A chatbot returns a text answer to the user's question. An agent, on the other hand: understands the question, thinks about which tools it needs to use, calls the tools, interprets the results, calls other tools if needed, and presents the final answer when the loop is complete. The difference is planning and action.

Since 2023, the concept of "agent" has become popular, but building a production-ready agent is still not easy. In this article, I lay out, as a practical framework, how the tool use protocol works, JSON schema design, multi-step orchestration, error handling, and cost control.

How the tool use protocol works

The paradigms of the three major LLM providers are the same, their names differ:

• Anthropic Claude: tools parameter, tool_use and tool_result content blocks
• OpenAI: tools parameter, function_call and tool_response messages (unified with GPT-5)
• Google Gemini: tools.functionDeclarations, functionCall and functionResponse

A typical Claude tool definition:

{
  "name": "search_pubmed",
  "description": "PubMed veritabaninda biyomedikal makale arama yapar. Anahtar kelime, yil araligi ve maksimum sonuc sayisi kabul eder. Sonuc olarak PMID listesi, baslik ve ozet doner.",
  "input_schema": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "PubMed query string, boolean operatorler desteklenir (AND, OR, NOT)"
      },
      "year_from": {
        "type": "integer",
        "description": "Baslangic yili, default 2020"
      },
      "max_results": {
        "type": "integer",
        "description": "Maksimum sonuc sayisi, default 20, ust sinir 100"
      }
    },
    "required": ["query"]
  }
}

When the model sees this definition, and the user asks "knee osteoarthritis AI segmentation since 2024," it produces a tool_use like this:

{
  "type": "tool_use",
  "id": "toolu_abc123",
  "name": "search_pubmed",
  "input": {
    "query": "(knee osteoarthritis) AND (artificial intelligence) AND (segmentation)",
    "year_from": 2024,
    "max_results": 20
  }
}

Your application receives this call, calls the real PubMed API, and feeds the result back to the model as a tool_result:

{
  "type": "tool_result",
  "tool_use_id": "toolu_abc123",
  "content": "[{\"pmid\": \"38123456\", \"title\": \"...\", \"abstract\": \"...\"}, ...]"
}

The model reads the result and produces an answer for the user or calls another tool.

JSON schema discipline

80% of tool use success depends on schema design. Three rules:

1. The description must be extremely clear. The model reads only the definition to select a function. The name "search_pubmed" is not enough; the description states what it does, what it returns, and when it should be used.

Bad: "description": "Searches PubMed" Good: "description": "Searches PubMed for biomedical articles. Use this when the user asks about medical research, clinical studies, or wants to find publications. Returns PMID, title, abstract, journal, authors and publication date. Does NOT return full text. Maximum 100 results per query. For full text retrieval, use get_full_text_article instead."

2. Parameter types must be strict. Enum instead of string, specific schema instead of a generic object, optional vs required made clear.

"modality": {
  "type": "string",
  "enum": ["CT", "MR", "X-RAY", "US"],
  "description": "Goruntuleme modalitesi"
}

vs

"modality": {
  "type": "string",
  "description": "Goruntuleme turu"
}

The first makes the model produce the correct value more often. The second is open to variations like "Computed Tomography," "ct," "CT scan."

3. Required fields must be kept to a minimum. If there are too many required fields, the model hesitates to call the function or produces empty values. Make fields that have a default optional, and mark only those that are truly necessary as required.

How tool selection is steered

Two techniques to help the model select the right tool:

System prompt directive:

You have access to the following tools. Use them as follows:
- search_pubmed: First step when user asks about medical literature
- get_full_text_article: After identifying relevant articles, retrieve full text
- summarize_article: After retrieving full text, generate a summary
- create_systematic_review_table: Final step, organize summaries into PRISMA-style table

Always plan multi-step tasks before executing. Use tools in sequence.
Do not retrieve full text for more than 10 articles in a single conversation.

Tool choice parameter: The tool_choice option in Anthropic and OpenAI:

• auto: the model decides freely
• any: it must use a tool (it chooses which one)
• tool: {name: "X"}: a specific tool is forced to be called

Use auto in the first turns, and force a specific tool at critical decision points.

Multi-step orchestration patterns

Three fundamental agent patterns:

1. ReAct (Reasoning + Acting): The model produces thought + action + observation at each step. The classic loop defined in the Yao 2022 paper:

Thought: Kullanici diz osteoartritinde AI segmentasyonu sistematik derleme istiyor. Once literatur taramam lazim.
Action: search_pubmed(query="...", year_from=2020)
Observation: 47 sonuc dondu.
Thought: 47 sonuc fazla, ilk 10'unu detayli incelemeliyim.
Action: get_full_text_article(pmid="38123456")
...

2. Plan-and-Execute: The model first produces the entire plan, then applies it in order. Fewer LLM calls, lower cost. But if the plan is bad, the entire execution is wasted.

Plan:
1. PubMed search ile son 5 yillik makaleleri bul
2. PRISMA flow icin tarama-eleme yap
3. Dahil edilen makalelerin full text'ini al
4. Her birinin metodolojik kalitesini degerlendir
5. Veri cikarimi yap
6. Final tablo olustur

Executing step 1...

3. Tree of Thoughts (multi-branch exploration): The model tries more than one path at each step and selects the best. It requires complex planning, cost is high. Rarely necessary in production.

Practical recommendation: Plan-and-Execute for simple tasks. ReAct for uncertain / dynamic tasks. Tree of Thoughts in a research context, not in production.

Error handling

What errors can occur during tool execution, and how they are handled:

1. Tool execution failure. API down, timeout, rate limit. An error message is written to tool_result:

{
  "type": "tool_result",
  "tool_use_id": "toolu_abc123",
  "is_error": true,
  "content": "PubMed API timeout after 30s. Retry suggested."
}

In this case the model may go the route of retry, selecting an alternative tool, or informing the user.

2. Schema validation failure. The model forgot a required parameter or produced its type incorrectly. It is caught on the application side with a JSON schema validator (Ajv, Pydantic) and fed back to the model as "input invalid, please retry."

3. Hallucinated tool. The model tries to call an undefined tool ("I am now calling summarize_with_grok()"). The application rejects the unknown tool and notifies the model.

4. Infinite loop. The model calls the same tool over and over, with no progress. A max iteration guard (for example, 20 turns) is mandatory. When the limit is exceeded, end the conversation and give the user a partial result.

5. PII data leakage. If there is sensitive data in the tool result (national ID number, patient name), this data enters the model context and can leak to other services in subsequent tool calls. PII filtering or redaction is mandatory at the tool layer.

Cost optimization

Multi-step agents can be 5-20x more expensive than a single-turn LLM call. Three optimizations:

1. Tool definitions cache. With Anthropic prompt caching, tool definitions (usually 2-5 thousand tokens) are cached. Multiple turns use the same cache.

2. Tool result truncation. If 100 articles return from PubMed, sending all 100 articles in full to the model is unnecessary. The first 10 titles, and a summary of the remaining 90 (they can be requested later with pagination).

3. Model tiering. Planning and tool selection with a large model (Claude Opus 4.7, GPT-5). Simple summarization with a small model (Claude Haiku 4.5, GPT-4o-mini). A router pattern within a single agent.

4. Early stopping. If the result of the first 3 tool calls is enough to answer the user's question, the model should say "enough." Be clear in the system prompt: "Use minimum number of tool calls. Stop as soon as you have sufficient information."

Production checklist

Before shipping a tool-use agent to production:

• Do the tool schemas pass a JSON Schema validator
• A unit test for each tool (mocked LLM call)
• Integration test (real LLM, real tool)
• Is the max iteration guard set (default 20)
• Tool execution timeout (default 30s, 60-120s for critical tools)
• Rate limiting (max tool calls per minute per user)
• Error tracking (Sentry, Datadog) for tool failure rate
• Cost monitoring (token + tool execution cost)
• PII filter on tool results
• Audit log (who called which tool, with which arguments)
• Fallback path (escalate to a human if a tool is down)

Real-world agent examples

Academic research assistant:

• Tools: search_pubmed, get_full_text, extract_data, generate_prisma_diagram, format_vancouver_references
• Workflow: Question → search literature → full text for relevant ones → extract data → PRISMA + reference list
• Cost: ~1-3 USD per use (15-30 min of human work)

Enterprise customer support agent:

• Tools: search_kb, lookup_order, create_ticket, send_email, escalate_to_human
• Workflow: Customer question → search knowledge base → order_lookup if specific customer info is needed → ticket if unresolved
• Cost: ~0.05-0.20 USD per conversation

Data analysis agent:

• Tools: query_database, run_python, plot_chart, summarize_findings
• Workflow: Analysis question → generate SQL → get data → Python analysis → visualize → summary
• Cost: ~0.50-2 USD per analysis

Defining tools with MCP

Anthropic's MCP (Model Context Protocol) standardizes tool definitions. An MCP server is an interface that provides tools, prompts, and resources. Claude Code, Claude Desktop, and custom applications can connect to the same MCP server.

Its advantage: write the tool definition once, call it from many clients. The Anthropic, Google, and OpenAI ecosystems adopted MCP throughout 2025-2026.

A simple MCP server typically offers this structure:

• tools/list: available tools
• tools/call: run a tool
• resources/list: accessible resources (for example, documents)
• prompts/list: ready-made prompt templates

An enterprise AI architecture can standardize tools over MCP servers and connect Claude, GPT, and Gemini agents to the same backend.

Practical core checklist

To build a tool-use agent soundly:

• A clear and detailed description for each tool
• Required parameters minimal, optional ones with defaults
• Use enums, avoid generic strings
• Tool usage guidance in the system prompt
• A clear choice between Plan-and-Execute or ReAct pattern
• Max iteration guard
• Tool timeout
• Schema validator on tool input
• Error handling for each tool (retry, fallback, escalate)
• PII filter on tool results
• Cost monitoring
• Audit log
• A/B testing when adding a new tool

Three common mistakes

Mistake 1: Too many tools, too little description. Defining 30 tools and giving each a 1-sentence description. The model gets lost and selects the wrong tool. 10 well-defined tools > 50 vague tools.

Mistake 2: No error recovery. A tool fails, the model crashes. In production, the failure mode of each tool must be considered, and the model should be given "retry / alternative / give up" options.

Mistake 3: Cost blind spot. Building a multi-step agent and not setting up cost monitoring. When you expect 50 USD in the first week, you see a 5000 USD bill. Track token + tool execution cost every turn.

Serteser Danismanlik support for tool use and agent architecture

Production-ready agent design requires LLM knowledge + system architecture + sector domain expertise. Serteser Danismanlik provides support in the following areas:

• Tool use schema architecture (Anthropic, OpenAI, Gemini)
• MCP server design
• Multi-step agent pattern selection (ReAct vs Plan-Execute)
• RAG + agent hybrid systems
• Error handling, retry mechanism, fallback path
• Cost optimization, model tiering
• PII / KVKK compliant tool layer
• Audit log and monitoring setup
• Production deployment and A/B testing

With clinical research experience published in an international peer-reviewed journal and active systematic review management, and with a research infrastructure that brings together the two disciplines of technical system architecture + research methodology, we provide end-to-end support for agent and tool use projects.

For one-on-one support in prompt engineering, you can take a look at the individual mentoring option.