Chapter 4: Function Calling and Tool Use

Introduction

Function Calling (also called Tool Use) is a capability that allows LLMs to interact with external systems, APIs, and tools. This transforms LLMs from text generators into powerful agents capable of taking real actions.

By the end of this chapter, you will be able to:

• Define tools using JSON Schema for LLM function calling
• Implement complete function calling workflows
• Design tools that follow MCP (Model Context Protocol) standards
• Handle errors gracefully in tool-using applications
• Orchestrate multiple tools for complex tasks

Function Calling Basics

What is Function Calling?

Function Calling enables LLMs to:

1. Understand available tools and their capabilities
2. Decide when to use a tool based on the user's request
3. Generate properly formatted arguments for the tool
4. Incorporate tool results into their response

Tool Definition with JSON Schema

Tools are defined using JSON Schema, which describes the function name, description, and parameters:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather for a specific location. Returns temperature, conditions, and humidity.",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "City name, e.g., 'Tokyo', 'New York', 'London'"
        },
        "unit": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"],
          "description": "Temperature unit (default: celsius)"
        }
      },
      "required": ["location"]
    }
  }
}

Complete Function Calling Example

from openai import OpenAI
import json

client = OpenAI()

# Define the tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "City name"
                    }
                },
                "required": ["location"]
            }
        }
    }
]

# Actual function implementation
def get_weather(location: str) -> dict:
    # In production, call a weather API
    return {
        "location": location,
        "temperature": 22,
        "unit": "celsius",
        "condition": "sunny"
    }

# Step 1: Send user message with tools
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools
)

# Step 2: Check if model wants to use a tool
message = response.choices[0].message

if message.tool_calls:
    # Step 3: Execute the tool
    tool_call = message.tool_calls[0]
    function_name = tool_call.function.name
    arguments = json.loads(tool_call.function.arguments)

    # Call the actual function
    result = get_weather(**arguments)

    # Step 4: Send tool result back to model
    final_response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "user", "content": "What's the weather in Tokyo?"},
            message,  # Assistant's tool call message
            {
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result)
            }
        ]
    )

    print(final_response.choices[0].message.content)
    # "The weather in Tokyo is currently 22C and sunny."

Tool Schema Design

Writing Effective Descriptions

The description field is crucial - it helps the LLM decide when to use the tool:

{
  "name": "search",
  "description": "Search function"
}

{
  "name": "search_knowledge_base",
  "description": "Search the company's internal knowledge base for policies, procedures, and documentation. Use this when users ask about company rules, HR policies, technical documentation, or internal processes. Returns relevant document snippets with source links."
}

Parameter Design Best Practices

Complex Tool Example

{
  "type": "function",
  "function": {
    "name": "query_database",
    "description": "Execute a read-only SQL query against the analytics database. Supports SELECT statements only. Use for retrieving customer data, sales metrics, and usage statistics.",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "SQL SELECT query. Tables available: customers, orders, products, usage_logs. Example: 'SELECT * FROM customers WHERE signup_date > '2024-01-01''"
        },
        "limit": {
          "type": "integer",
          "description": "Maximum rows to return (default: 100, max: 1000)",
          "default": 100,
          "maximum": 1000
        },
        "format": {
          "type": "string",
          "enum": ["json", "csv", "markdown"],
          "description": "Output format for results",
          "default": "json"
        }
      },
      "required": ["query"]
    }
  }
}

Model Context Protocol (MCP)

MCP is an open standard (developed by Anthropic) for connecting LLMs to external data sources and tools. It provides a unified interface that works across different LLM providers and applications.

MCP Architecture

MCP Concepts

Building an MCP Server

from mcp.server import Server
from mcp.types import Tool, TextContent
import mcp.server.stdio

# Create server instance
server = Server("weather-server")

# Define tools
@server.list_tools()
async def list_tools():
    return [
        Tool(
            name="get_weather",
            description="Get current weather for a city",
            inputSchema={
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "City name"
                    }
                },
                "required": ["city"]
            }
        ),
        Tool(
            name="get_forecast",
            description="Get 5-day weather forecast",
            inputSchema={
                "type": "object",
                "properties": {
                    "city": {"type": "string"},
                    "days": {"type": "integer", "default": 5}
                },
                "required": ["city"]
            }
        )
    ]

# Implement tool execution
@server.call_tool()
async def call_tool(name: str, arguments: dict):
    if name == "get_weather":
        city = arguments["city"]
        # Call actual weather API here
        weather_data = fetch_weather(city)
        return [TextContent(
            type="text",
            text=f"Weather in {city}: {weather_data['temp']}C, {weather_data['condition']}"
        )]

    elif name == "get_forecast":
        city = arguments["city"]
        days = arguments.get("days", 5)
        forecast = fetch_forecast(city, days)
        return [TextContent(type="text", text=format_forecast(forecast))]

# Run the server
async def main():
    async with mcp.server.stdio.stdio_server() as (read, write):
        await server.run(read, write)

if __name__ == "__main__":
    import asyncio
    asyncio.run(main())

MCP Configuration

{
  "mcpServers": {
    "weather": {
      "command": "python",
      "args": ["/path/to/weather_server.py"]
    },
    "database": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-postgres"],
      "env": {
        "DATABASE_URL": "postgresql://user:pass@localhost/db"
      }
    },
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/allowed/path"]
    }
  }
}

Error Handling

Robust error handling is critical for tool-using applications. Tools can fail due to network issues, invalid inputs, rate limits, or external service outages.

Error Handling Strategies

import json
from typing import Union

class ToolError(Exception):
    """Custom exception for tool errors"""
    def __init__(self, message: str, recoverable: bool = True):
        self.message = message
        self.recoverable = recoverable
        super().__init__(message)

def execute_tool(tool_name: str, arguments: dict) -> Union[dict, str]:
    """Execute a tool with comprehensive error handling"""
    try:
        # Validate arguments
        if not validate_arguments(tool_name, arguments):
            raise ToolError("Invalid arguments provided", recoverable=True)

        # Execute the tool
        if tool_name == "get_weather":
            result = get_weather(**arguments)
        elif tool_name == "search_database":
            result = search_database(**arguments)
        else:
            raise ToolError(f"Unknown tool: {tool_name}", recoverable=False)

        return result

    except ConnectionError as e:
        # Network issues - suggest retry
        return {
            "error": True,
            "type": "network_error",
            "message": f"Could not connect to service: {e}",
            "suggestion": "The service may be temporarily unavailable. Please try again."
        }

    except RateLimitError as e:
        # Rate limiting - provide wait time
        return {
            "error": True,
            "type": "rate_limit",
            "message": "API rate limit exceeded",
            "retry_after": e.retry_after,
            "suggestion": f"Please wait {e.retry_after} seconds before retrying."
        }

    except ValidationError as e:
        # Invalid input - help fix it
        return {
            "error": True,
            "type": "validation_error",
            "message": str(e),
            "suggestion": "Please check the input format and try again."
        }

    except ToolError as e:
        return {
            "error": True,
            "type": "tool_error",
            "message": e.message,
            "recoverable": e.recoverable
        }

    except Exception as e:
        # Unexpected error - log and return generic message
        logger.exception(f"Unexpected error in tool {tool_name}")
        return {
            "error": True,
            "type": "internal_error",
            "message": "An unexpected error occurred",
            "suggestion": "Please try a different approach or contact support."
        }

Prompt for Error Recovery

You have access to several tools. When using tools:

1. **Before calling a tool**: Verify you have all required parameters
2. **If a tool returns an error**:
   - Read the error message and suggestion carefully
   - If recoverable, try to fix the issue (e.g., correct parameters)
   - If not recoverable, explain the limitation to the user
   - Suggest alternative approaches if available

3. **Error response format**:
   When a tool fails, you'll receive:
   ```json
   {
     "error": true,
     "type": "error_type",
     "message": "description",
     "suggestion": "what to do"
   }
   ```

4. **Never**:
   - Retry the exact same call more than twice
   - Ignore errors and make up information
   - Expose internal error details to users

5. **Always**:
   - Acknowledge limitations honestly
   - Offer alternative solutions when possible
   - Ask for clarification if inputs are unclear

Multi-Tool Orchestration

Complex tasks often require coordinating multiple tools. Effective orchestration involves planning tool sequences and handling dependencies.

Tool Planning

**User Request**: "Find all customers who haven't logged in for 30 days and send them a re-engagement email"

**Available Tools**:
1. query_database - Query customer data
2. send_email - Send emails to customers
3. log_action - Record actions for audit

**Tool Execution Plan**:

Step 1: Query inactive customers
Tool: query_database
Input: {
  "query": "SELECT email, name, last_login FROM customers WHERE last_login < NOW() - INTERVAL '30 days'"
}

Step 2: For each customer, send email
Tool: send_email (loop)
Input: {
  "to": "[customer_email]",
  "template": "re_engagement",
  "variables": {"name": "[customer_name]", "days_inactive": "[days]"}
}

Step 3: Log the campaign
Tool: log_action
Input: {
  "action": "re_engagement_campaign",
  "details": {"customers_contacted": "[count]", "timestamp": "[now]"}
}

Parallel Tool Execution

from openai import OpenAI
import json
import asyncio

client = OpenAI()

# Define multiple tools
tools = [
    {"type": "function", "function": {"name": "get_weather", ...}},
    {"type": "function", "function": {"name": "get_traffic", ...}},
    {"type": "function", "function": {"name": "get_events", ...}}
]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Plan my day trip to Tokyo tomorrow"}],
    tools=tools,
    parallel_tool_calls=True  # Enable parallel calls
)

message = response.choices[0].message

if message.tool_calls:
    # Model requested multiple tools in parallel
    # Execute them concurrently
    async def execute_tools(tool_calls):
        tasks = []
        for tc in tool_calls:
            func_name = tc.function.name
            args = json.loads(tc.function.arguments)
            tasks.append(execute_tool_async(func_name, args))
        return await asyncio.gather(*tasks)

    results = asyncio.run(execute_tools(message.tool_calls))

    # Send all results back
    tool_messages = [
        {
            "role": "tool",
            "tool_call_id": tc.id,
            "content": json.dumps(result)
        }
        for tc, result in zip(message.tool_calls, results)
    ]

    final_response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "user", "content": "Plan my day trip to Tokyo tomorrow"},
            message,
            *tool_messages
        ]
    )

Tool Chaining Pattern

def handle_complex_request(user_message: str, max_iterations: int = 5):
    """Handle requests that may require multiple tool calls"""

    messages = [{"role": "user", "content": user_message}]

    for i in range(max_iterations):
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=tools
        )

        assistant_message = response.choices[0].message
        messages.append(assistant_message)

        # Check if model is done (no more tool calls)
        if not assistant_message.tool_calls:
            return assistant_message.content

        # Execute all requested tools
        for tool_call in assistant_message.tool_calls:
            result = execute_tool(
                tool_call.function.name,
                json.loads(tool_call.function.arguments)
            )
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result)
            })

    # Max iterations reached
    return "I wasn't able to complete the task within the allowed steps."

Security Considerations

DESTRUCTIVE_TOOLS = {"delete_file", "drop_table", "send_email", "make_payment"}

def execute_with_confirmation(tool_name: str, arguments: dict, user_id: str):
    """Execute tool with confirmation for destructive actions"""

    if tool_name in DESTRUCTIVE_TOOLS:
        # Generate confirmation token
        token = generate_confirmation_token(tool_name, arguments, user_id)

        return {
            "requires_confirmation": True,
            "action": tool_name,
            "details": arguments,
            "confirmation_token": token,
            "message": f"This action will {describe_action(tool_name, arguments)}. Please confirm."
        }

    # Non-destructive: execute immediately
    return execute_tool(tool_name, arguments)

Exercises

Exercise 1: Tool Definition (Difficulty: Easy)

Exercise 2: Error Handling (Difficulty: Medium)

Exercise 3: MCP Server (Difficulty: Medium)

Exercise 4: Multi-Tool Orchestration (Difficulty: Advanced)

Exercise 5: Security Implementation (Difficulty: Advanced)

Chapter Summary

Next Steps

In Chapter 5, we'll put everything together with practical projects:

• Building a document analysis system
• Vision prompting for multimodal applications
• Prompt caching for cost optimization
• Production deployment considerations

References

• OpenAI. (2024). Function Calling Documentation
• Anthropic. (2024). Model Context Protocol Specification
• Google. (2024). Gemini Function Calling Guide
• LangChain. (2024). Tool Use in LangChain

Update History

• 2026-01-12: v2.0 Initial release with MCP coverage