Tool use and function calling is the mechanism that allows LLMs to interact with the outside world. By itself, a language model can only generate text — it cannot query a database, call an API, read a file, or perform calculations with guaranteed accuracy. Function calling bridges this gap: the model analyzes a user’s request, determines which external tool or function is needed, generates a structured JSON object with the function name and arguments, and the application layer executes it. The result is fed back to the model, which incorporates it into its response.
This capability is what transforms a chatbot into an agent. Without tool use, an LLM is limited to the knowledge baked into its weights. With tool use, it can access real-time data, perform precise computations, interact with enterprise systems, and take actions in the world. Every major LLM provider — OpenAI, Anthropic, Google, Mistral, and others — now supports function calling as a first-class API feature.
For AI engineers, understanding the mechanics of tool use is foundational. The quality of your tool definitions directly determines how reliably the model selects the right tool and provides correct arguments. Poor tool schemas lead to hallucinated parameters, wrong tool selection, and cascade failures in agentic workflows.
Function calling follows a strict request-response cycle between the LLM and the application:
tool_call object containing the function name and a JSON object of arguments.tool_result message.The LLM never executes functions directly. It only decides which function to call and generates the arguments. The application layer is responsible for actual execution, validation, and error handling. This separation is essential for security — the model proposes, the application disposes.
Each tool is defined using JSON Schema, which specifies the function’s name, a natural-language description (critical for the model’s tool selection), and the expected parameters with their types and constraints.
// Tool definition for OpenAI-style API
{
"type": "function",
"function": {
"name": "search_properties",
"description": "Search for hotel properties by location, star rating, or amenities. Use this when the user asks about available properties or wants to find specific hotels.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City or region to search in (e.g., 'Miami, FL')"
},
"min_rating": {
"type": "number",
"minimum": 1,
"maximum": 5,
"description": "Minimum star rating (1-5)"
},
"amenities": {
"type": "array",
"items": { "type": "string" },
"description": "Required amenities (e.g., ['pool', 'spa', 'gym'])"
},
"max_results": {
"type": "integer",
"default": 10,
"description": "Maximum number of results to return"
}
},
"required": ["location"]
}
}
}Anthropic’s Claude uses a similar but slightly different format. Tools are passed in the tools array of the API request, and the model responds with tool_use content blocks:
// Anthropic tool definition
{
"name": "get_weather",
"description": "Get the current weather for a given location.",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City and state, e.g., 'San Francisco, CA'"
}
},
"required": ["location"]
}
}
// Model response with tool_use
{
"role": "assistant",
"content": [
{
"type": "tool_use",
"id": "toolu_01A09q90qw90lq917835lq9",
"name": "get_weather",
"input": { "location": "San Francisco, CA" }
}
]
}Modern LLMs can emit multiple tool calls in a single response. If the tools are independent (e.g., fetching weather for three cities), the model generates all three calls at once, and the application can execute them in parallel. If tools are dependent (the output of one is the input to another), the model generates them sequentially across multiple turns.
When a model emits multiple tool_call objects in a single response, the application should execute them concurrently. This reduces round-trip latency from N sequential LLM calls to 1 LLM call + 1 parallel execution batch. For example, “What’s the weather in NYC, LA, and Chicago?” triggers three parallel get_weather calls instead of three sequential turns.
The model selects tools based on the natural-language description in the tool definition. This makes description quality critical:
Function calling and structured outputs are related but distinct capabilities. Function calling is about triggering actions — the model decides to call a function. Structured outputs are about formatting responses — the model generates JSON conforming to a schema. In practice, providers like OpenAI use the same underlying mechanism (JSON Schema enforcement via constrained decoding) for both features. Anthropic implements structured outputs through tool definitions, using a tool as a “schema container” even when no external function is actually called.
Models learn function calling during fine-tuning, not pre-training. The training data includes thousands of examples of conversations where the correct response is a tool call rather than a text completion. The model learns patterns like: when the user asks for real-time information → emit a search tool call; when the user asks to calculate something → emit a calculator tool call. During inference, the model’s logits are constrained to produce valid JSON matching the tool’s schema — this is why “strict mode” in OpenAI’s API guarantees schema compliance.
In retrieval-augmented generation, the retrieval step is itself a tool call. The model receives a user query, generates a search_knowledge_base tool call with a reformulated search query, receives the retrieved chunks as the tool result, and then generates a grounded response. This makes the retrieval step explicit and auditable — you can log exactly what query the model used and what chunks were returned.
Function calling enables LLMs to interact with existing enterprise APIs without building custom integrations. A hotel management chatbot might have tools for check_availability, create_reservation, get_guest_profile, and update_room_status. Each tool maps directly to an existing REST API endpoint, with the JSON Schema derived from the API’s OpenAPI specification.
In the ReAct pattern, tool use is the “Action” step. The agent thinks about what to do (Thought), calls a tool (Action), observes the result (Observation), and repeats. Complex tasks may involve dozens of tool calls across multiple tools — reading files, searching the web, running code, querying databases — all orchestrated by the LLM’s planning capability.
Real-world agents typically have access to 5-50 tools. The model must not only select the right tool but also chain tools together. For example: (1) search_contacts to find a customer, (2) get_order_history with the customer ID, (3) create_support_ticket with order details. Each tool call depends on the previous result, requiring the model to maintain state across multiple turns.
For high-stakes tool calls (e.g., delete_account, process_refund, send_email), production systems implement approval gates. The model proposes the tool call, but execution is paused until a human approves. This is a direct application of the human-in-the-loop pattern applied specifically to tool use.
required fields, type constraints, enum values, and descriptions on every parameter.