Tool Calling

Tool calling is how a model reaches outside itself — to look something up, query a database, or call an API.

It is also the most misunderstood part of agent work, because of one detail: the model never runs your code. Get that straight and everything else follows.

What a tool is

A tool is a Python function plus a description of it that a model can read.

Four parts:

def get_ticket(ticket_id: int) -> str:
    """Fetch an incident ticket by its id."""
    return db.fetch(ticket_id)

The name, the docstring and the type hints are not documentation here. They are the interface the model sees, and a framework turns them into JSON Schema:

{
  "name": "get_ticket",
  "description": "Fetch an incident ticket by its id.",
  "parameters": {
    "type": "object",
    "properties": {"ticket_id": {"type": "integer"}},
    "required": ["ticket_id"]
  }
}

Notice how each piece of the function maps onto the schema. int became "integer". No default made it required. The docstring became the description.

The exchange, step by step

• You send the question and the tool schemas
• The model replies: "call get_ticket with {"ticket_id": 4471}" — as structured text, not an action
• Your code looks up the function and runs it
• You append the result to the messages
• You send everything back
• The model uses the result to answer, or asks for another tool

Steps 3 and 4 are yours. The model only ever produces text.

What that means in practice

Because your code sits in the middle, you decide:

def run_tool(call):
    name = call["function"]["name"]
    args = json.loads(call["function"]["arguments"])

    if name not in TOOLS:
        return f"Error: no tool called {name}"

    if name in DANGEROUS and not user_approved(name, args):
        return "Error: not approved by the user"

    logger.info("tool=%s args=%s", name, args)

    try:
        return TOOLS[name](**args)
    except Exception as e:
        logger.exception("tool %s failed", name)
        return f"Error: {e}"

Validation, permissions, logging and an approval gate — none of which the model can bypass, because it was never in control. An agent that executes whatever comes back, unchecked, is a security hole rather than a design.

The description is the most important line

The model chooses a tool by reading its description. That sentence is doing real work.

Weak:

def search(q: str) -> str:
    """Search."""

Strong:

def search_runbook(query: str) -> str:
    """Search the internal runbook for troubleshooting steps.
    Use this for questions about how to fix a known issue.
    Do not use it for looking up ticket details."""

Saying what a tool is not for is often what stops the model reaching for the wrong one. When an agent keeps picking the wrong tool, the fix is almost always the description, not the model.

Same for parameters:

class SearchInput(BaseModel):
    query: str = Field(description="Search terms in plain English, not a ticket id")
    limit: int = Field(default=5, ge=1, le=20, description="How many results")

Vague descriptions produce vague arguments.

Letting a framework do it

from langchain_core.tools import tool

@tool
def get_ticket(ticket_id: int) -> str:
    """Fetch an incident ticket by its id."""
    return db.fetch(ticket_id)

The decorator reads the function and builds the schema. Two things fail quietly if you forget them:

• no type hints — the schema cannot say what ticket_id is
• no docstring — the model has no idea when to use it

Neither raises an error. The agent simply behaves badly, which is much harder to diagnose than a crash.

Several tools at once

A model may ask for more than one in a single turn.

for call in reply.get("tool_calls", []):
    result = run_tool(call)
    messages.append({
        "role": "tool",
        "tool_call_id": call["id"],
        "content": str(result),
    })

Each result must carry the tool_call_id it answers, so the model can match them up. If they are independent, run them concurrently with asyncio.gather and take the time of the slowest rather than the sum.

Designing tools well

• One job each. get_ticket and update_ticket, not manage_ticket with a mode flag.
• Few, distinct tools. Twenty overlapping tools make the choice harder, not the agent more capable. Start with three.
• Simple arguments. Strings, numbers, booleans. Deeply nested objects get filled in badly.
• Return text the model can read. A short, factual sentence beats a raw object dump.
• Return errors, do not raise them. "Error: no ticket 4471" lets the agent recover; an exception ends the run.
• Keep results small. Everything you return is re-sent on every later turn, so a 50,000-character dump is paid for repeatedly.

That last point catches people. A tool that returns an entire document does not just cost once — it sits in the message history for the rest of the conversation.

How the model actually chooses

It does not reason about your code, which it has never seen. It matches the question against the descriptions, and produces arguments in the shape the schema demands.

So when tool selection goes wrong, the three things to look at, in order:

• is the description specific, and does it say what the tool is not for?
• are two tools too similar to tell apart?
• do the parameter descriptions make the right value obvious?

Changing the model is almost never the fix.

Tools, connectors and MCP

Three words for neighbouring things:

• a tool is a function in your own code
• a connector is usually a pre-built tool for a common service
• MCP is a standard for exposing tools over a protocol, so one tool server can serve many agents

All three end up as the same thing from the model's point of view: a name, a description, and a schema.

Common mistakes

• Believing the model runs your function. It does not.
• No docstring, so the tool is never chosen.
• No type hints, so the schema is incomplete.
• Vague descriptions, then blaming the model for poor selection.
• Too many tools, with overlapping purposes.
• Raising instead of returning an error, killing the run.
• Returning enormous results that are re-sent every turn.
• Executing without validation, especially anything irreversible.

Practise this

• Write a tool function with full type hints and a docstring that says what it is not for.
• Write out, by hand, the JSON schema you would expect a framework to generate from it.
• Write a dispatcher that looks a tool up by name and returns an error string for an unknown one.
• Take two tools with similar purposes and rewrite the descriptions so a model could tell them apart.