Build a Reliable Python AI Agent with Structured Tool Calling

What you will build

A small Python agent that can call one tool, a weather lookup, through a validated schema. You will run it, break it on purpose, and add a test for every failure. The tool is a stub so the guide runs offline; swap in a real API later.

Prerequisites

• Python 3.12 installed
• Comfort with Python functions and type hints
• An API key for the model provider you use
• Git, to clone the starter repo

Architecture

The model never touches your function directly. It proposes a tool call, your code validates the arguments, runs the tool with a timeout, and returns the result.

Two paths matter most. The retry path re-asks the model after a recoverable error. The error path stops cleanly and tells the user, instead of looping.

Watch the walkthrough

Walkthrough: a tool-calling agent, start to finish

A second video can come from anywhere. This one is a plain video file; a Vimeo, Loom or any other embed link works the same way.

Setup

git clone https://github.com/analogdata/ai-agent-starter
cd ai-agent-starter
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Tool schema

Describe the tool arguments as a model. Anything the agent sends that does not match is rejected before it reaches your function.

from typing import Literal
from pydantic import BaseModel, Field


class WeatherArgs(BaseModel):
    city: str = Field(min_length=2, max_length=80)
    unit: Literal["celsius", "fahrenheit"] = "celsius"


def get_weather(args: WeatherArgs) -> dict:
    # Stub so the guide runs offline. Replace with a real API call.
    return {"city": args.city, "temperature": 24, "unit": args.unit}

Agent loop

A small dispatcher validates the arguments, runs the tool with a timeout, and retries a bounded number of times.

import asyncio
from pydantic import ValidationError

MAX_RETRIES = 2
TOOL_TIMEOUT_S = 5.0


async def run_tool(name: str, raw_args: dict) -> dict:
    tool = TOOLS[name]  # maps tool name -> (ArgsModel, function)
    args_model, fn = tool
    try:
        args = args_model.model_validate(raw_args)
    except ValidationError as err:
        return {"error": "invalid_arguments", "detail": err.errors(include_url=False)}

    for attempt in range(MAX_RETRIES + 1):
        try:
            async with asyncio.timeout(TOOL_TIMEOUT_S):
                return await asyncio.to_thread(fn, args)
        except TimeoutError:
            if attempt == MAX_RETRIES:
                return {"error": "timeout", "detail": f"{name} exceeded {TOOL_TIMEOUT_S}s"}

1. Validate

   Reject bad arguments before anything runs.

2. Run with a timeout

   Never wait on a tool forever.

3. Retry, then stop

   A fixed limit keeps a slow tool from becoming a loop.

Run and inspect

python -m agent "Weather in Bengaluru, in celsius?"

Log the tool name, the validated arguments and the result. Redact anything secret before it reaches the logs.

Failure modes

Testing

import pytest
from agent import run_tool


@pytest.mark.asyncio
async def test_rejects_invalid_args():
    out = await run_tool("get_weather", {"city": "", "unit": "kelvin"})
    assert out["error"] == "invalid_arguments"


@pytest.mark.asyncio
async def test_valid_call_returns_result():
    out = await run_tool("get_weather", {"city": "Bengaluru"})
    assert out["city"] == "Bengaluru"

• Test one invalid call
• Test a tool that times out
• Test that retries stop at the limit

Costs and limits

Cost depends on the model you pick and how many calls each task makes. Retry limits and timeouts keep both predictable.

FAQs

Resources