How to Implement LLM Function Calling in Python

Why let the LLM choose a tool?

Language models are poor at arithmetic and have no live data - but they are good at understanding intent. Function calling uses that: you describe a set of tools, the model replies with a structured tool choice, and your code executes it deterministically. No hallucinated math, no stale answers.

The model is the router. Your functions are the workers.

The shape of the loop

# 1. Ask the model - it replies with a JSON tool choice
# 2. Parse {"tool": "calculator", "input": "47*9"}
# 3. Look up the tool function and call it
# 4. Return the result (not the model's text)

A minimal Flask implementation

import json
import requests
from flask import Flask, request, jsonify

app = Flask(__name__)
LLM_URL = "http://llm-service:8080/v1/chat/completions"

def _calc(expr):
    return eval(expr, {"__builtins__": {}}, {})

TOOLS = {
    "calculator": _calc,
}

def ask_llm(message: str) -> str:
    body = {
        "model": "the-model",
        "messages": [
            {
                "role": "system",
                "content": 'Choose a tool. Reply ONLY with JSON: {"tool": "calculator", "input": "<expr>"}',
            },
            {"role": "user", "content": message},
        ],
        "temperature": 0,
    }
    r = requests.post(LLM_URL, json=body, timeout=60)
    r.raise_for_status()
    return r.json()["choices"][0]["message"]["content"]

@app.post("/api/ask")
def ask():
    message = (request.get_json(silent=True) or {}).get("message", "")
    raw = ask_llm(message)

    try:
        choice = json.loads(raw)
        tool_name = choice["tool"]
        tool_input = choice["input"]
    except Exception:
        return jsonify(error="could not parse a tool choice", raw=raw), 502

    if tool_name not in TOOLS:
        return jsonify(error=f"unknown tool: {tool_name}"), 400

    result = TOOLS[tool_name](tool_input)
    return jsonify(tool=tool_name, result=result)

The endpoint returns {"tool": "calculator", "result": 423} - not the model's raw text.

Three things that matter in production

• Prompt for raw JSON - "Reply ONLY with JSON" in the system message keeps responses parseable. If the model wraps the JSON in markdown fences, strip them before json.loads (see parse json from llm).
• Validate before executing - check that tool_name is in your allowlist and that tool_input matches the expected schema. Never eval() untrusted input in a real system - dispatch to named functions only.
• Use native function-calling APIs when available - OpenAI and Anthropic both expose structured tool-call APIs that return the tool request as typed JSON, bypassing brittle string parsing entirely. The loop stays the same; the parsing step goes away.

Extending to multiple tools

Add more entries to TOOLS and describe all of them in the system prompt. The model picks one per turn. For multi-step agent loops, feed the tool result back into the conversation as a new message and ask the model again - repeat until it stops calling tools.