How to Build an LLM Agent Loop (Decide, Act, Observe)

What makes it an agent

A one-shot LLM call sends a question and returns a reply. An agent adds a control loop: the model can emit an action (e.g. ACTION: calculator | 12*8), your code runs the tool, and the result goes back into the message history so the model can reason over the real output. This is the ReAct pattern (Reason + Act).

Without the loop, the model returns ACTION: calculator | 12*8 as the "answer" - the tool never runs and the model is just guessing.

The loop in Python

import re, requests

LLM_URL = "http://llm-service:8080/v1/chat/completions"

SYSTEM = (
    "You are an agent. To use a tool, reply exactly 'ACTION: calculator | <expr>'. "
    "When you have the answer, reply exactly 'FINAL: <answer>'."
)

def calculator(expr):
    return eval(expr, {"__builtins__": {}}, {})

TOOLS = {"calculator": calculator}

def call_llm(messages):
    r = requests.post(LLM_URL, json={"model": "gpt-4o-mini",
                                      "messages": messages, "temperature": 0},
                      timeout=60)
    r.raise_for_status()
    return r.json()["choices"][0]["message"]["content"].strip()

def run(question, max_steps=5):
    messages = [
        {"role": "system", "content": SYSTEM},
        {"role": "user",   "content": question},
    ]
    steps = []
    for _ in range(max_steps):
        reply = call_llm(messages)
        if reply.startswith("FINAL:"):
            return {"answer": reply[len("FINAL:"):].strip(), "steps": steps}
        m = re.match(r"ACTION:\s*(\w+)\s*\|\s*(.+)", reply)
        if not m:
            return {"answer": reply, "steps": steps}
        tool, tool_input = m.group(1), m.group(2).strip()
        result = TOOLS[tool](tool_input)
        steps.append({"tool": tool, "input": tool_input, "observation": result})
        messages.append({"role": "assistant", "content": reply})
        messages.append({"role": "user",      "content": f"Observation: {result}"})
    return {"answer": "(stopped: max steps reached)", "steps": steps}

Key design decisions

• Append both sides of each turn - after a tool call, add the assistant's ACTION message and the Observation: <result> user message. The model needs both in context to know what it did and what came back.
• Bound the loop with max_steps - a confused or adversarial prompt can spin the loop forever and burn tokens. Five to ten steps is usually plenty; return a graceful fallback when the limit is hit.
• Parse the action strictly - use a regex (ACTION:\s*(\w+)\s*\|\s*(.+)) so an unexpected model reply exits cleanly instead of crashing or calling a nonexistent tool.
• Keep TOOLS as a plain dict - easy to add new tools (web search, database query, file read) without changing the loop logic.

Relation to frameworks

LangGraph, the OpenAI Agents SDK, and CrewAI all implement this same decide-act-observe loop under the hood, with added features: parallel tool calls, structured tool-call APIs (no regex needed), retry logic, and tracing. Writing the raw loop first makes the framework's abstractions obvious rather than magical.