How to Build an LLM Text Classifier

The pattern

A classifier asks the LLM to pick a category and return only that word. The system prompt does two things: lists the valid labels, and forbids any explanation. Set "temperature": 0 so the output is deterministic.

import requests, sys

LLM_URL = "http://your-llm-service/v1/chat/completions"

SYSTEM = (
    "You are a customer support classifier. "
    "Classify the message into exactly one of these categories: "
    "billing, technical, general. "
    "Reply with only the single category word, nothing else."
)

def classify(message: str) -> str:
    resp = requests.post(LLM_URL, json={
        "model": "your-model",
        "messages": [
            {"role": "system", "content": SYSTEM},
            {"role": "user",   "content": message},
        ],
        "temperature": 0,
        "max_tokens": 10,
    }, timeout=30)
    resp.raise_for_status()
    return resp.json()["choices"][0]["message"]["content"].strip().lower()

if __name__ == "__main__":
    print(classify(" ".join(sys.argv[1:])))

Run it from the CLI:

python3 classify.py "I was charged twice this month"
# billing
python3 classify.py "The app crashes on login"
# technical
python3 classify.py "How do I change my username?"
# general

Why the prompt is the hard part

The reliability of a one-shot classifier lives almost entirely in the system prompt. Three things matter:

• Name the label set explicitly - list every valid value. If you write "classify as billing, technical, or general", the model knows not to invent "account" or "other".
• Forbid explanation - "Reply with only the single category word, nothing else" prevents the model from adding "The category is: billing." strip().lower() handles minor whitespace but you need the prompt to stop prose output.
• Set temperature: 0 - deterministic sampling gives you the same label for the same input, which matters for regression testing and reproducibility.

Validate the output

Even with a tight prompt, add a guard:

VALID = {"billing", "technical", "general"}

def classify(message: str) -> str:
    label = _call_llm(message).strip().lower()
    if label not in VALID:
        raise ValueError(f"Unexpected label: {label!r}")
    return label

A ValueError is much easier to debug than silently routing a message to the wrong queue.

When to use this shape

Single-call classifiers, extractors, and formatters make up a large share of production LLM workloads - far more than chat loops. This pattern works for sentiment analysis, intent detection, priority scoring, and any case where the output is one of a small, known set of values. Once the function is solid, you can pipe it into bash, call it from a service, or schedule it as a cron job.