How to Do Sentiment Analysis With NLTK vs an LLM Classifier

Sentiment analysis with NLTK gives you a fast lexicon score, but an LLM classifier reads context NLTK misses - as long as you constrain the output. The fix for a chatty model is a tight system prompt ("Reply with exactly one word: positive, negative, or neutral") plus a regex that pulls the label out, so /classify returns {"sentiment": "positive"} instead of a paragraph.

AI Engineerpythonllmprompt-engineering

NLTK vs an LLM for sentiment analysis

The classic Python approach is nltk.sentiment.vader, a rule-based lexicon that scores text without any API call:

import nltk
nltk.download("vader_lexicon")
from nltk.sentiment import SentimentIntensityAnalyzer

sia = SentimentIntensityAnalyzer()
print(sia.polarity_scores("I absolutely love this!"))
# {'neg': 0.0, 'neu': 0.192, 'pos': 0.808, 'compound': 0.6696}

VADER is fast and free, but it scores words in isolation - it misses sarcasm, negation across a clause, and domain slang. An LLM classifier reads the whole sentence in context, which is why teams reach for one when accuracy matters more than latency. The catch: an LLM will happily return a paragraph unless you force it not to.

Why the LLM returns a paragraph instead of a label

Clients of a POST /classify endpoint expect exactly one of positive, negative, or neutral. A loose system prompt like "Analyze the sentiment of the text the user provides" invites the model to explain itself:

The text has a positive sentiment because the user expresses enthusiasm...

That fails validation and returns a 502. The LLM API is fine - the prompt is the bug. Fixing it is pure prompt engineering: narrow the output space.

Step 1 - constrain the system prompt

Tell the model the exact format, the allowed values, and to skip the explanation:

payload = {
    "model": MODEL,
    "messages": [
        {
            "role": "system",
            "content": "You are a sentiment classifier. Reply with exactly one word: positive, negative, or neutral. No explanation, no punctuation."
        },
        {"role": "user", "content": text}
    ],
    "temperature": 0.1,
    "max_tokens": 200
}

Naming the three labels stops the model from inventing "mixed" or "unclear", and "No explanation, no punctuation" kills the prose.

Step 2 - extract and validate the label

Even with a tight prompt, a model may return Positive. with a capital letter or a trailing period. Do not trust the raw string - pull the label out with a regex and validate it:

import re

raw = data["choices"][0]["message"]["content"].strip()
_raw = raw.lower().strip()
m = re.search(r"positive|negative|neutral", _raw)
normalized = m.group(0) if m else _raw

if normalized not in ("positive", "negative", "neutral"):
    return jsonify({"error": "LLM returned unexpected format",
                    "raw_response": raw}), 502
return jsonify({"sentiment": normalized})

The regex handles the whitespace and punctuation the prompt cannot fully guarantee, so Positive. still normalizes to positive.

Step 3 - restart and verify

Restart the Flask app so it picks up the source change, then curl the endpoint:

pkill -f "python3 classifier.py"; python3 classifier.py > /tmp/app.log 2>&1 &
sleep 1 && curl http://localhost:5000/health

curl -X POST http://localhost:5000/classify \
  -H 'Content-Type: application/json' \
  -d '{"text": "I love this!"}'
# {"sentiment": "positive"}

The output space is now a set of three known values - the same discipline that makes any single-call LLM classifier reliable in production.

Want to try it hands-on? HeyDevJob gives you this exact setup in a live cloud workspace in your browser - edit it, run it, and see it work. Free, nothing to install.

Try it in a workspace →

What you'll practice

Comparing NLTK VADER lexicon scoring against an LLM classifier for sentiment
Writing a constrained system prompt that forces a single-word label
Extracting and validating the label with a regex so punctuation is tolerated

FAQ

How do I do sentiment analysis with NLTK in Python?

Install nltk, download the vader_lexicon, then call SentimentIntensityAnalyzer().polarity_scores(text). It returns pos, neu, neg, and a compound score without any API call. VADER is a rule-based lexicon, so it is fast and free but scores words in isolation and can miss sarcasm or negation.

Is an LLM better than NLTK for sentiment analysis?

An LLM reads the full sentence in context, so it handles sarcasm, negation, and domain slang that NLTK VADER misses. The tradeoff is latency and cost per call. For high-volume, simple text NLTK is often good enough; for nuanced text where accuracy matters, a constrained LLM classifier wins.

Why does my LLM sentiment classifier return a paragraph instead of one word?

The system prompt is too loose. A model told only to "analyze the sentiment" will explain its reasoning. Constrain it explicitly - "Reply with exactly one word - positive, negative, or neutral. No explanation." - and set a small max_tokens so it cannot fit extra prose.

How do I get a clean single-word label from an LLM response?

Constrain the prompt, then extract the label with a regex like re.search(r"positive|negative|neutral", text.lower()) and validate it against your allowed set. This tolerates capitalization and trailing punctuation such as "Positive." and returns a 502 when the model output does not match any known label.

Keep learning

Build an LLM Text ClassifierAI/ML project Parse JSON From an LLM (Strip Markdown Fences)AI/ML project Build an LLM Content Moderation FilterAI/ML project AI/ML roadmapStep by step to hired AI/ML interview questionsSTAR answers All AI/ML projectsProjects hub

Learn it by doing. Open this in a live cloud workspace, make the change yourself, and keep a record of the work you can share.

Open the workspace →