How to Parse JSON From an LLM Response

Why json.loads crashes

You ask the model for JSON and it replies:

```json
{"name": "Alice", "role": "admin"}
```

The JSON is valid, but json.loads chokes on the surrounding ```json fence and the trailing ```. Models do this inconsistently - sometimes bare JSON, sometimes fenced, sometimes with a "Here's the JSON:" preamble - which is why it fails intermittently.

Strip the fences before parsing

Pull the JSON out of any markdown fence, then parse:

import re, json

def parse_llm_json(text: str):
    text = text.strip()
    # remove a leading ```json / ``` and a trailing ```
    m = re.search(r"```(?:json)?\s*(.*?)\s*```", text, re.DOTALL)
    if m:
        text = m.group(1)
    return json.loads(text)

A more forgiving version extracts the first {...} or [...] block, which also survives a stray preamble:

m = re.search(r"(\{.*\}|\[.*\])", text, re.DOTALL)
return json.loads(m.group(1)) if m else json.loads(text)

Better: don't rely on cleanup alone

• Prompt for raw JSON - "Respond with only valid JSON, no markdown, no explanation." Reduces (but won't eliminate) fences.
• Use a structured-output / JSON mode if your provider has one - it guarantees parseable JSON and removes the whole class of bug.
• Always guard the parse - wrap in try/except and handle the case where the model returned prose instead of JSON, so one bad response doesn't 500 the endpoint.

Get the model to return clean JSON, then validate it

Stripping fences is a patch; the durable fix is to stop the mess at the source and never trust the output blindly.

• Ask for JSON only. A system instruction like "Respond with a single JSON object and nothing else" removes most prose and fences.
• Use a JSON mode if the provider has one. OpenAI's response_format={"type": "json_object"} and similar features guarantee parseable JSON, so json.loads stops failing.
• Extract defensively as a fallback. If a model still wraps output, pull the first balanced object with re.search(r"\{.*\}", text, re.S) before parsing.
• Validate the shape. Even valid JSON can have missing or wrong-typed fields. Parse into a Pydantic model (or check keys and types) so a malformed-but-valid response fails loudly instead of corrupting downstream code.

Together these turn an unreliable string into a contract: the model returns JSON, your code proves it matches the schema, and only then do you use it.