Stop Parsing Prose – Blog Directory

Asking an LLM for JSON in English often returns JSON wrapped in extra text: a preamble, markdown fences, or a trailing remark. json.loads() rejects all of those.

The failure mode is the default conversational training objective, not a one-off.

1 Failure mode

Typical extra tokens:

preamble (Sure, here's the JSON you requested:)
markdown fences
comments inside supposed JSON
trailing pleasantries

Workarounds that treat this as a prompting problem:

regex to strip preambles
retry loops on parse failure
prompts that say “respond with ONLY the JSON”

Those are heuristics. They are not a schema.

2 Prefill and stop sequences

Before the API enforced a shape, two tricks were used together:

Prefill the assistant turn with { so generation starts inside the object.
Stop sequence on } so generation cannot continue after the first close brace.

# The old way — do not use this on current models.
response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    stop_sequences=["}"],
    messages=[
        {"role": "user", "content": ticket},
        {"role": "assistant", "content": "{"},   # prefill
    ],
)

raw = "{" + response.content[0].text + "}"       # reattach what you clipped
data = json.loads(raw)                           # ...and hope

Limits:

Nested objects break a stop sequence on the first }.
The caller must reattach the clipped braces.
Prefill returns HTTP 400 on Sonnet 5, Opus 5, Fable 5, and Opus/Sonnet 4.6+. Haiku 4.5 still accepts it.

3 JSON Schema via `output_config`

Pass a JSON Schema through output_config.format. The API emits only tokens that conform to that schema.

The ticket below is synthetic, written to stand in for a real ingestion job where mis-routing has a cost.

import json
import anthropic

client = anthropic.Anthropic()

ticket = """
Our nightly export has failed at 90% completion three nights running.
We report to the board Friday and this data feeds that report directly.
Please help ASAP, this is blocking a hard deadline.
"""

schema = {
    "type": "object",
    "properties": {
        "category": {"type": "string", "enum": ["bug", "feature_request", "question", "billing"]},
        "urgency": {"type": "string", "enum": ["low", "medium", "high", "critical"]},
        "sentiment": {"type": "string", "enum": ["neutral", "satisfied", "frustrated", "angry"]},
        "summary": {"type": "string"},
        "suggested_team": {"type": "string", "enum": ["engineering", "support", "billing", "sales"]},
    },
    "required": ["category", "urgency", "sentiment", "summary", "suggested_team"],
    "additionalProperties": False,
}

response = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=4096,                  # shared by thinking + response; leave headroom
    messages=[{"role": "user", "content": ticket}],
    output_config={"format": {"type": "json_schema", "schema": schema}},
)

if response.stop_reason != "end_turn":         # refusal, or truncated mid-object
    raise RuntimeError(f"no conforming output: {response.stop_reason}")

text = next(b.text for b in response.content if b.type == "text")
data = json.loads(text)

This post did not call the API. The object below is hand-written to show the guaranteed shape, not a captured transcript.

{
  "category": "bug",
  "urgency": "critical",
  "sentiment": "frustrated",
  "summary": "Nightly export has failed at 90% completion for three consecutive nights, blocking a Friday board report.",
  "suggested_team": "engineering"
}

The same contract applies to any output that must become a program value: log lines, resumes, tool-call arguments.

4 Guarantees

A schema guarantees shape, not correctness.

You will not get malformed JSON.
You can still get a wrong category in valid syntax. Classifications still need spot-checks.

The guarantee applies only to a turn that finishes:

Safety refusal or max_tokens truncation yields an empty or clipped body.
Check stop_reason before json.loads.
Thinking shares max_tokens with the response; budget for both.

Those failures surface in stop_reason before downstream code acts on the body.

1 Failure mode

2 Prefill and stop sequences

3 JSON Schema via output_config

4 Guarantees

5 References

3 JSON Schema via `output_config`