What the context is
┌──────────────── context window: 1M tokens on Claude 5.5 ────────────────┐
│ tools tool descriptions hundreds of tokens each │
│ system role, rules, constant materials │
│ messages user → assistant → user → … → the last question │
│ (+ files, images, tool results) │
├─────────────────────────────────────────────────────────────────────────┤
│ answer reasoning + text ≤ max_tokens │
└─────────────────────────────────────────────────────────────────────────┘
A chat in the browser does the same: it bundles its own system prompt, the whole conversation and the project files into one request. That’s why a long chat gets more expensive and duller (Long conversations).
The request, block by block
{
"model": "claude-sonnet-5-5",
"max_tokens": 16000,The model and the answer ceiling. max_tokens is required: hit it and the answer is cut off, with stop_reason set to max_tokens. The model doesn’t know about the ceiling and doesn’t adjust. Without streaming, set around 16,000; with streaming, up to 64,000; up to 128K only with streaming, or the request runs into an HTTP timeout.
"output_config": { "effort": "medium" },Reasoning depth. On 5.5 models reasoning is on by default; effort from low to max decides how much the model thinks and how many tokens it spends. Reasoning is output tokens at the output price. By default the reasoning text isn’t returned (display: omitted), but it’s billed.
default:effort high on Sonnet 5.5, medium on Opus 5.5 and Haiku 5.5
"system": [
{
"type": "text",
"text": "Server configs: Ubuntu 24.04, Traefik 3.7. Commands come with a rollback.",
"cache_control": { "type": "ephemeral" }
}
],The system prompt: who the model is here, rules, constant materials. It’s read first, and the model ranks its rules above requests in the conversation. It changes rarely — so add a cache marker: later requests with the same start read it 10 times cheaper (Prompt cache). Any changed byte — a date, a counter — invalidates the cache for everything after it.
"tools": [
{
"name": "read_file",
"description": "Reads a file from the config repository",
"input_schema": {
"type": "object",
"properties": { "path": { "type": "string" } },
"required": ["path"]
}
}
],Tools: a name, a description and a JSON input schema. The description is text in the context too, and you pay for it on every request. Twenty tools are several thousand tokens before the question even starts. How the model calls them — Agents and tools.
"messages": [
{ "role": "user", "content": "Why the 404 on memos?" },
{ "role": "assistant", "content": [
{ "type": "text", "text": "I need the container's labels" }
] },The history. The API stores nothing: every request carries all past turns. Roles alternate, and the first is user. Return the model’s answer in full, with all its blocks (thinking, tool_use), not just the text — otherwise the reasoning and tool calls are lost.
{
"role": "user",
"content": [
{ "type": "text", "text": "<labels>…</labels>" },
{ "type": "text", "text": "Here are the labels. Where's the mistake?" }
]
}
]
}The new question. Long material first, the question last: the model answers more precisely that way (Long document first). Files and images go as separate blocks of the same message.
"temperature": 0,
"top_p": 0.9,Randomness in token choice. On Claude Opus 4.7 and newer these fields are gone, and a request with them gets 400; Sonnet 5.5 accepts only the default value. Other vendors and local models still have them: 0 means almost always the most likely token, 1 the usual spread.
{ "role": "assistant", "content": "{" }Prefill: a final assistant message so the model continues from {. The old way to get JSON. On 4.6 models and newer it returns 400; use structured output, output_config.format, instead (JSON by schema).
What comes back
{
"id": "msg_…",
"model": "claude-sonnet-5-5",
"content": [
{ "type": "thinking", "thinking": "", "signature": "…" },
{ "type": "text", "text": "The router has no `entrypoints`…" }
],
"stop_reason": "end_turn",
"usage": { "input_tokens": 412, "cache_read_input_tokens": 1830, "output_tokens": 690 }
}
content is a list of blocks, and the first is often thinking with empty text. Take the answer text with a filter by
type, not content[0]: jq -r '.content[] | select(.type == "text") | .text'.
stop_reason |
What happened | What to do |
|---|---|---|
end_turn |
the model finished on its own | nothing |
max_tokens |
hit the ceiling, the answer is cut off | raise max_tokens; stream long answers |
tool_use |
the model wants to call a tool | run it, and send the result in the next user message (Agents and tools) |
stop_sequence |
a configured stop string appeared | nothing, that’s by design |
pause_turn |
a long turn on server tools was paused | send the request again; it continues |
refusal |
a refusal under the safety rules | look at stop_details.category; rephrase or use a fallback model |
If it doesn’t work
400 mentioning temperature after switching to a new model
- Cause
- On Claude Opus 4.7 and newer,
temperature,top_pandtop_kwere removed; Sonnet 5.5 accepts only the default value. - Fix
- Remove the parameters from the request; tame answer variance with format and checks (Repeatable answers).
400 on a request that ends with an assistant message
- Cause
- Answer prefill is disabled on 4.6 models and newer.
- Fix
- Set the format with structured output,
output_config.format(JSON by schema), or with an instruction insystem.
400 with thinking: {type: "disabled"} on Opus 5.5
- Cause
- Opus 5.5 can’t turn reasoning off at any
effortlevel. - Fix
- Remove
thinkingand seteffort: "low"(Match reasoning to the task).
The model “doesn’t see” the previous answer
- Cause
- The program sent only the last question: the API stores no history; the client sends it.
- Fix
- Put every past
userandassistantpair intomessages, the model’s answer in full, with all its blocks.