basforge

Search titles, text and the “If it doesn’t work” cases.

    RU
    ConfigAIModel APIs

    A model request, block by block: what goes out every time

    A model doesn’t remember past requests: every time it gets everything at once — the system prompt, tool descriptions, the whole conversation and the new question. That’s the context. Here’s a Claude API request block by block: what each field means, what it costs in tokens and what changed in the 5.5 models.

    What the context is

    ┌──────────────── context window: 1M tokens on Claude 5.5 ────────────────┐
    │ tools     tool descriptions                  hundreds of tokens each    │
    │ system    role, rules, constant materials                               │
    │ messages  user → assistant → user → … → the last question               │
    │           (+ files, images, tool results)                               │
    ├─────────────────────────────────────────────────────────────────────────┤
    │ answer    reasoning + text ≤ max_tokens                                 │
    └─────────────────────────────────────────────────────────────────────────┘

    A chat in the browser does the same: it bundles its own system prompt, the whole conversation and the project files into one request. That’s why a long chat gets more expensive and duller (Long conversations).

    The request, block by block

    POST https://api.anthropic.com/v1/messages
    {
      "model": "claude-sonnet-5-5",
      "max_tokens": 16000,

    The model and the answer ceiling. max_tokens is required: hit it and the answer is cut off, with stop_reason set to max_tokens. The model doesn’t know about the ceiling and doesn’t adjust. Without streaming, set around 16,000; with streaming, up to 64,000; up to 128K only with streaming, or the request runs into an HTTP timeout.

      "output_config": { "effort": "medium" },

    Reasoning depth. On 5.5 models reasoning is on by default; effort from low to max decides how much the model thinks and how many tokens it spends. Reasoning is output tokens at the output price. By default the reasoning text isn’t returned (display: omitted), but it’s billed.

    default:effort high on Sonnet 5.5, medium on Opus 5.5 and Haiku 5.5

      "system": [
        {
          "type": "text",
          "text": "Server configs: Ubuntu 24.04, Traefik 3.7. Commands come with a rollback.",
          "cache_control": { "type": "ephemeral" }
        }
      ],
    Key part

    The system prompt: who the model is here, rules, constant materials. It’s read first, and the model ranks its rules above requests in the conversation. It changes rarely — so add a cache marker: later requests with the same start read it 10 times cheaper (Prompt cache). Any changed byte — a date, a counter — invalidates the cache for everything after it.

      "tools": [
        {
          "name": "read_file",
          "description": "Reads a file from the config repository",
          "input_schema": {
            "type": "object",
            "properties": { "path": { "type": "string" } },
            "required": ["path"]
          }
        }
      ],

    Tools: a name, a description and a JSON input schema. The description is text in the context too, and you pay for it on every request. Twenty tools are several thousand tokens before the question even starts. How the model calls them — Agents and tools.

      "messages": [
        { "role": "user", "content": "Why the 404 on memos?" },
        { "role": "assistant", "content": [
          { "type": "text", "text": "I need the container's labels" }
        ] },

    The history. The API stores nothing: every request carries all past turns. Roles alternate, and the first is user. Return the model’s answer in full, with all its blocks (thinking, tool_use), not just the text — otherwise the reasoning and tool calls are lost.

        {
          "role": "user",
          "content": [
            { "type": "text", "text": "<labels>…</labels>" },
            { "type": "text", "text": "Here are the labels. Where's the mistake?" }
          ]
        }
      ]
    }

    The new question. Long material first, the question last: the model answers more precisely that way (Long document first). Files and images go as separate blocks of the same message.

      "temperature": 0,
      "top_p": 0.9,

    Randomness in token choice. On Claude Opus 4.7 and newer these fields are gone, and a request with them gets 400; Sonnet 5.5 accepts only the default value. Other vendors and local models still have them: 0 means almost always the most likely token, 1 the usual spread.

        { "role": "assistant", "content": "{" }

    Prefill: a final assistant message so the model continues from {. The old way to get JSON. On 4.6 models and newer it returns 400; use structured output, output_config.format, instead (JSON by schema).

    What comes back

    {
      "id": "msg_…",
      "model": "claude-sonnet-5-5",
      "content": [
        { "type": "thinking", "thinking": "", "signature": "…" },
        { "type": "text", "text": "The router has no `entrypoints`…" }
      ],
      "stop_reason": "end_turn",
      "usage": { "input_tokens": 412, "cache_read_input_tokens": 1830, "output_tokens": 690 }
    }

    content is a list of blocks, and the first is often thinking with empty text. Take the answer text with a filter by type, not content[0]: jq -r '.content[] | select(.type == "text") | .text'.

    stop_reason What happened What to do
    end_turn the model finished on its own nothing
    max_tokens hit the ceiling, the answer is cut off raise max_tokens; stream long answers
    tool_use the model wants to call a tool run it, and send the result in the next user message (Agents and tools)
    stop_sequence a configured stop string appeared nothing, that’s by design
    pause_turn a long turn on server tools was paused send the request again; it continues
    refusal a refusal under the safety rules look at stop_details.category; rephrase or use a fallback model

    If it doesn’t work

    400 mentioning temperature after switching to a new model

    Cause
    On Claude Opus 4.7 and newer, temperature, top_p and top_k were removed; Sonnet 5.5 accepts only the default value.
    Fix
    Remove the parameters from the request; tame answer variance with format and checks (Repeatable answers).

    400 on a request that ends with an assistant message

    Cause
    Answer prefill is disabled on 4.6 models and newer.
    Fix
    Set the format with structured output, output_config.format (JSON by schema), or with an instruction in system.

    400 with thinking: {type: "disabled"} on Opus 5.5

    Cause
    Opus 5.5 can’t turn reasoning off at any effort level.
    Fix
    Remove thinking and set effort: "low" (Match reasoning to the task).

    The model “doesn’t see” the previous answer

    Cause
    The program sent only the last question: the API stores no history; the client sends it.
    Fix
    Put every past user and assistant pair into messages, the model’s answer in full, with all its blocks.