basforge

Search titles, text and the “If it doesn’t work” cases.

    RU

    Ask the documentation, not the model, about its limits, price and version

    A model was trained before it was released, so it knows less about itself than about Docker: it can get its window, price, cutoff, capabilities and even its name wrong. Take such facts from the vendor’s page or the API — Anthropic’s GET /v1/models returns each model’s window and output limit.

    What it looks like

    Question to the model Why the answer is unreliable Where the truth is
    “what’s this model’s context window?” the figures in training are about earlier models the model page, GET /v1/models (max_input_tokens)
    “how much does this model cost?” prices change after release the pricing page (Models as of October 2026)
    “can the model read PDFs?” capabilities depend on the app, not just the model the app’s or API’s documentation
    “which model version is this?” the model knows its name from the system prompt, if someone put it there the API response’s model field
    “will this be remembered next time?” memory is an app feature the model can’t see the app’s memory settings

    Check through the API

    curl -s https://api.anthropic.com/v1/models/claude-sonnet-5-5 \
      -H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" |
      jq '{id, display_name, max_input_tokens, max_tokens}'

    max_input_tokens is the context window, max_tokens the output length limit. The key is handled as in First API request.

    Same goes for the model explaining why it answered that way

    A model composes the explanation of its own answer the same way it composes the answer: it’s a plausible story, not a record of what happened inside. Useful as a hint about where to look for the error, but not as proof. Check the result, not the explanation.