What it looks like
| Question to the model | Why the answer is unreliable | Where the truth is |
|---|---|---|
| “what’s this model’s context window?” | the figures in training are about earlier models | the model page, GET /v1/models (max_input_tokens) |
| “how much does this model cost?” | prices change after release | the pricing page (Models as of October 2026) |
| “can the model read PDFs?” | capabilities depend on the app, not just the model | the app’s or API’s documentation |
| “which model version is this?” | the model knows its name from the system prompt, if someone put it there | the API response’s model field |
| “will this be remembered next time?” | memory is an app feature the model can’t see | the app’s memory settings |
Check through the API
curl -s https://api.anthropic.com/v1/models/claude-sonnet-5-5 \
-H "x-api-key: $ANTHROPIC_API_KEY" -H "anthropic-version: 2023-06-01" |
jq '{id, display_name, max_input_tokens, max_tokens}'
max_input_tokens is the context window, max_tokens the output length limit. The key is handled as in
First API request.
Same goes for the model explaining why it answered that way
A model composes the explanation of its own answer the same way it composes the answer: it’s a plausible story, not a record of what happened inside. Useful as a hint about where to look for the error, but not as proof. Check the result, not the explanation.