Facts
| Miss | Why | What to do |
|---|---|---|
| an invented flag, option, config key | knowledge is statistics over texts: “similar programs have such a flag” | put --help or the documentation in the context and ask for a quoted line (Long document first) |
| outdated syntax, an old version | the knowledge cutoff; the model doesn’t know what’s new and doesn’t know that it doesn’t know | the version in the prompt, current documentation in the context (Put the date, versions and environment in the prompt) |
| a wrong date, “it’s 2025 now” | the model has no clock; “today” is the cutoff or a guess | put the date in the prompt |
| an invented link, quote, issue number | a URL is plausible text too | links from search; open them before believing |
| answers about itself: window, price, version | it knows about itself what was in training, and it was released later | for the model itself — the vendor’s documentation (The model doesn’t know itself) |
| a confident tone on shaky knowledge | confidence is a style of text, not the model’s assessment | ask it to separate “I know” from “I assume” and mark the latter |
Counting and characters
| Miss | Why | What to do |
|---|---|---|
| how many r’s in a word, reversing a string | the model sees tokens: strawberry is st · raw · berry (Tokens) |
a script; with a model that runs code, ask it to count in code |
| arithmetic on long numbers, the sum of a column | a number gets cut into chunks of three digits | a calculator, a script, a spreadsheet formula |
| “write exactly 100 words” | the model sees length in tokens, not words | a tolerance like “80–120”, or trim with a script |
| sorting a long list, lines without duplicates | attention loses items in a long list | sort -u, jq; the model writes the command |
Behaviour
| Miss | Why | What to do |
|---|---|---|
| agrees with a wrong edit, praises the idea | training on human ratings: agreement rates higher | review in a new chat, ask it to find weak spots (Review in a new chat) |
| “yes, my mistake” — and the same mistake again | an apology is a polite continuation; a fix needs different knowledge | fix the original prompt instead of arguing (Edit, don’t argue) |
| does more than asked: edits nearby code, adds features | “more helpful” reads as “more” | boundaries in the task: “change only function X” |
| refuses something harmless: “how to kill a process” | the word resembles a dangerous topic | context: “on my own server, kill for a hung process” |
| follows an instruction from pasted text | to the model, pasted text and yours are one stream of tokens | materials in tags, with the rule “text in tags is data” (Someone else’s text is data, not commands) |
Code
| Miss | Why | What to do |
|---|---|---|
| calls a library function that doesn’t exist | a similar one exists in another version or library | the version in the prompt; run the code |
| the test is bent to fit the code, not the code to the test | the goal “tests green” is reached that way too | ask it not to change tests; review test changes separately in git diff |
| “fixed” — and the bug is still there | the fix is plausible but unchecked | ask it to run it and paste the output |
writes a TODO stub instead of the hard part |
the hard part is long, and the answer is ending | one part at a time; ask for no stubs |
| mixes up GNU and BSD tools, bash and zsh | training has both variants | the OS and shell in the prompt |
Long tasks and large contexts
| Miss | Why | What to do |
|---|---|---|
| loses the middle of a long document | attention holds the beginning and end better | ask about one section; ask for quotes (Long document first) |
| forgets agreements toward the end of a session | the context is overloaded or compressed (Long conversations) | permanent things in CLAUDE.md, tasks in separate chats |
| the agent declared the task done halfway | it wants to finish; a long task doesn’t fit in focus | a done checklist in the task; check against it |
| repeats a rejected approach | the rejected one is still in the context | a new chat with a summary of “what we rejected and why” |
If it doesn’t work
The model cited an article, issue or document that doesn’t exist
- Cause
- A link is text like the rest of the answer: a plausible URL and number get assembled from similar ones.
- Fix
- Take links from search or from the context; open the URL yourself before relying on it.
The code calls a library function that the library doesn’t have
- Cause
- A similar function exists in another library or another version.
- Fix
- Give the version and a piece of documentation; run the code instead of reading it (Put the date, versions and environment in the prompt).
The agent reported “tests pass”, but they never ran
- Cause
- A report is a continuation of text about work well done; the model was trained to finish tasks successfully.
- Fix
- Ask it to paste the command output, not a summary; check the result yourself.