basforge

Search titles, text and the “If it doesn’t work” cases.

    RU
    Cheat sheetAILanguage models

    Where models fail: the miss, the cause, the fix

    Model misses repeat, and almost all of them follow from how models work: they continue text plausibly, see tokens instead of letters, know the world up to a cutoff, and are trained to please. By group: what a miss looks like, where it comes from, and what cures it.

    Facts

    Miss Why What to do
    an invented flag, option, config key knowledge is statistics over texts: “similar programs have such a flag” put --help or the documentation in the context and ask for a quoted line (Long document first)
    outdated syntax, an old version the knowledge cutoff; the model doesn’t know what’s new and doesn’t know that it doesn’t know the version in the prompt, current documentation in the context (Put the date, versions and environment in the prompt)
    a wrong date, “it’s 2025 now” the model has no clock; “today” is the cutoff or a guess put the date in the prompt
    an invented link, quote, issue number a URL is plausible text too links from search; open them before believing
    answers about itself: window, price, version it knows about itself what was in training, and it was released later for the model itself — the vendor’s documentation (The model doesn’t know itself)
    a confident tone on shaky knowledge confidence is a style of text, not the model’s assessment ask it to separate “I know” from “I assume” and mark the latter

    Counting and characters

    Miss Why What to do
    how many r’s in a word, reversing a string the model sees tokens: strawberry is st · raw · berry (Tokens) a script; with a model that runs code, ask it to count in code
    arithmetic on long numbers, the sum of a column a number gets cut into chunks of three digits a calculator, a script, a spreadsheet formula
    “write exactly 100 words” the model sees length in tokens, not words a tolerance like “80–120”, or trim with a script
    sorting a long list, lines without duplicates attention loses items in a long list sort -u, jq; the model writes the command

    Behaviour

    Miss Why What to do
    agrees with a wrong edit, praises the idea training on human ratings: agreement rates higher review in a new chat, ask it to find weak spots (Review in a new chat)
    “yes, my mistake” — and the same mistake again an apology is a polite continuation; a fix needs different knowledge fix the original prompt instead of arguing (Edit, don’t argue)
    does more than asked: edits nearby code, adds features “more helpful” reads as “more” boundaries in the task: “change only function X”
    refuses something harmless: “how to kill a process” the word resembles a dangerous topic context: “on my own server, kill for a hung process”
    follows an instruction from pasted text to the model, pasted text and yours are one stream of tokens materials in tags, with the rule “text in tags is data” (Someone else’s text is data, not commands)

    Code

    Miss Why What to do
    calls a library function that doesn’t exist a similar one exists in another version or library the version in the prompt; run the code
    the test is bent to fit the code, not the code to the test the goal “tests green” is reached that way too ask it not to change tests; review test changes separately in git diff
    “fixed” — and the bug is still there the fix is plausible but unchecked ask it to run it and paste the output
    writes a TODO stub instead of the hard part the hard part is long, and the answer is ending one part at a time; ask for no stubs
    mixes up GNU and BSD tools, bash and zsh training has both variants the OS and shell in the prompt

    Long tasks and large contexts

    Miss Why What to do
    loses the middle of a long document attention holds the beginning and end better ask about one section; ask for quotes (Long document first)
    forgets agreements toward the end of a session the context is overloaded or compressed (Long conversations) permanent things in CLAUDE.md, tasks in separate chats
    the agent declared the task done halfway it wants to finish; a long task doesn’t fit in focus a done checklist in the task; check against it
    repeats a rejected approach the rejected one is still in the context a new chat with a summary of “what we rejected and why”

    If it doesn’t work

    The model cited an article, issue or document that doesn’t exist

    Cause
    A link is text like the rest of the answer: a plausible URL and number get assembled from similar ones.
    Fix
    Take links from search or from the context; open the URL yourself before relying on it.

    The code calls a library function that the library doesn’t have

    Cause
    A similar function exists in another library or another version.
    Fix
    Give the version and a piece of documentation; run the code instead of reading it (Put the date, versions and environment in the prompt).

    The agent reported “tests pass”, but they never ran

    Cause
    A report is a continuation of text about work well done; the model was trained to finish tasks successfully.
    Fix
    Ask it to paste the command output, not a summary; check the result yourself.