basforge

Search titles, text and the “If it doesn’t work” cases.

    RU
    Topic · AI

    Language models

    How models work, which ones exist, where they fail

    Entries
    8
    Cases
    13
    Updated
    2026-10-13
    Series · where to startOverview

    Language models: how they work, what kinds exist and how they differ

    A language model continues text: from everything in its context it picks the next token, then the next. Knowledge is smeared across the weights rather than stored in a database, so the model writes plausibly, not necessarily correctly. Models differ by tier (price and strength), reasoning, context window, what they accept as input, and where they run — at a provider or on your own hardware.

    Read the overview
    1. Who’s who · Cheat sheetModels as of October 2026: families, tiers, prices and windowsFamilies and tiers as of October 2026: prices per million tokens, windows, open models.
    2. Task and model · Cheat sheetTask and model: which tier and mode for translation, code, logs and documentsTranslation, code, log analysis, data extraction — which tier and mode to use.
    3. Tokens and money · Manualfrom Tokens and costTokens: measuring text size and the price of a requestWhat a token is, why some languages cost more than English, and how to price a request before sending it.
    4. A request, block by block · Configfrom Model APIsA model request, block by block: what goes out every timesystem, history, tools, max_tokens: what goes to the model in every request.
    5. Long conversations · Manualfrom PromptsLong conversations: how not to hit the context limit and not lose what you agreed onWhy the model gets duller toward the end of a chat and how to move work to a fresh context without losses.
    6. A prompt, block by block · Configfrom PromptsA prompt, block by block: context, materials, task, format, examplesRole, materials, task, format, examples — what goes in each block and why.
    7. Reasoning · Tipfrom PromptsMatch reasoning to the task: an effort level instead of “think harder”When to turn on reasoning and the effort level, what it costs and when it doesn’t help.
    8. Where they fail · Cheat sheetWhere models fail: the miss, the cause, the fixInvented flags, counting letters, flattery, outdated versions: the cause of each miss and what to do.
    9. First API request · Manualfrom Model APIsFirst API request: a key that stays out of history, curl, the response and the billA key that stays out of history, curl, reading the response, stop_reason and usage, errors 429 and 529.
    10. Your own data · OverviewYour own data in a model: context, RAG or fine-tuningDocuments in context, RAG or fine-tuning: what to choose for your own knowledge base.
    11. Agents · Overviewfrom AI agentsAgents and tools: how a model reads files and runs commandsHow a model calls tools, what MCP is and how much access to give an agent.
    12. Your own hardware · Manualfrom Local modelsA model on your own hardware: Ollama, an API on port 11434 and a web UIOllama on a computer or server: a model, an API on port 11434, a web UI.
    13. Memory for a model · Cheat sheetfrom Local modelsHow much memory a model needs: parameters, quants, context and speedHow many gigabytes an 8B or 70B model takes at Q4 and how much the context eats.
    14. Glossary · Cheat sheetGlossary: language model terms in one line eachToken, quant, MoE, RAG, LoRA and fifty more words — one line each.

    Tips

    18
    from PromptsNew task, new chatA conversation about Traefik, then backups, then an email to a colleague, all in one chat, is a context where every new task drags the tails of the earlier ones along: other files, cancelled decisions, tone.from PromptsWrong answer? Edit your question instead of arguing with the answerIf an answer went the wrong way, edit the original message and resend it instead of writing “no, not like that” as the next turn.from PromptsNeed a particular kind of answer? Show an example instead of describing itOne example of the answer you want sets length, tone, detail and format more precisely than a paragraph of description.from PromptsA rule in a prompt gets a reason instead of capsA newer model executes “NEVER change the ports” literally and more broadly than needed.from PromptsLong document first, question last, answer with quotesPut a document of dozens of pages at the start of the prompt and the question after it, and ask the model first to write out the quotes its answer relies on.from PromptsPut the date, versions and environment in the promptThe model doesn’t know today’s date, which version of a program you have, or what came out after its knowledge cutoff.Ask the documentation, not the model, about its limits, price and versionA model was trained before it was released, so it knows less about itself than about Docker: it can get its window, price, cutoff, capabilities and even its name wrong.from PromptsA big task: questions and a plan first, then the workBefore a big task, ask the model not to act but to ask what’s missing and propose a plan.from PromptsReview in a new chat, without your version, asking for mistakesA model tends to agree: in the same chat where it wrote the answer, or where you praised it, a review will confirm it.Keep passwords, keys and personal data out of the chatEverything pasted into a chat or read by an agent goes to the provider and stays there: in history, in logs, and on free and consumer plans sometimes in training data.from AI agentsSomeone else’s text is data, not commands: prompt injectionTo a model, your question and pasted text are one stream of tokens.from PromptsConsistent answers come from format and checks, not temperature 0A model picks the next token with some randomness, so one question gives different wordings.from Model APIsNeed JSON? Give a schema instead of asking “answer in JSON”“Answer in JSON format” sometimes gives JSON wrapped in explanations, with a stray comma or a missing field.from Tokens and costThe constant part of a prompt goes first, under the cacheThe prompt cache keeps the start of a request for 5 minutes (or an hour): a repeat request with the same start reads it at 10–40 times less than regular input and answers faster.from Tokens and costNon-urgent API requests go as a batch at half priceLabelling three thousand notes, translating an archive, running a check over every page — none of it needs to happen in a second.from Tokens and costPrice a model by the solved task, not the tokenA cheap model that needs three attempts and your correction costs more than a strong one that got it right the first time.from PromptsWhich language to prompt in: non-English costs more tokens, not more misunderstandingThe same text in Russian takes 1.16 times as many tokens as in English on the o200k_base vocabulary; on the 2023 vocabulary it was 1.64 times.from Tokens and costA subscription for hands-on work, the API for programsA chat subscription (claude.ai, ChatGPT, Gemini) is a fixed monthly sum for work in the app and the agent, with a message limit per time window.