basforge

Search titles, text and the “If it doesn’t work” cases.

    RU

    Language models: how they work, what kinds exist and how they differ

    A language model continues text: from everything in its context it picks the next token, then the next. Knowledge is smeared across the weights rather than stored in a database, so the model writes plausibly, not necessarily correctly. Models differ by tier (price and strength), reasoning, context window, what they accept as input, and where they run — at a provider or on your own hardware.

    What problem it solves

    A model takes text and writes a continuation that fits its meaning. If the text is a question, the continuation looks like an answer. If the text is a config plus “why the 502”, the continuation is an analysis of the config. Hence the whole range of tasks: summarize, translate, explain, write code, read a log, pull fields from an email into JSON, and with tools, run commands too.

    Where a model is strong

    • text to text: summaries, translation, rewriting in another tone
    • code from a description, explaining someone else’s code, finding a bug from a trace
    • working through what’s in the context: a log, a config, a 300-page document
    • a draft, a plan, options — things a person checks afterwards

    Where a model is weak

    • exact facts that aren’t in the context: versions, flags, prices, dates
    • counting and character-level work: how many r’s in “strawberry”, the sum of a column
    • “today”, “the latest version” — without a date and search, the model lives at its cutoff
    • work you can’t check: a plausible mistake looks just like the truth

    How it works: one token per step

    request text

    1. Tokens“Reconfiguring” = Re · config · uringText is cut into pieces — tokens. English text averages 4.2 characters per token, Russian 3.4 (o200k_base, measured on this site’s pages). The model sees token numbers, not letters.
    2. Contextsystem + history + filesEverything the model takes into account: the system prompt, the whole conversation, attached files, tool descriptions. There’s no memory between requests — everything is sent again every time (A model request, block by block).
    3. Modelbillions of weightsA transformer neural network. Attention links every token to every previous one, so the model sees that “it” at the end of a paragraph is the container from the beginning.
    4. Probabilities“container” 0.41 · “service” 0.22 · …The output is a probability for every token in the vocabulary (o200k_base has about 200,000): what the next one could be.
    5. Pick and repeatsamplingOne token is taken — not always the most likely, with some randomness, which is why one question yields different wordings (Repeatable answers). The token is appended to the context and everything repeats: a 1,000-token answer is a thousand passes of the model. Hence speed measured in tokens per second, and output priced higher than input.

    answer

    There’s no knowledge “in a database”: what the model read during training is dissolved in the weights as statistics of “what usually follows what”. The common stuff — Docker, nginx, Python — it knows well. The rare stuff — a flag of a program with 200 GitHub stars — it reconstructs from similar ones and may invent. That’s a hallucination: the same plausible continuation, only wrong.

    How a model is made: three stages

    Stage What happens What it gives and what it costs
    Pretraining the model learns to continue texts: trillions of tokens from the web, books, code knowledge and language; the cutoff is the last thing that made it into the data
    Supervised fine-tuning (SFT) “question — good answer” examples written by people and models the model answers instead of continuing a question with another question
    Reinforcement learning ratings of answers by people and grader models; tasks with checkable answers — code tests, math politeness, refusals, step-by-step reasoning; as a side effect, flattery and eagerness to please

    The last stage produced reasoning models: before answering, the model writes a draft of its thinking, and on tasks with checkable answers that gives a noticeable gain. Reasoning is the same output tokens, and you pay for them (Match reasoning to the task).

    How models differ

    Axis What it means Where to look
    Tier every vendor has a flagship, a mid-size and a small model: Claude Opus and Haiku, Gemini Pro and Flash. Opus 5.5 costs 40 times more than Haiku 5.5 Models as of October 2026
    Reasoning whether the model thinks before answering and how much: effort from low to max Match reasoning to the task
    Context window how many tokens fit in one request: 1M for Claude 5.5, tens of thousands for a local model on 16 GB of VRAM Long conversations, How much memory a model needs
    Input text only, or also images, PDFs, audio, video Task and model
    Tools can it call functions, search the web, run code Agents and tools
    Cutoff up to which month the model knows the world Put the date, versions and environment in the prompt
    Where it runs at a provider via API and app — or on your own hardware with open weights A model on your own hardware
    Speed tokens per second and latency to the first token; small models are several times faster Models as of October 2026
    Character verbosity, flattery, how often it refuses, how well it holds a format. Tables don’t show it — only your own tasks do Review in a new chat

    Benchmarks compare models on someone else’s tasks, and some of the tests leaked into training data. They’re a rough guide. What decides is running your own ten typical tasks on two or three models.

    Kinds of models besides language models

    This set

    Language models (LLMs)

    Text in and out; modern ones also take images and PDFs as input. Claude, GPT, Gemini, Qwen, DeepSeek.

    Image generation

    A separate class — diffusion models: they draw from noise according to a description. Text in images and the exact number of fingers are their weak spots. Chat apps call them as a tool.

    Speech

    Recognition (speech to text, Whisper and the like) and synthesis (text to voice). A chat’s voice mode is a chain of these and a language model, or one model with audio in and out.

    Embeddings

    They turn text into a vector of hundreds of numbers so that texts with similar meaning give similar vectors. The basis of semantic search and RAG (Your own data in a model).

    Classifiers and rerankers

    Small models for one task: spam or not, which of ten documents is closest to the question. Hundreds of times cheaper than an LLM, and they answer in milliseconds.

    Other ways to solve these tasks

    How it works When it beats a model
    Search and documentation the exact text from the primary source you need a fact: a version, a flag, a price. A model with search is the same search plus a retelling
    A script, grep, jq, regex deterministic, the same on a million lines the task is formal and repeats: pick a field from JSON, count lines
    Classic ML a small model trained on your own examples thousands of uniform decisions per second, predictability required
    A person knows the context, is accountable for the decision the cost of a mistake is high and there’s no way to check the result
    If… Use
    text, code, analyzing a document, a draft you’ll check later a mid-tier language model; a flagship for hard cases
    you need a fresh fact search, or a model with search and a link in the answer
    one uniform operation over thousands of lines a script; a model to write that script
    data must not go to a provider an open model on your own hardware (A model on your own hardware)
    your own document base that you ask about every day a model plus search over it — RAG (Your own data in a model)

    What you get and what it costs

    What you get

    • Draft speed. A compose file, a script, a migration plan — in a minute instead of an hour; the checking stays with you.
    • Reading large volumes. A 1M-token window holds around 4 million characters of English text (4.2 characters per token in o200k_base; Claude has its own tokenizer) — several books or the whole code of a mid-size project.
    • Explanations at your level. You can ask a fifth time and request an example built on your own config.
    • Working with unstructured input. An email, a screenshot of an error, a PDF invoice — into JSON fields, with no parsing templates (JSON by schema).

    What it costs

    • Plausible mistakes. An invented flag looks as confident as a real one. Test everything that goes to a server by action (Where models fail).
    • Money for tokens. In an hour of work an agent sends the history hundreds of times: without caching that’s tens of dollars (Prompt cache, Price of a solved task).
    • Data goes to the provider. Everything in the chat sits with them; keys and passwords don’t go there (No secrets in the chat).
    • Different answers to the same thing. Token selection is random; repeatability comes only from checks (Repeatable answers).
    • Obsolescence. Models and prices change every few months; the Models as of October 2026 table ages first.

    Where to start

    1. Overviewthis pageLanguage models: how they work, what kinds exist and how they differ
    2. Cheat sheetModels as of October 2026: families, tiers, prices and windowsFamilies and tiers as of October 2026: prices per million tokens, windows, open models.
    3. Cheat sheetTask and model: which tier and mode for translation, code, logs and documentsTranslation, code, log analysis, data extraction — which tier and mode to use.
    4. ManualTokens and costTokens: measuring text size and the price of a requestWhat a token is, why some languages cost more than English, and how to price a request before sending it.
    5. ConfigModel APIsA model request, block by block: what goes out every timesystem, history, tools, max_tokens: what goes to the model in every request.
    6. ManualPromptsLong conversations: how not to hit the context limit and not lose what you agreed onWhy the model gets duller toward the end of a chat and how to move work to a fresh context without losses.
    7. ConfigPromptsA prompt, block by block: context, materials, task, format, examplesRole, materials, task, format, examples — what goes in each block and why.
    8. TipPromptsMatch reasoning to the task: an effort level instead of “think harder”When to turn on reasoning and the effort level, what it costs and when it doesn’t help.
    9. Cheat sheetWhere models fail: the miss, the cause, the fixInvented flags, counting letters, flattery, outdated versions: the cause of each miss and what to do.
    10. ManualModel APIsFirst API request: a key that stays out of history, curl, the response and the billA key that stays out of history, curl, reading the response, stop_reason and usage, errors 429 and 529.
    11. OverviewYour own data in a model: context, RAG or fine-tuningDocuments in context, RAG or fine-tuning: what to choose for your own knowledge base.
    12. OverviewAI agentsAgents and tools: how a model reads files and runs commandsHow a model calls tools, what MCP is and how much access to give an agent.
    13. ManualLocal modelsA model on your own hardware: Ollama, an API on port 11434 and a web UIOllama on a computer or server: a model, an API on port 11434, a web UI.
    14. Cheat sheetLocal modelsHow much memory a model needs: parameters, quants, context and speedHow many gigabytes an 8B or 70B model takes at Q4 and how much the context eats.
    15. Cheat sheetGlossary: language model terms in one line eachToken, quant, MoE, RAG, LoRA and fifty more words — one line each.

    If it doesn’t work

    The model confidently named a flag or option that doesn’t exist

    Cause
    A model’s knowledge is statistics over texts, not a reference book. The plausible continuation “similar programs have such a flag” beats “I don’t know”.
    Fix
    Put the documentation or the --help output into the context and ask for an answer citing a line from it (Long document first, Where models fail).

    The answer breaks off mid-word

    Cause
    The output limit ran out: in the API stop_reason is max_tokens; chat apps have their own reply length limit.
    Fix
    In the API, raise max_tokens and turn on streaming; in chat, write “continue from where you stopped” (First API request).

    Toward the end of a long chat the model forgets what was agreed

    Cause
    The context is full: early messages are compressed into a summary or drown among a hundred later ones, and mistakes and abandoned ideas are in the context too.
    Fix
    Ask for a summary of decisions and open a new chat with it (Long conversations, New task, new chat).

    It suggests syntax from an old version of a program

    Cause
    The model has a knowledge cutoff: it doesn’t know what came out later and doesn’t know that it doesn’t know.
    Fix
    Put the version in the prompt and give it a piece of current documentation (Put the date, versions and environment in the prompt).

    It agrees with any edit you make, even a wrong one

    Cause
    The model was fine-tuned on human ratings, and people like agreement. Hence the habit of giving in.
    Fix
    Review in a new chat without your version in the context, and ask it to find mistakes rather than confirm (Review in a new chat).

    The same question gets different answers

    Cause
    The next token is chosen with some randomness; on newer Claude models temperature can’t be set at all.
    Fix
    Get repeatability from format and checks, not from hoping for identical text (Repeatable answers).