basforge

Search titles, text and the “If it doesn’t work” cases.

    RU
    Cheat sheetAILanguage models

    Glossary: language model terms in one line each

    Words that show up in documentation, price lists and discussions of models — grouped, with a link to the page where the topic is covered in detail.

    How it works

    Term What it means
    LLM a large language model: a neural network that continues text token by token (Language models)
    token a piece of text, the unit of counting: about 4.2 characters on average in English (Tokens)
    tokenizer the vocabulary of pieces and the cutting rule; each family has its own
    parameters, weights the numbers inside the network; 8B is 8 billion
    transformer the architecture of nearly every LLM: attention layers and feed-forward layers
    attention the mechanism that links every token to every previous one
    context, context window everything the model takes into account in one request, and its limit in tokens (A model request, block by block)
    inference running a finished model to generate an answer, as opposed to training
    MoE mixture of experts: part of the subnetworks works on each token; 30B-A3B means 3B active out of 30B (How much memory a model needs)
    multimodal model accepts more than text: images, PDFs, audio
    embedding a vector of hundreds of numbers, close for texts close in meaning (Your own data in a model)

    Training

    Term What it means
    pretraining training to predict the next token on trillions of tokens of text
    knowledge cutoff the last thing that made it into the data: the model knows nothing later (Put the date, versions and environment in the prompt)
    base model after pretraining: continues text but doesn’t answer questions
    instruct, chat model fine-tuned to answer in a dialogue
    SFT fine-tuning on “question — good answer” examples
    RLHF reinforcement learning from human ratings; the source of politeness and flattery
    RLAIF, constitutional AI the same, but a model grades by written principles
    reasoning model trained to write its thinking before the answer (Match reasoning to the task)
    distillation a small model learns from a big one’s answers
    fine-tuning further training of a finished model on your own examples (Your own data in a model)
    LoRA cheap fine-tuning: you train a small add-on to the weights, not all the weights

    Request and response

    Term What it means
    system prompt instructions for the whole conversation, with priority over turns (A model request, block by block)
    prompt the whole request text; narrowly, your instruction (A prompt, block by block)
    few-shot a few example answers right in the prompt (An example over a description)
    sampling picking the next token from the probabilities with some randomness
    temperature, top_p, top_k knobs for that randomness; removed on newer Claude models (Repeatable answers)
    max_tokens the ceiling on answer length, required in the API
    stop_reason why the model stopped: end_turn, max_tokens, tool_use (A model request, block by block)
    effort Claude’s reasoning depth: low … max (Match reasoning to the task)
    streaming the text arrives in chunks as it’s generated
    structured output an answer strictly by a JSON schema (JSON by schema)
    prefill writing the start of the answer for the model; disabled on newer Claude models
    prompt cache a repeated request prefix is read 10–40 times cheaper (Prompt cache)
    batch processing requests in a batch, answered within 24 hours, at half price (Non-urgent API requests go as a batch at half price)
    compaction replacing the old part of the history with a summary as the window fills up (Long conversations)
    TTFT time to first token of the answer
    tokens per second generation speed (How much memory a model needs)

    Size and running

    Term What it means
    open weights the weights can be downloaded and run locally; training data is usually closed
    quantization, quant storing weights with fewer bits: Q4_K_M, Q8_0 (How much memory a model needs)
    GGUF the model file format for llama.cpp and Ollama
    llama.cpp an engine that runs models on CPU and GPU; Ollama is built on it
    Ollama a program that downloads models, runs them and serves an API on 11434 (A model on your own hardware)
    vLLM a server for multi-user GPU inference that handles many requests at once
    VRAM video memory: the model and its context must fit in it (How much memory a model needs)
    KV cache memory for the context, growing with every token (How much memory a model needs)
    speculative decoding a small model guesses tokens and a big one verifies them in a batch — faster
    Term What it means
    RAG retrieving document chunks before answering and putting them into the prompt (Your own data in a model)
    vector database stores embeddings and finds the nearest ones: pgvector, Qdrant
    chunk a piece of a document for the index, usually 300–800 tokens
    reranker a model that re-sorts search results by closeness to the question
    BM25 classic keyword search; finds exact names that vector search misses
    grounding, citation an answer backed by a source with a pointer to the place in it (Long document first)

    Agents

    Term What it means
    tool, function calling a function the model can ask to have called (Agents and tools)
    agent a model with tools in a “call — result” loop until the task is done
    MCP an open standard for connecting tools to agents
    server tool runs at the provider: web search, code execution
    subagent an agent the main one hands part of the task to, with its own clean context
    sandbox an isolated environment where the agent is allowed anything
    CLAUDE.md, AGENTS.md a file in the repository with standing instructions for the agent (Long conversations)

    Quality and risks

    Term What it means
    hallucination a plausible but wrong answer (Where models fail)
    sycophancy agreeing with the user against the facts (Review in a new chat)
    prompt injection instructions hidden in the input: a web page, a file, an email (Someone else’s text is data, not commands)
    jailbreak an attempt to get around the model’s rules by wording
    refusal the model won’t answer under its safety rules; in the API, stop_reason: refusal
    benchmark a set of tasks for comparing models; some of it may have leaked into training
    eval your own task set for testing a model or prompt on your own work
    data contamination test tasks ended up in the training data, inflating the score