basforge

Search titles, text and the “If it doesn’t work” cases.

    RU

    Non-urgent API requests go as a batch at half price

    Labelling three thousand notes, translating an archive, running a check over every page — none of it needs to happen in a second. The batch APIs from Anthropic, OpenAI and Google cost half the regular price and answer within 24 hours, usually within an hour.

    How it works

    1. send a batch of requests, each with its own custom_id
    2. wait: processing_status goes in_progress … ended
    3. collect the results; they come in any order, match them by custom_id

    A request inside a batch is the same as a regular one: model, system, messages, tools, cache. Only the delivery differs.

    How much it saves

    Labelling 3,000 notes at 600 input tokens and 20 output tokens each on Haiku 5.5:

    regular API:  3000 × (600 × $0.10 + 20 × $0.50) / 1M ≈ $0.21
    batched:      ≈ $0.105

    On Haiku that’s pocket change either way. On Opus 5.5 with long input the same job costs tens of dollars, and half of that stays in your pocket.

    When it doesn’t fit

    • You need the answer now: a chat, an agent, a reply to a user.
    • The next request depends on the previous answer — a batch can’t do chains.
    • A batch of five requests: the saving is smaller than the time spent on the code.

    Batching and caching stack: the shared start of every request in the batch (instructions and examples) is read from the cache.