Token Budget Lab
Free online token counter for OpenAI o200k_base and cl100k_base. Paste a prompt or a whole conversation JSON to see tokens per message, how much of the context window each one eats, and exactly which messages get dropped when you exceed the limit.
Token Budget Lab
Free online token counter for OpenAI o200k_base and cl100k_base. Paste a prompt or a whole conversation JSON to see tokens per message, how much of the context window each one eats, and exactly which messages get dropped when you exceed the limit.
About the token budget lab
How it works
Your text is encoded with the same byte-pair encoding the model uses, so the split you see is the real one, not an estimate from character counts. If the input parses as a messages array, each message is counted separately and its share of the total is shown. The truncation preview then walks the messages in order, discarding until the remainder fits your limit - which is why the result often stops short of the limit rather than landing on it.
When to reach for it
Deciding how many rules a system prompt can still hold. Working out how many turns of history to keep before the oldest ones silently disappear. Explaining why a request that worked yesterday now returns a context length error. Checking whether a long tool result is worth its share of the window before you wire it in.
Privacy
Everything is encoded locally in your browser, inside a Web Worker. Nothing you paste is uploaded, logged or sent to any model provider. Prompts routinely contain credentials, customer data and unreleased product details, so this is not a setting - there is no server side to send them to.
FAQ
Common questions and answers about this topic.
Why does my count differ from what the provider bills me?
This counts the content you paste. A real API call also spends tokens on the chat format itself - a few per message for role markers and separators - plus any tool or function schemas you attach, and of course the model's own reply. Treat the number here as the floor for your input, not the final bill.
Do different models produce different token counts for the same text?
Yes, and often by a lot. GPT-4o era models use o200k_base while GPT-4 and GPT-3.5 use cl100k_base; the newer vocabulary is markedly more efficient for CJK text. Anthropic and Llama models use entirely different tokenizers, which is why this tool does not offer them rather than showing a number that would be wrong.
What actually happens when a conversation exceeds the context window?
It depends on who is holding the conversation. A raw API call simply fails with a context length error. Frameworks and chat clients usually trim first - most often dropping the oldest messages, sometimes pinning the system prompt, sometimes summarising instead. The strategy switch here lets you see each of those outcomes against your own conversation.
How many tokens is one Chinese or Japanese character?
There is no fixed ratio, which is exactly why guessing goes wrong. Common words often merge into a single token while rare characters can cost several, and o200k_base is considerably more efficient here than cl100k_base. Paste a representative sample and switch encodings to see the difference on your own text rather than relying on a rule of thumb.
Does the system prompt count against the context window?
Yes. It is an ordinary message as far as the budget is concerned, and a long one competes with your conversation history for the same space. What is special about it is only that most clients refuse to drop it, which is what the keep system strategy models here - the cost of pinning it is that more history has to go instead.
Is anything I paste uploaded anywhere?
No. The encoding vocabulary is downloaded to your browser and the counting happens in a Web Worker on your machine. There is no request carrying your text, no logging and no model provider involved - which matters because prompts routinely carry credentials and customer data.