How the LLM token calculator works
Paste the text you want to measure and choose a supported tokenizer. The token total updates as you edit, while the colored pieces and numeric IDs show where the count comes from. A raw encoding measures exactly the text in the editor; a compatible model view needs the accuracy boundary shown on the relevant model guide.
Use the budget controls below the editor to add input that is outside the pasted text, reserve an output allowance and enter a context limit. The initial values are editable planning examples, not model specifications. Choose the actual limit of your deployed model and include the conversation history and request components your application will send.
LLM token calculator vs word counter
A word count describes the draft, but a context budget needs the units processed by the selected tokenizer. Source code, logs and multilingual documents can produce very different token totals for the same number of words. Counting the actual text helps you decide whether to shorten examples, remove repetition or split a document into smaller inputs.
The budget calculation is prompt tokens plus additional input tokens plus reserved output tokens. For a 1,000-token prompt, 200 additional input tokens and a 500-token output reserve, the planned total is 1,700. Against a 4,096-token limit, that leaves 2,396 tokens. This is arithmetic on your assumptions, not a forecast of generated output.
LLM token count by model
Match the counting method to the model family before comparing budgets. The directory table below shares its model and encoding data with the main model directory. Exact base text covers the supplied text under a published tokenizer; compatible views and provider-only models need extra checks before you rely on the number for a full request.
Recalculate after changing a model, template, tool definition or document. If the provider exposes a structured input-counting endpoint, use that result to validate your input allowance. The calculator does not automatically identify hidden provider formatting, select a model's maximum context, or determine future output and reasoning length.
| Model | Encoding / tokenizer | Is the count exact? |
|---|---|---|
| GPT-5.6 | o200k_base (compatible view) | Compatible |
| DeepSeek V4 Flash | Published DeepSeek V4 tokenizer | Exact base text |
| Kimi K3 | Published Kimi K3 vocabulary | Exact base text |
| Grok 4.5 | Provider tokenizer via xAI TokenizeText | Exact via API |
| Gemini 3.6 Flash | Provider tokenizer via Gemini countTokens | Exact via API |
| Claude Sonnet 5 | Provider tokenizer via Anthropic count_tokens | Exact via API |
| DeepSeek R1 | Published DeepSeek R1 tokenizer | Exact base text |
| GPT-OSS 20B & 120B | o200k_base (Harmony wrapper excluded) | Compatible base text |