How the token counter works
Paste text into the editor above. The token counter splits your prompt using the selected tokenizer, counts the resulting tokens and displays their numeric IDs. The result updates automatically, so you can shorten a prompt and immediately see how its token count changes.
The default o200k_base encoding counts raw text without adding a chat wrapper. Use the selector at the top to switch to cl100k_base or another supported encoding. Hover over a colored token to connect its text with its ID, and turn on Show whitespace to inspect spaces and line breaks.
For example, “Hello world! 你好，世界！” contains 8 tokens with o200k_base and 11 with cl100k_base. Paste the text inside the quotation marks into this token counter and switch encodings to compare the token IDs. The same text produces different token counts because each encoding splits it differently.
Token counter vs word counter
A word counter measures words; a token counter measures the units produced by a model's tokenizer. A token can represent a whole word, part of a word, punctuation or whitespace. Code, numbers, emoji and multilingual text can split differently even when their word counts look similar.
There is no fixed word-to-token ratio that works for every prompt and model. Count the actual text with the appropriate encoding when comparing prompt length or planning context usage. Leave room for conversation history, tool definitions and the model's response when checking a context limit.
Token count by model
The same prompt can have different token counts across models because vocabularies and request formats differ. This comparison uses the same model and accuracy data as the model directory. Follow a model link for its counting method and limitations.
Exact base text means the published tokenizer counts the text you enter. Compatible means the encoding is useful for inspection but is not a guarantee of complete request usage. Exact via API requires the provider's counting endpoint; those rows do not imply that its tokenizer runs in this browser.
| Model | Encoding / tokenizer | Is the count exact? |
|---|---|---|
| GPT-5.6 | o200k_base (compatible view) | Compatible |
| DeepSeek V4 Flash | Published DeepSeek V4 tokenizer | Exact base text |
| Kimi K3 | Published Kimi K3 vocabulary | Exact base text |
| Grok 4.5 | Provider tokenizer via xAI TokenizeText | Exact via API |
| Gemini 3.6 Flash | Provider tokenizer via Gemini countTokens | Exact via API |
| Claude Sonnet 5 | Provider tokenizer via Anthropic count_tokens | Exact via API |
| DeepSeek R1 | Published DeepSeek R1 tokenizer | Exact base text |
| GPT-OSS 20B & 120B | o200k_base (Harmony wrapper excluded) | Compatible base text |