GPT Token Calculator

How the GPT token calculator works

Paste your prompt and select a raw encoding to count exactly the text you entered. The default o200k_base view shows both the number of tokens and their numeric IDs. Selecting a GPT model preset provides its labeled compatible view; it does not guarantee that all provider-side request formatting is present in the editor.

Use the budget controls to reserve space for a response and add input that is not already counted, such as omitted history or known template overhead. Enter the context limit of your exact model. The initial 8,192-token setting is only an editable example, and the calculator does not infer a model limit from its name.

# pip install tiktoken
import tiktoken

prompt = "Hello world! 你好，世界！"
for name in ["o200k_base", "cl100k_base"]:
    encoding = tiktoken.get_encoding(name)
    ids = encoding.encode(prompt)
    print(name, len(ids), ids)
# Raw text only; no complete API request wrapper.

GPT token calculator vs word counter

A word-count shortcut cannot explain how an encoding handles punctuation, numbers, code or multilingual text. Count the actual draft when trimming a GPT prompt. Shortening one sentence may save fewer tokens than removing a repeated instruction, a long example or an unnecessary block of structured data.

For budget planning, add measured prompt tokens, additional input and reserved output. A prompt of 800 tokens, 100 tokens of extra input and a 400-token output reserve needs 1,300 tokens of planned space. A 4,096-token context budget would leave 2,796. These allowances are choices for planning, not predictions of what the response will consume.

GPT token count by encoding and model

Use the encoding comparison to understand a vocabulary and the linked model guide to understand a preset. For the mixed-language sample in the Python example, o200k_base produces 8 tokens while cl100k_base produces 11. Those are raw-text results; a structured request can contain more than that sample sentence.

Before a production request, confirm the current model ID and the provider's supported input-counting method. Chat roles, tool schemas, images and response accounting are not interchangeable with a raw token list. Recount when the model or message format changes, and inspect returned usage when measuring the completed interaction.

Tiktokenizer Reference Sources

Tokenizer behavior and model limits change. Verify production decisions with current provider documentation:

• OpenAI: tiktoken library and examples
• OpenAI Cookbook: Count tokens with tiktoken