OpenAI and tiktoken encodings

Tiktoken Online: OpenAI Tokenizer

Inspect token boundaries and IDs across current GPT models and raw o200k_base, cl100k_base, p50k_base, r50k_base, and GPT-2 encodings.

Accuracy note

Raw encoding selections are exact for the selected encoding. Model selections are compatible views and may not include complete API request overhead.

Frequently asked questions

Which tiktoken encodings are available?

The tool includes o200k_base, cl100k_base, p50k_base, p50k_edit, r50k_base, and GPT-2.

Can I see individual token IDs?

Yes. Token segments and IDs are linked by hover highlighting.