How to Count Claude Sonnet 5 Tokens
Call Anthropic's Messages token counting endpoint with model claude-sonnet-5 and the same system prompt, messages, tools, images, or documents you plan to send. The response returns an input_tokens value before message generation.
Anthropic describes the result as an estimate because the final message request can differ slightly and may include system-added optimization tokens. The endpoint is nevertheless the authoritative preflight method for Claude Sonnet 5 and is more reliable than an OpenAI or character-ratio proxy.
import anthropic
client = anthropic.Anthropic()
result = client.messages.count_tokens(
model="claude-sonnet-5",
system="You are a concise assistant.",
messages=[{"role": "user", "content": "Count this prompt."}],
)
print(result.input_tokens)Claude Sonnet 5 Uses a Newer Tokenizer
Anthropic states that Claude Sonnet 5 uses the newer tokenizer also used by Claude Opus 4.7 and later Opus models, Claude Fable 5, and related current models. The same text produces approximately 30 percent more tokens than on earlier Claude models, although the exact change depends on the content.
That does not mean Sonnet 5 is always more expensive or less efficient; it means token-based limits and prices must be evaluated with Sonnet 5 counts. Recount real prompts rather than copying a Sonnet 4.6 token total into a new budget.
What count_tokens Includes
| Request component | Supported by preflight count | Caveat |
|---|---|---|
| System and messages | Yes | Use the exact conversation structure |
| Tool definitions | Yes | Server tool counts can depend on the sampling stage |
| Images and PDFs | Yes | Use the same media payload structure |
| Generated answer | No | Read output usage after generation |
| System-added optimization tokens | May appear | Anthropic says these are not billed to you |
Claude Sonnet 5 Context Planning
Claude Sonnet 5 supports a one-million-token context window and up to 128,000 output tokens. Because the newer tokenizer represents the same source text with more tokens on average than Sonnet 4.6, the effective amount of text that fits in a fixed token budget can be lower.
Leave headroom for the answer and for request components that are easy to overlook, especially tool schemas, documents, and long multi-turn histories. Run count_tokens again whenever the model ID, system prompt, tool set, or document set changes.
Tiktokenizer Reference Sources
Tokenizer behavior and model limits change. Verify production decisions with current provider documentation: