Tokenizer Tax
LLMs charge per token, and the same sentence takes more tokens in some languages than others. Paste your own text to see what it costs on each model, or look up how much more your language pays than English.
Paste your text
Tokenized in your browser; nothing is sent anywhere.
| Tokenizer | Tokens | Chars / token | vs fewest | $ / 1M tokens | Per month |
|---|
Enter each model's input price (it changes too often to hard-code) to get a monthly cost. Large tokenizers load on demand; their files come straight from Hugging Face. Claude's tokenizer only runs on Anthropic's servers, so it isn't here; its per-language numbers are below.
Each shaded block is one token. A small number after a block means that text took several tokens: the tokenizer never learned those characters, so it spells them out byte by byte. That is where the biggest taxes come from.
Every language, ranked
Tokens needed relative to English for the same 1,012 sentences. Log scale; English is 1×.