What Is an LLM Token?
Large language models don't read characters or words — they read tokens: chunks of text produced by a byte-pair-encoding (BPE) tokenizer. A token is usually a word fragment. In typical English prose one token averages about four characters, so 1,000 tokens is roughly 750 words. Code, JSON, and non-Latin scripts (Chinese, Japanese, emoji) tokenize less efficiently — a single CJK character often costs one or more tokens by itself.
Every API bill, context-window limit, and rate limit is denominated in tokens, which is why counting them matters. A model with a 128K-token context does not fit 128K words, and a prompt that looks short can be expensive if it contains dense code or Unicode. This token counter runs the real OpenAI BPE (o200k_base for GPT-5/GPT-4o, cl100k_base for older GPT-4 models) directly in your browser via WebAssembly-free JavaScript — your text never leaves the page.
Token counts also explain surprising model behavior: words split at unusual boundaries affect how models see rhymes, character counts, and string manipulation. If you have ever wondered why a model struggles to count the letters in a word, the answer is that it never saw letters — only tokens.
▶How to Count Tokens and Estimate LLM API Cost
- 1Paste your text. Drop a prompt, document, code snippet, or full chat transcript into the input box. Counting happens locally — nothing is uploaded.
- 2Read the token counts. The table shows the token count per model: exact BPE counts for OpenAI models, clearly-marked estimates for Claude and Gemini.
- 3Compare API costs. Input cost is what sending this text as a prompt costs; output cost is what it would cost if the model generated it. Prices are per million tokens from the official pricing pages.
▶How Accurate Are These Counts?
For OpenAI models the counts here are exact: GPT-5, GPT-4o and their minis use the o200k_base encoding, and this page runs the same BPE the API uses (the JavaScript equivalent of Python's tiktoken). What you see is what the API bills.
Anthropic (Claude) and Google (Gemini) do not publish their tokenizers, so no third-party tool can count their tokens exactly. Rows marked ≈ are estimates: Claude's tokenizer typically produces around 15% more tokens than OpenAI BPE on the same text, so we scale the o200k count accordingly. For billing-critical Claude numbers, use Anthropic's official count_tokens API endpoint — it is free to call and returns the exact number for a specific model.
Prices in the table are the official published rates per million tokens at the date shown. Providers change pricing and add cache or batch discounts frequently — treat the cost columns as planning estimates, and check the provider's pricing page before committing to a budget.
▶Input vs Output Tokens: How API Cost Is Calculated
LLM APIs price input (prompt) tokens and output (completion) tokens separately, and output is typically 4-5× more expensive. A request's cost is input_tokens × input_price + output_tokens × output_price, both per million tokens. That asymmetry drives most optimization advice: trimming a verbose system prompt saves money on every call, but capping response length usually saves more.
Reasoning models add a wrinkle: their internal thinking is billed as output tokens even when you never see it, so a one-line answer can cost thousands of output tokens. Prompt caching (supported by OpenAI, Anthropic, and Google) discounts repeated input prefixes by up to 90% — if you send the same long system prompt on every request, caching matters more than any manual trimming. To inspect what is actually inside a prompt before sending it, pair this counter with the JSON Formatter for payloads or the JWT Decoder to see how much of your token budget an auth token wastes.
▶Code Examples
▶JavaScript (gpt-tokenizer)
// npm install gpt-tokenizer
import { encode } from "gpt-tokenizer/encoding/o200k_base";
const text = "How many tokens is this sentence?";
const tokens = encode(text);
console.log(tokens.length); // exact count for GPT-5 / GPT-4o▶Python (tiktoken)
# pip install tiktoken
import tiktoken
enc = tiktoken.get_encoding("o200k_base") # GPT-5 / GPT-4o
tokens = enc.encode("How many tokens is this sentence?")
print(len(tokens))
# Older GPT-4 / GPT-3.5 models use cl100k_base
enc_legacy = tiktoken.get_encoding("cl100k_base")▶Python (Anthropic count_tokens — exact for Claude)
# pip install anthropic — the endpoint is free to call
from anthropic import Anthropic
client = Anthropic()
resp = client.messages.count_tokens(
model="claude-opus-4-8",
messages=[{"role": "user", "content": "How many tokens is this?"}],
)
print(resp.input_tokens) # exact, model-specific count▶Frequently Asked Questions
▶How many tokens is a word?
In typical English prose, one token averages about 0.75 words (roughly 4 characters), so 1,000 tokens is around 750 words. Code, JSON, and non-Latin text tokenize less efficiently — a single Chinese character or emoji often costs one or more tokens by itself.
▶Why do Claude and Gemini counts show as estimates?
Anthropic and Google do not publish their tokenizers, so exact counts are impossible outside their APIs. This tool shows OpenAI-exact counts and scales them (~+15% for Claude) as clearly-marked estimates. For exact Claude numbers, call Anthropic's free count_tokens API endpoint.
▶What is the difference between input and output tokens?
Input tokens are what you send (the prompt, system message, and conversation history); output tokens are what the model generates. Output is typically 4-5x more expensive per token, and reasoning models bill their hidden thinking as output too.
▶Is my text uploaded when I count tokens?
No. The BPE tokenizer runs entirely in your browser as JavaScript — nothing you paste is sent to any server, so it is safe to count confidential prompts and documents.
▶How can I reduce my LLM API costs?
The biggest levers are: enable prompt caching for repeated system prompts (up to 90% off cached input), cap response length, choose a smaller model tier for routine tasks, and batch non-urgent requests (typically 50% off). Trimming verbose prompts helps on every single call.