What Is a Token
An LLM processes text not in "words" but in units called "tokens." Tokens are often finer than words; frequent words are 1 token, rare words/symbols split into multiple tokens.
Example (English tokenizer)
- "hello" → 1 token
- "electricity" → 1 token
- "prestidigitation" → 4 tokens ("pre", "stid", "ig", "itation")
- "こんにちは" → 3-4 tokens (Japanese is close to character-unit)
- "AI" → 1 token
Token-Count Guide
| Language | Per token |
|---|---|
| English | ~0.75 words, 4 chars |
| Japanese | ~0.5-1 char |
| Chinese | ~0.5-1 char |
| Code | ~3-5 chars |
For text of the same meaning, Japanese consumes 1.5-2x the tokens of English. Mind this in cost estimation. Token counts also shift between model generations: Claude switched tokenizers with the 4.7 generation, so the same text now produces roughly 30% more tokens. Re-measure whenever you switch models.
Context Window
The max tokens the LLM can "see at once" in one request. Beyond it, older content is forgotten.
| Model | Context window |
|---|---|
| GPT-5.6 (Sol / Terra / Luna) | 1.05M |
| Claude Fable 5 / Opus 5 / Sonnet 5 | 1M |
| Claude Haiku 4.5 | 200K |
| Gemini 3.1 Pro | 2M |
| Llama 4 Behemoth | 10M |
The Limit of Long Context
Even with a large context window, the "the middle gets forgotten" (Lost in the Middle) phenomenon is known. Head and tail info are strongly reflected, but the middle is easily overlooked. In practice:
- Put important info at the head or tail
- Pass long text in a summary + detail hierarchy
- RAG-searching and passing only needed parts is more efficient
The Billing Mechanism
Many APIs bill by "input tokens × rate + output tokens × rate." Output rate is generally 5-6x the input.
Example: 1 Request with GPT-5.6 Terra
- Input 5,000 tokens × $2 / 1M = $0.010
- Output 500 tokens × $12 / 1M = $0.006
- Total: ~$0.016
Prompt Caching
Reusing the same system prompt makes the repeated prefix far cheaper. On OpenAI and Anthropic, cache reads are billed at 1/10 the standard input rate (90% off); Anthropic charges a premium on cache writes (1.25x for a 5-minute cache, 2x for an hour), and OpenAI applies caching automatically to shared prefixes of 1,024 tokens or more. Essential for agent operation.
Token-Reduction Tips
- Keep system prompts short
- Pass only relevant parts via RAG
- YAML/Markdown can be lighter than JSON
- Often few examples + a CoT instruction suffices
- For translation tasks, send in English and localize the result (halves cost)
Cumulative-Cost Estimate Example
1 user × 50 requests/day × avg $0.016 × 10,000 users × 365 days = ~$2.9M/year. Cumulatively it balloons to frontier-model training-cost levels.
Summary
Tokens are the LLM's computation unit and billing unit. Mind that Japanese consumes 1.5-2x English; control cost with 3 points: (1) short prompts, (2) caching, (3) RAG for only what's needed.
