What Is a Token
An LLM processes text not in "words" but in units called "tokens." Tokens are often finer than words; frequent words are 1 token, rare words/symbols split into multiple tokens.
Example (GPT-4-line tokenizer)
- "hello" → 1 token
- "electricity" → 1 token
- "prestidigitation" → 4 tokens ("pre", "stid", "ig", "itation")
- "こんにちは" → 3-4 tokens (Japanese is close to character-unit)
- "AI" → 1 token
Token-Count Guide
| Language | Per token |
|---|---|
| English | ~0.75 words, 4 chars |
| Japanese | ~0.5-1 char |
| Chinese | ~0.5-1 char |
| Code | ~3-5 chars |
For text of the same meaning, Japanese consumes 1.5-2x the tokens of English. Mind this in cost estimation.
Context Window
The max tokens the LLM can "see at once" in one request. Beyond it, older content is forgotten.
| Model | Context window |
|---|---|
| GPT-3.5 | 16K |
| GPT-4 / 4o | 128K |
| GPT-5.4 | 400K |
| Claude Opus 4.7 | 1M |
| Gemini 3.1 Pro | 2M |
| Llama 4 Behemoth | 10M |
The Limit of Long Context
Even with a large context window, the "the middle gets forgotten" (Lost in the Middle) phenomenon is known. Head and tail info are strongly reflected, but the middle is easily overlooked. In practice:
- Put important info at the head or tail
- Pass long text in a summary + detail hierarchy
- RAG-searching and passing only needed parts is more efficient
The Billing Mechanism
Many APIs bill by "input tokens × rate + output tokens × rate." Output rate is generally 4-5x the input.
Example: 1 Request with GPT-5.4
- Input 5,000 tokens × $2.50 / 1M = $0.0125
- Output 500 tokens × $10 / 1M = $0.005
- Total: ~$0.018 (~2.7 yen)
Prompt Caching
A cache feature that discounts input tokens up to 90% when reusing the same system prompt. Provided by Anthropic, OpenAI, Google. Essential for agent operation.
Token-Reduction Tips
- Keep system prompts short
- Pass only relevant parts via RAG
- YAML/Markdown can be lighter than JSON
- Often few examples + a CoT instruction suffices
- For translation tasks, send in English and localize the result (halves cost)
Cumulative-Cost Estimate Example
1 user × 50 requests/day × avg $0.018 × 10,000 users × 365 days = ~$3.3M/year. Cumulatively it balloons to frontier-model training-cost levels.
Summary
Tokens are the LLM's computation unit and billing unit. Mind that Japanese consumes 1.5-2x English; control cost with 3 points: (1) short prompts, (2) caching, (3) RAG for only what's needed.
