3 Concepts to Pin Down Before LLM Development
Understanding tokens, context, and pricing before using LLM APIs makes both code and cost estimation smooth.
Tokens
The smallest unit an LLM processes—not "words" but word fragments and symbols too. Pricing is billed by token count.
- English: 1 word ≈ 1.3 tokens (≈ 4 chars)
- Japanese: 1 char ≈ 1-2 tokens (varies by kana/kanji); ~1.5-2x English for the same meaning
- Code: symbols/whitespace are all tokens
- Images/audio/video are also counted as tokens on a separate budget (varies by API)
How to Check
- OpenAI: tiktoken library, web Tokenizer tool
- Anthropic: count_tokens API (roughly like English)
Sense Values
| Text | Approx tokens |
|---|---|
| "Hello" | 3-5 |
| 1 paragraph (500 chars) | 500-800 |
| 1 article (3000 chars) | 3,000-5,000 |
| 1 book | 80,000-150,000 |
Context Window
The max tokens one API call can handle (input + output combined). As of 2026 the frontier is roughly ~1 million tokens, but it differs by model/offering and changes often—always check official. Even with a large window, the middle of long text tends to be dropped, so narrowing to key points, splitting, or passing only what's needed via RAG is practical.
Behavior on Overflow
- Explicit error (OpenAI / Anthropic)
- Old conversation truncated (some chat UIs)
- Handling: summarize, split, separate sessions
Pricing Structure
Many APIs bill "input tokens × rate + output tokens × rate," and the output rate is typically several times the input. Concrete rates and model names change frequently, so this article omits a price table. Hold only the decision-relevant tendencies:
- Output costs more than input: longer answers grow cost
- Lightweight models are far cheaper: mini / Haiku / Flash class is often enough for easy tasks
- Prompt caching: a shared long system prompt is heavily discounted via cache (OpenAI / Anthropic / Google)
- Batch API: large non-urgent jobs are often discounted
- Check the latest rates on official sites or price-comparison sites (converging but volatile)
A Single Estimate (the Idea)
Example: 1 chat response (input 500 / output 300 tokens) cost ≈ 500 × input_rate/1M + 300 × output_rate/1M → plug in the official latest rates. Output dominates, so just "answer briefly" and "summarize history" already help a lot.
Ways to Run for Free (Tendencies)
- Google AI Studio (Gemini API): has a free tier (caps change—check official)
- Signup credits: providers sometimes grant initial credit
- Local LLMs (Ollama / vLLM): try with zero API cost (detailed in a later article)
Additional Concepts Needed in Production
- Rate Limit: call cap per unit time; retry with exponential backoff
- Latency: response time; varies by model/context length
- Streaming: progressive output improves perceived speed
- Tool Use / structured output: receive in a form easy to pass downstream
Next Step
Once you grasp the basics, get an actual key in the next article "API Account Creation and Pricing."
