The 3-Stage Lifecycle
An LLM works in 3 stages: "pre-training → post-training → inference." Each differs greatly in cost structure and difficulty.
1. Pre-training
The stage of making the model learn "how to use language" and "knowledge of the world."
- Data: tens of trillions of tokens from web, books, papers, code, images
- Task: "predict the next word" (next-token prediction)
- Compute: GPT-4 class equals USD 10B-50B of electricity
- Period: weeks to months, thousands to tens of thousands of GPUs running continuously
- Who: limited players like OpenAI, Anthropic, Google, Meta, Mistral
In this phase, "world common sense," "grammar," "the seed of logical reasoning" form.
2. Post-training
Pre-training alone is just a "next-word predictor," so adjustment is needed to follow human instructions, not say harmful things, have natural dialogue.
SFT (Supervised Fine-Tuning)
Fine-tune with "question → ideal answer" pairs. Acquires initial instruction-following.
RLHF (Reinforcement Learning from Human Feedback)
Humans pick "answer A vs B" pairs, train a reward model, and use it to reinforcement-learn the model.
DPO (Direct Preference Optimization)
A simplified RLHF. Optimizes pair preferences directly without a reward model. Mainstream from 2024.
Constitutional AI
Developed by Anthropic. A loop where the AI itself self-critiques → improves based on a principles document (Constitution). Adopted in the Claude series.
3. Inference
The phase users actually use. The processing of inputting a prompt and getting output.
- Cost: a few to tens of yen per request (by model and token volume)
- Latency: response time 0.5 sec to tens of seconds
- Cumulative: huge with 1 user × 1 day × dozens of requests × millions of users
As of 2026, cumulative inference cost often greatly exceeds training cost.
User-Side Fine-Tuning
Full Fine-tuning
Re-train all model parameters. Needs hundreds of thousands to millions of GPU hours, not realistic for frontier models.
LoRA / QLoRA
Pseudo fine-tuning by adding low-rank matrices. Possible on little GPU; Llama 3 7B in a few hours on 1 GPU. The most practical.
Adapter / Prefix Tuning
Freeze the model body, train only added layers. A lightweight method close to LoRA.
Selective Use of RAG vs Fine-Tuning
| Use | Recommended |
|---|---|
| Want to add new knowledge | RAG (search latest docs) |
| Want to teach a specialized style/tone | Fine-tuning (LoRA) |
| Want to train on confidential data | Fine-tuning (on-prem) |
| Frequently updated info | RAG (no retraining) |
| Custom task (classification, extraction) | Fine-tuning |
2026 Trends
- Evolution RLHF → DPO → IPO (further simplification)
- Expanded use of synthetic data (AI generating training data among themselves)
- Distillation: transfer knowledge from a big model to a small one, cut inference cost
- Continual learning: models that keep learning while in production
Summary
An LLM is 3 stages: world knowledge by pre-training, human preference by post-training, practical use by inference. Cases needing fine-tuning in practice are limited; most can be handled by RAG and prompt engineering. Remember LoRA as the realistic answer for "lightweight FT."
