Why Evaluation Is Needed
Conventional software was fine with "works or doesn't" tests, but LLMs are probabilistic and answer quality varies continuously. Running in production without an evaluation framework is like flying a plane without instruments.
3 Kinds of Evaluation
1. Golden Set (Manually Defined)
Create 50-500 pairs of "for this input this answer is ideal." Score with this every time a new version comes out.
- Created by task owners (don't leave to engineers)
- Include edge cases and common mistakes
- Update quarterly
2. LLM-as-a-Judge
Have another strong LLM (GPT-5, Claude Opus) score "this answer is good/bad." Enables mass evaluation.
- State the scoring rubric ("accuracy 0-5, clarity 0-5")
- Bias measures: swap positions in A/B comparison, vote across multiple models
- Periodically verify it matches humans
3. User Feedback
👍 / 👎, star ratings, detailed comments in production. Aggregate with LangSmith or Helicone.
Evaluation Metrics
| Use | Metrics |
|---|---|
| Classification | Accuracy, Precision, Recall, F1 |
| Extraction | Exact Match, F1 (ROUGE) |
| Summarization | BERTScore, LLM-as-a-Judge |
| RAG | Context Recall, Faithfulness, Answer Relevance (RAGAS) |
| Dialogue | Coherence, Helpfulness, Safety |
| Code | HumanEval, MBPP, your own unit tests |
Main Tools
LangSmith
LangChain's paid service. Logs, evaluation, dataset management integrated. Standard for mid-scale+.
Langfuse
OSS, self-hostable. Rising in popularity as a LangSmith alternative.
Promptfoo
OSS, CLI-based. Easy to embed in CI/CD. Strong at prompt A/B comparison.
OpenAI Evals
OpenAI official OSS. Standard for benchmark-like evaluation.
Ragas
A RAG-dedicated evaluation library. Easily measure Context Recall, Faithfulness.
Embedding Regression Tests
To run in production, auto-verify with the Golden Set in CI/CD on prompt change/model switch. Block merge if below threshold.
npm test → promptfoo eval ./golden-set.json
→ pass if threshold 90%+Production Monitoring
- Response time (P50, P95, P99)
- Cost (input/output tokens)
- Error rate (API failures, rate limits)
- Quality metrics (user 👍 rate, follow-up edit rate)
- Anomaly detection (sudden response-length change, frequent keywords)
Failure Patterns
- Golden Set stays old, latest use cases unreflected
- Blindly trusting LLM-as-a-Judge, no human verification
- Not cross-checking with production logs (eval and reality diverge)
- API billing explodes without measuring cost metrics
2026 Trends
- Improved accuracy of LLM auto-eval
- Multimodal evaluation (quality measurement of image/audio AI)
- Agent evaluation (accuracy of multi-stage tool calls)
- UI-test integration via browser-compute UI
Summary
LLM-app evaluation is built with the triangle of "Golden Set + LLM-as-a-Judge + user feedback." Embed Regression Tests in CI and monitor in production. Without this, model switching and prompt revision become "pray-and-run operation."
