Raising Retrieval Quality: Rerankers and Hybrid Search
The main cause of RAG being "plausible but off-target" is retrieval accuracy. The standard improvements are rerankers and hybrid search.
Hybrid Search
- Use vector search (semantic nearness) and keyword search (word match) together
- Proper nouns, model numbers, abbreviations favor keywords; using both reduces misses
Reranker
- Retrieve a generous set of candidates in primary search
- Re-evaluate relevance to the question with a reranker model and reorder
- Pass only the top ones to the generation prompt (noise reduction)
Other Improvements
- Query rewriting (convert vague questions for search)
- Revisiting chunk design (many accuracy problems are here)
- Metadata filters (narrow by permission/period)
Caution
Adopt improvements only after measuring effect with evaluation data (same principle as the prompt-evaluation chapter). Adding blindly only makes it slower and costlier.
Chapter Summary
RAG is "90% retrieval." Raise retrieval quality with hybrid + rerank + good chunk design, and always measure the effect. This is the core of practical RAG.
