Hybrid Search
Authoritative Definition
A unified retrieval approach that combines sparse lexical matching and dense vector search, typically merged via scoring algorithms like Reciprocal Rank Fusion (RRF).
Overview & Technical Description
Hybrid search mitigates the inherent weaknesses of purely vector-based or purely lexical retrieval systems by combining their strengths. It captures exact keyword matches—such as part numbers, proper names, and code IDs—through sparse indexing, while simultaneously understanding conceptual intent and semantic similarity through dense embeddings. Modern vector databases and search engines implement hybrid search by running concurrent sparse and dense queries. The results are then normalized and combined using algorithms like Reciprocal Rank Fusion (RRF) or convex combination scoring (alpha weighting). This ensures that documents containing exact critical terms are not missed, even if they fall outside the immediate vector neighborhood of the query. This approach is widely considered the industry standard for production-grade Retrieval-Augmented Generation (RAG) pipelines, offering significantly higher recall and precision across a diverse range of user queries and domain-specific vocabularies.
Editorial Notes
Industry best practice for production search and RAG. Tuning the weighting between sparse and dense scores (e.g., alpha parameter) is heavily domain-dependent and requires rigorous evaluation against a golden dataset.