Hybrid Search: Fusing Dense Vectors and Sparse BM25 Correctly

Three weeks into production, a user searched "OpenAI API key" in your internal documentation system. The RAG pipeline returned nothing useful. The correct document existed, had been indexed, and was semantically about authentication and API credentials. But the exact string "OpenAI API key" appeared only once in a single heading, and the dense embedding model — trained on billions of tokens of general text — had smeared that specificity into a generic "authentication" neighborhood. Pure vector search returned documents about login flows and OAuth. The user filed a support ticket. Classic.

This is not an edge case. Proper nouns, product codes, version strings, technical identifiers, and any term rare enough that the embedding model hasn't seen it thousands of times all have this problem. Dense retrieval is strong on semantic variation — "car" and "automobile" are neighbors in embedding space — but it blurs exact-match signals precisely because it's been trained to generalize.

The fix isn't a better embedding model. It's hybrid search: run BM25 keyword scoring in parallel, fuse the two ranked lists, and let each system do what it's actually good at.

What dense search does well (and where it falls apart)

Dense retrieval, covered in depth in What Are Embeddings?, converts both documents and queries into fixed-dimensional vectors and retrieves by vector similarity. The geometry of the embedding space clusters semantically related content: a query about "ML model deployment" finds documents discussing "serving inference endpoints" even though none of those exact words appear in the query.

This is genuinely powerful. The failure modes are equally genuine.

Out-of-vocabulary terms. Your embedding model was trained on a fixed corpus. Rare proper nouns (product names, person names, ticker symbols), version strings ("v2.4.1"), and internal codes ("SKU-4829-B") often fall into a dense region where the model's nearest neighbors are contextually plausible but semantically wrong. The model sees "v2.4.1" and routes it toward software versioning documents — maybe the right category, but possibly not the one that contains that exact version.

High-specificity queries. When users know exactly what they're looking for, they tend to search with specific terms. "AWS S3 ListObjectsV2 pagination token" is a precise API query. The embedding-space neighborhood for that phrase may include related S3 content but also a lot of general AWS storage documentation that doesn't answer the narrow question.

Distributional shift. The embedding model was trained on data with a particular vocabulary. If your corpus uses domain jargon, internal terminology, or a language mix that's underrepresented in training data, the embedding space is sparse precisely where your documents live.

None of these fail loudly. Your ANN index returns exactly the top-K results you asked for, and they look reasonable. Precision drops silently.

BM25: why it's still relevant

BM25 (Best Match 25) is a probabilistic keyword scoring function from 1994 that remains the backbone of Elasticsearch, Lucene, Tantivy, and every serious full-text search system. The formula assigns each document a score based on the query terms it contains, weighted by:

• Term frequency saturation: more occurrences of the term increases score, but with diminishing returns (controlled by k1, typically 1.2–2.0).
• Inverse document frequency: terms that appear in few documents score higher than common terms.
• Document length normalization: shorter documents get a small boost so they're not penalized for having fewer opportunities to contain query terms (controlled by b, typically 0.75).

BM25(q, d) = Σ IDF(t) × [TF(t,d) × (k1+1)] / [TF(t,d) + k1×(1 - b + b×|d|/avgdl)]

This is exactly right for the failure cases above. "OpenAI API key" returns the document where that exact phrase appears, weighted by how rare each term is across the corpus. Product codes, version strings, and proper nouns all have high IDF values and score strongly.

BM25's failure modes are the mirror image. Synonyms, paraphrases, and semantic variation are invisible to it. A query for "how to reduce inference cost" gets nothing for documents discussing "GPU utilization optimization" that don't contain the word "cost". Conceptual queries — "what causes hallucination" — may return documents that use the word "hallucination" but don't actually explain the mechanism, while the most relevant mechanistic explanation uses entirely different vocabulary.

The two methods fail in complementary regions. This is why hybrid search isn't a marginal improvement — it's structural.

The fusion problem

The obvious fusion approach is: normalize both scores to [0, 1], multiply by weights, add. Something like:

final_score = alpha * cosine_score + (1 - alpha) * normalized_bm25_score

This is broken in a non-obvious way.

BM25 scores are corpus-dependent. The IDF component depends on log(N/df) where N is total document count and df is the number of documents containing the term. As you add documents to the index, IDF values shift. The raw BM25 score for the same document-query pair changes as the corpus grows. Min-max normalization to [0, 1] pins the scale to the current corpus's score range, which also shifts as documents are added.

Cosine similarity is bounded [-1, 1] by definition. Trained retrieval embeddings produce mostly positive similarities in practice — an artifact of how the embedding space is shaped during training — which is why [0, 1] is a common working assumption. Either way, the scale doesn't shift with corpus size.

When you add them, you're comparing two quantities that have fundamentally different statistical properties. The result is dominated by whichever has higher variance in your corpus at query time. Worse, this dominance relationship can flip as your corpus grows — a system that worked well at 100K documents may silently degrade at 10M.

This is the core reason the research literature converged on rank-based fusion rather than score-based fusion.

Reciprocal Rank Fusion

Reciprocal Rank Fusion (RRF), introduced by Cormack, Clarke and Buettcher (SIGIR 2009), solves the score incompatibility problem by ignoring scores entirely and fusing on ranks:

RRF_score(d) = Σᵢ 1 / (k + rankᵢ(d))

Where rankᵢ(d) is document d's position in result list i (1-indexed), and k is a smoothing constant (default 60). The final ranking sorts documents by their summed RRF scores, descending.

The intuition: a document at rank 1 in the dense list gets 1/61 ≈ 0.0164. A document at rank 10 gets 1/70 ≈ 0.0143. A document at rank 100 gets 1/160 ≈ 0.0063. Documents appearing near the top of both lists accumulate high scores from both contributions. Documents only in one list get only their contribution from that list. Documents absent from a list get zero contribution from it.

from collections import defaultdict

def reciprocal_rank_fusion(
    ranked_lists: list[list[str]],
    k: int = 60
) -> list[tuple[str, float]]:
    """
    Fuse multiple ranked document lists using RRF.
    
    Args:
        ranked_lists: Each inner list is doc IDs ordered best-to-worst.
        k: Smoothing constant. k=60 is the conventional default.
    
    Returns:
        List of (doc_id, rrf_score) sorted descending.
    """
    scores: dict[str, float] = defaultdict(float)
    
    for ranked_list in ranked_lists:
        for rank, doc_id in enumerate(ranked_list, start=1):
            scores[doc_id] += 1.0 / (k + rank)
    
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)


# Usage: fuse dense and sparse ranked results
dense_results = ["doc_A", "doc_C", "doc_B", "doc_E", "doc_D"]   # by cosine
sparse_results = ["doc_B", "doc_A", "doc_F", "doc_C", "doc_G"]  # by BM25

fused = reciprocal_rank_fusion([dense_results, sparse_results])
# doc_A: 1/61 + 1/62 ≈ 0.0325 (top of dense, second in sparse)
# doc_B: 1/63 + 1/61 ≈ 0.0322 (third in dense, first in sparse)
# doc_C: 1/62 + 1/64 ≈ 0.0317

The k=60 constant was derived empirically in the original paper. It prevents documents at rank 1 from dominating too heavily. In practice, k=60 is a solid default; the research shows relatively flat performance between k=40 and k=80.

{ "type": "hybrid-search", "alpha": 0.5, "title": "Dense + BM25 → RRF Fusion" }

Why RRF works in practice

The rank-based framing has a useful property: if one retrieval system returns a clearly irrelevant result at rank 1 (a false positive), it contributes 1/(k+1) to that document's score. But a document that's rank 3 in the dense list and rank 2 in the sparse list contributes 1/63 + 1/62 ≈ 0.0321, far outscoring a document that's rank 1 in only one list (1/61 ≈ 0.0164). Consensus across retrieval methods is rewarded.

The failure case for RRF: when only one retrieval method returns a relevant document and it does so at a very high rank (rank 1–3), RRF may bury it behind documents that appear in the middle of both lists. If your BM25 index is missing a document entirely (because it has no query terms), RRF can't compensate — the document simply contributes zero from that source. This is an argument for good sparse tokenization (stemming, stop-word handling, synonym expansion in BM25) rather than an argument against RRF.

Score normalization and alpha blending

When you have labeled query-document relevance data, you can tune a weighted fusion:

score(d) = α × norm_dense(d) + (1 - α) × norm_sparse(d)

where norm_* indicates min-max normalization within each result set. This is more interpretable than RRF and directly tuneable with offline evaluation. The tradeoff: you need representative labeled data, and the normalization parameters shift as your corpus grows.

The practical workflow:

1. Start with RRF (k=60) — no tuning needed, works well across query types.
2. If you have labeled relevance judgments (human ratings or click data), sweep α from 0.3 to 0.9 and measure NDCG@10 or MAP on your eval set.
3. Consider dynamic alpha: classify queries into "keyword" (high specificity, rare terms) vs "semantic" (abstract, conceptual) and apply different alphas per class.

Dynamic weighted fusion, a more recent variation, assigns per-query weights to the ranked lists based on query analysis before fusion. Simple heuristics — query length, presence of unusual tokens, IDF of query terms — can classify queries with reasonable accuracy. Reported experiments show low single-digit percentage-point precision gains over static alpha on mixed-query workloads — real but modest, and unlike RRF there's no canonical paper to cite, so measure on your own traffic before believing any vendor's number. Whether the gain is worth the added per-query classification step depends on your query mix.

flowchart TD
    Q[Incoming query] --> QA["Query analyzer\n(classify query type)"]
    QA -->|"High specificity\n(product codes, names)"| HKW["α = 0.3 dense / 0.7 sparse"]
    QA -->|"Conceptual / semantic\n(how, why, explain)"| SEM["α = 0.7 dense / 0.3 sparse"]
    QA -->|"Mixed / unclear"| MIX["α = 0.5 / 0.5\n(or static RRF)"]
    HKW --> FUSE["Weighted fusion"]
    SEM --> FUSE
    MIX --> FUSE
    FUSE --> RESULT["Ranked results"]
    style QA fill:#00e5ff,color:#0a0a0f
    style FUSE fill:#0e7490,color:#fff

SPLADE and learned sparse models

BM25 is a non-learned scoring function: it uses term frequencies and IDF from your corpus, with no optimization for retrieval quality. SPLADE (SParse Lexical AnD Expansion) replaces this with a learned sparse model that produces a sparse vector over the full vocabulary, trained end-to-end on retrieval tasks.

The key difference: SPLADE performs implicit query and document expansion. When you index a document about "automobiles", SPLADE may also assign weight to "car", "vehicle", and "sedan" in the sparse vector, even if they don't appear in the document. A query for "car" then matches — closing the vocabulary gap that BM25 can't bridge.

Benchmarks on BEIR (a heterogeneous retrieval benchmark) show SPLADE consistently outperforming BM25 by 5–15% on NDCG@10. The cost:

• SPLADE requires running inference at index time for every document (not just counting terms). On a 100K-document corpus, this adds hours of index build time versus minutes for BM25.
• Query-time inference adds ~5–20ms per query for SPLADE expansion vs milliseconds for BM25 tokenization.
• The infrastructure is more complex: you need a model serving endpoint for SPLADE, not just an inverted index.

For most production RAG systems, BM25 is the right default: it's free (Elasticsearch/Tantivy/Lucene), fast, and well-understood. Upgrade to SPLADE when you have a domain where vocabulary mismatch is the primary failure mode and the infrastructure cost is acceptable.

flowchart LR
    subgraph BM25["BM25 (non-learned)"]
        direction TB
        B1["Document text"] --> B2["Tokenize + stem"]
        B2 --> B3["TF-IDF weighted\ninverted index"]
        B3 --> B4["Score = Σ IDF × TF_sat"]
    end

    subgraph SPLADE["SPLADE (learned sparse)"]
        direction TB
        S1["Document text"] --> S2["BERT-like encoder"]
        S2 --> S3["Sparse vocab vector\n(RELU + log activation)"]
        S3 --> S4["Expanded inverted index\n(implicit synonyms)"]
        S4 --> S5["Score = sparse dot product"]
    end

    BM25 -->|"+5–15% NDCG@10\nbut: inference at index time"| SPLADE
    style BM25 fill:#0e7490,color:#fff
    style SPLADE fill:#00e5ff,color:#0a0a0f

Database implementation realities

How you implement hybrid search in practice depends heavily on which vector database you're using. This matters: the differences aren't cosmetic.

Database	BM25 / sparse support	Fusion algorithm	Notes
Weaviate	Native BM25 (BlockMax WAND)	RSF or RRF, configurable	Best-in-class native hybrid; WAND is faster than brute-force BM25 at scale
Milvus 2.5+	Native Sparse-BM25 (no separate index)	Weighted + RRF	Full pipeline in one query; strong at billion-scale
Qdrant v1.9+	Named-vector hybrid (sparse + dense)	RRF, configurable k	Learning-based filtered query planner; strong on filtered hybrid
Pinecone	Separate sparse and dense indexes	Client-side, manual	Must manage two indexes; fusion is your code
pgvector + pg_search	BM25 via pg_search (ParadeDB, Tantivy)	Client-side join	SQL join on rankings; functional but not optimized
Elasticsearch / OpenSearch	Native BM25, dense via kNN	Reciprocal rank window	Mature ops story; OpenSearch 3.0 adds GPU acceleration

See the Vector Database Landscape 2026 article for the full operational cost comparison; the hybrid search implementation quality is one reason Weaviate and Milvus come out ahead for RAG workloads.

A concrete Qdrant example. Since v1.10, the Query API does the RRF fusion server-side in one round trip — prefetch runs the dense and sparse searches, FusionQuery merges them. No client-side merge loop, matching the "RRF, configurable k" in the table above:

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://localhost:6333")

# Assume collection has both "dense" and "sparse" named vectors.
# Dense: 1536-dim embeddings. Sparse: BM25 token weights.

def hybrid_search(
    dense_embedding: list[float],
    sparse_indices: list[int],
    sparse_values: list[float],
    top_k: int = 10,
) -> list[models.ScoredPoint]:
    return client.query_points(
        collection_name="docs",
        prefetch=[
            models.Prefetch(
                query=dense_embedding,
                using="dense",
                limit=top_k * 2,  # over-fetch before fusion
            ),
            models.Prefetch(
                query=models.SparseVector(
                    indices=sparse_indices,
                    values=sparse_values,
                ),
                using="sparse",
                limit=top_k * 2,
            ),
        ],
        query=models.FusionQuery(fusion=models.Fusion.RRF),
        limit=top_k,
    ).points

(The reciprocal_rank_fusion function from earlier is what you'd run yourself on Pinecone or pgvector, where the table says "fusion is your code".)

One question the snippet glosses over: where do sparse_indices and sparse_values come from? The dense embedding is one API call, but BM25 weights need corpus statistics — that's the hard part of client-side hybrid. In practice you use a sparse encoder:

from fastembed import SparseTextEmbedding

bm25_encoder = SparseTextEmbedding(model_name="Qdrant/bm25")
sparse = next(bm25_encoder.query_embed("OpenAI API key rotation"))
# sparse.indices → hashed token ids, sparse.values → term weights

Pair this with modifier=idf on the collection's sparse vector config, so the encoder supplies term frequencies and Qdrant applies IDF from live collection statistics at query time. (Pinecone's equivalent is the pinecone-text BM25 encoder, which you must fit on a corpus sample yourself.) An encoder that bakes IDF in client-side is exactly where the stale-weights failure mode below lives.

Where reranking fits

Fusion — whether RRF or weighted blending — determines which documents make it into the top-K candidates. A cross-encoder reranker then re-scores those candidates by reading the query and document text together (not as separate embeddings), which is more accurate but too slow to run on the full index.

The standard pattern:

1. Retrieve top-50 candidates via hybrid search (dense + sparse, fused).
2. Run a cross-encoder reranker over those 50 candidates.
3. Return the top-10 by reranker score.

The reranker sees the actual query-document relevance signal, not a proxy. Models like Cohere Rerank, Jina Reranker v2, or a locally hosted cross-encoder/ms-marco-MiniLM-L-6-v2 typically add 5–15pp precision over fusion alone at a cost of 30–150ms GPU inference time (batched).

The mental model: hybrid search is cheap filtering; reranking is expensive scoring. Do the filtering broadly (top-50), then score narrowly.

import cohere

co = cohere.Client()

def rerank_candidates(
    query: str,
    candidates: list[dict],  # each has "id" and "text"
    top_n: int = 10,
) -> list[dict]:
    docs = [c["text"] for c in candidates]
    
    response = co.rerank(
        model="rerank-v3.5",
        query=query,
        documents=docs,
        top_n=top_n,
    )
    
    return [
        {**candidates[r.index], "rerank_score": r.relevance_score}
        for r in response.results
    ]

What breaks

Score leakage from one retrieval system

If you weight the fusion heavily toward one system (alpha=0.9 dense), you get near-identical results to pure dense retrieval. You've built hybrid infrastructure without getting hybrid benefits. The baseline evaluation: measure NDCG@10 for dense-only, sparse-only, and hybrid on a labeled eval set. If hybrid isn't beating both individually by at least 2–3pp, your fusion weights are wrong or your sparse index has a problem.

Empty sparse results

BM25 requires at least one query term to appear in the indexed documents. If your BM25 index uses aggressive stemming or stop-word removal that strips meaningful terms, you'll get empty sparse result sets. RRF degrades gracefully — it just uses the dense list — but you lose the hybrid benefit for those queries. Monitor sparse_recall_rate: the fraction of queries where sparse returns at least one result.

Vocabulary mismatch at indexing time

Dense embeddings are built from full text. BM25 is built from tokenized text. If your BM25 tokenizer strips meaningful tokens (dashes in "text-embedding-3-large", hyphens in "GPT-4o") that users search for, BM25 won't find the relevant documents even though they're indexed. Check your tokenizer settings against your actual query vocabulary.

Stale sparse weights

Inverted-index engines — Lucene, Tantivy, Elasticsearch — compute IDF at query time from live index statistics, so their BM25 scores stay fresh as documents arrive. (The one wrinkle: deleted documents keep inflating the statistics until segment merges clean them up.) The stale-IDF problem lives in the other architecture: precomputed sparse vectors. A client-side BM25 or SPLADE encoder bakes term weights in at encode time, calibrated to whatever corpus snapshot the encoder was fitted on. A sparse vector encoded when you had 10K documents carries IDF signals that are wrong at 1M — the rare-term boost that made hybrid worth building quietly decays. Either re-encode periodically as the corpus grows, or push IDF to the server (Qdrant's modifier=idf does exactly this) so it tracks live collection statistics.

Reranker as a bottleneck

Cross-encoder rerankers are serial: you can't start reranking until retrieval finishes, and reranking is GPU-bound. At high QPS, the reranker becomes the latency ceiling. Mitigations: reduce candidate count from 50 to 20 (accept slightly lower recall), use a smaller/faster cross-encoder, or parallelize requests across GPU workers. The question to answer before adding a reranker is whether your p99 latency budget absorbs an additional 30–150ms.

The decision in practice

The decision tree for a new RAG system:

Start with pure dense search if your query traffic is entirely conceptual (users ask open-ended questions, never search for specific identifiers). This is uncommon in practice.

Add hybrid search immediately if your corpus contains product codes, names, identifiers, version strings, or any exact-match terms users will search for. Use RRF with k=60. This is the correct default for most production RAG.

Tune alpha or use dynamic weighting if you have a labeled eval set and NDCG@10 is important enough to measure. This is worth doing for any system with >1K daily queries.

Add a reranker when you can absorb the latency and precision matters more than speed — customer-facing search, high-stakes document retrieval, compliance-critical Q&A. Skip it if you're already below 50ms total budget.

Upgrade to SPLADE only when BM25 vocabulary mismatch is measurably your top failure mode (run a failure analysis on low-score retrievals first) and you have the infrastructure headroom for model inference at index time.

The ANN Indexes Under the Hood article covers the dense retrieval mechanics; the RAG Evaluation with Ragas article covers how to measure whether your hybrid configuration is actually working. Retrieval quality is the part of RAG you can measure precisely — run the eval before committing to any fusion strategy.

Hybrid search is not the most glamorous part of building a RAG system. It's also the part that's most likely to be the difference between a system users trust and one they've learned to work around.