Hybrid Search and Reranking: Combining Vector + Keyword
# Vector search misses exact matches. Keyword search misses semantic matches. Hybrid search combines both, and reranking reorders results by relevance. Together they cut RAG retrieval errors by 30–50%.
Vector search is great at semantic similarity but terrible at exact matches — "API version 3.2.1" and "SDK release 3.2.1" might not match a query for "3.2.1 changelog". Keyword search handles exact matches but misses synonyms and paraphrases. Hybrid search gets both. Add reranking and you get the best retrieval stack available in 2026.
Why Hybrid > Pure Vector or Pure Keyword
| Query type | Vector | Keyword | Hybrid | |-----------|--------|---------|--------| | "How does authentication work?" | ✓ | ✗ | ✓ | | "Error: ECONNREFUSED" | ✗ | ✓ | ✓ | | "JWT token expiry" | ✓ | ✓ | ✓✓ | | "v3.2.1 breaking changes" | ✗ | ✓ | ✓ | | "similar to OAuth but simpler" | ✓ | ✗ | ✓ |
Hybrid wins across all query types. Pure vector misses ~15% of high-precision queries. Pure keyword misses ~30% of semantic queries.
BM25 Keyword Search Implementation
BM25 is the standard keyword ranking algorithm (used in Elasticsearch, Solr, and Qdrant's sparse vectors):
// Simple BM25 implementation for small corpora
class BM25Index {
private corpus: string[];
private idf: Map<string, number> = new Map();
private tf: Map<string, Map<string, number>> = new Map();
private avgDocLength: number = 0;
private readonly k1 = 1.5;
private readonly b = 0.75;
constructor(documents: string[]) {
this.corpus = documents;
this.buildIndex(documents);
}
private buildIndex(docs: string[]) {
const N = docs.length;
const docLengths: number[] = [];
const dfMap: Map<string, number> = new Map();
for (const doc of docs) {
const terms = this.tokenize(doc);
docLengths.push(terms.length);
const termFreq = new Map<string, number>();
const uniqueTerms = new Set(terms);
for (const term of terms) {
termFreq.set(term, (termFreq.get(term) ?? 0) + 1);
}
this.tf.set(doc, termFreq);
for (const term of uniqueTerms) {
dfMap.set(term, (dfMap.get(term) ?? 0) + 1);
}
}
this.avgDocLength = docLengths.reduce((a, b) => a + b, 0) / N;
for (const [term, df] of dfMap) {
this.idf.set(term, Math.log((N - df + 0.5) / (df + 0.5) + 1));
}
}
score(query: string, document: string): number {
const terms = this.tokenize(query);
const termFreq = this.tf.get(document) ?? new Map();
const docLength = [...termFreq.values()].reduce((a, b) => a + b, 0);
let score = 0;
for (const term of terms) {
const tf = termFreq.get(term) ?? 0;
const idf = this.idf.get(term) ?? 0;
const tfNorm = (tf * (this.k1 + 1)) /
(tf + this.k1 * (1 - this.b + this.b * docLength / this.avgDocLength));
score += idf * tfNorm;
}
return score;
}
search(query: string, topK = 10): Array<{ document: string; score: number }> {
return this.corpus
.map((doc) => ({ document: doc, score: this.score(query, doc) }))
.sort((a, b) => b.score - a.score)
.slice(0, topK);
}
private tokenize(text: string): string[] {
return text.toLowerCase().split(/\W+/).filter((t) => t.length > 2);
}
}
For production, use Qdrant's built-in sparse vectors (faster, more accurate) or Elasticsearch.
Hybrid Search: Reciprocal Rank Fusion
Combine vector and keyword results using Reciprocal Rank Fusion (RRF):
async function hybridSearch(
query: string,
topK = 10,
rrfK = 60 // RRF constant, 60 is the standard
): Promise<string[]> {
// Run both searches in parallel
const [vectorResults, keywordResults] = await Promise.all([
vectorSearch(query, topK * 2),
bm25Index.search(query, topK * 2),
]);
// RRF score: 1 / (k + rank)
const scores = new Map<string, number>();
function addRRFScore(results: string[]) {
results.forEach((doc, rank) => {
scores.set(doc, (scores.get(doc) ?? 0) + 1 / (rrfK + rank + 1));
});
}
addRRFScore(vectorResults);
addRRFScore(keywordResults.map((r) => r.document));
return [...scores.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, topK)
.map(([doc]) => doc);
}
RRF is robust to score scale differences between vector similarity (0–1) and BM25 scores (unbounded). It only uses rank positions, not scores.
Qdrant Native Hybrid Search
Qdrant 1.7+ supports hybrid search natively with sparse + dense vectors:
import { QdrantClient } from "@qdrant/js-client-rest";
const qdrant = new QdrantClient({ url: "http://localhost:6333" });
// Create collection with both dense and sparse vectors
await qdrant.createCollection("hybrid-docs", {
vectors: {
dense: { size: 1536, distance: "Cosine" },
},
sparse_vectors: {
sparse: { index: { on_disk: false } },
},
});
// Index document with both vector types
async function indexDocument(text: string, id: number) {
const [denseEmbedding] = await embedTexts([text]);
const sparseVector = computeSparseVector(text); // BM25 sparse representation
await qdrant.upsert("hybrid-docs", {
points: [{
id,
vector: {
dense: denseEmbedding,
sparse: sparseVector,
},
payload: { text },
}],
});
}
// Hybrid query with RRF fusion
async function hybridQueryQdrant(query: string, topK = 5): Promise<string[]> {
const [denseEmbedding] = await embedTexts([query]);
const sparseQuery = computeSparseVector(query);
const results = await qdrant.query("hybrid-docs", {
prefetch: [
{ query: denseEmbedding, using: "dense", limit: topK * 3 },
{ query: sparseQuery, using: "sparse", limit: topK * 3 },
],
query: { fusion: "rrf" },
limit: topK,
with_payload: true,
});
return results.map((r) => r.payload?.text as string);
}
Reranking: The Secret Weapon
Hybrid search returns top-K candidates. Reranking reorders them by actual relevance to the query using a more expensive cross-encoder model:
// Cross-encoder reranking using Cohere's reranker
async function rerankResults(
query: string,
documents: string[],
topK = 5
): Promise<string[]> {
const response = await fetch("https://api.cohere.com/v1/rerank", {
method: "POST",
headers: {
"Authorization": `Bearer ${process.env.COHERE_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: "rerank-english-v3.0",
query,
documents,
top_n: topK,
}),
});
const { results } = await response.json();
return results.map((r: { document: { text: string } }) => r.document.text);
}
// Or use LLM-based reranking (no external API needed)
async function llmRerank(query: string, documents: string[], topK = 5): Promise<string[]> {
const response = await anthropic.messages.create({
model: "claude-haiku-4-5-20251001", // fast + cheap for ranking
max_tokens: 200,
messages: [{
role: "user",
content: `Rank these ${documents.length} documents by relevance to: "${query}"
Return a JSON array of indices (0-based), most relevant first.
Documents:
${documents.map((d, i) => `[${i}] ${d.slice(0, 200)}`).join("\n")}`,
}],
});
const ranking: number[] = JSON.parse(response.content[0].text);
return ranking.slice(0, topK).map((i) => documents[i]);
}
Reranking typically adds 100–300ms but improves top-1 precision by 15–25%. Worth it for high-stakes retrieval.
Full Pipeline
async function bestRAGRetrieval(query: string, topK = 5): Promise<string[]> {
// 1. Hybrid search: get 20 candidates
const candidates = await hybridSearch(query, topK * 4);
// 2. Rerank: select best 5
const reranked = await rerankResults(query, candidates, topK);
return reranked;
}
FAQ
Does hybrid search require running two indexes? Yes — one for dense vectors (Qdrant/pgvector), one for keyword search (BM25 or Elasticsearch). Qdrant simplifies this by supporting both in one collection. Otherwise, run them separately and merge results with RRF.
What's the performance overhead of reranking? Cohere's reranker: ~150ms for 20 documents. LLM reranker: ~400–600ms with Haiku. Compared to the LLM generation step (1–5s), this overhead is modest for quality-critical applications.
Should I always use hybrid + reranking? Not for simple Q&A or prototypes. Add hybrid when pure vector misses important exact-match queries. Add reranking when the difference between rank-1 and rank-3 matters (high-stakes answers).
How do I evaluate whether hybrid improves over pure vector? Use your golden dataset (from Testing LLM Applications). Measure hit@1, hit@3, hit@5 for each retrieval strategy. If hybrid + reranking doesn't beat pure vector by >5% on your dataset, the added complexity isn't worth it.
What's the best sparse vector implementation? SPLADE is the state of the art — it learns sparse representations through training. For a simpler, non-ML approach, BM25 sparse vectors work well. Qdrant's built-in sparse support handles both.