Hybrid search

Embeddings have a blind spot, and every production RAG team finds it the same way: a user searches for HL-1042 or 'Juniper Throw' and semantic search returns… vaguely related fluff. Exact tokens — SKUs, order numbers, product names, error codes, legal section numbers — are where keyword search still beats meaning search. An embedding compresses a chunk into gist; 'HL-1042' isn't gist, it's an identifier, and identifiers survive compression badly.

The keyword side: BM25 in one paragraph

The classic engine is BM25: score a chunk by how often the query's words appear in it, weighted so rare words count most (matching 'trivet' means far more than matching 'the'), with a correction so long chunks don't win by volume. Decades old, unreasonably effective, and it costs almost nothing to run beside your vector index.

Fusing the two: RRF

You now have two rankings per query — semantic and keyword — with incomparable scores (cosine ∈ [-1,1], BM25 ∈ [0,∞)). Don't mix scores; mix ranks. Reciprocal Rank Fusion (RRF) gives each chunk Σ 1/(60 + rank) across the lists and re-sorts:

def rrf(rankings: list[list[str]], c: int = 60) -> list[str]:
    scores = {}
    for ranking in rankings:
        for rank, chunk_id in enumerate(ranking, 1):
            scores[chunk_id] = scores.get(chunk_id, 0) + 1 / (c + rank)
    return sorted(scores, key=scores.get, reverse=True)
# Rank #1 anywhere is strong evidence; agreement across lists compounds.
# c=60 is the standard damping constant — resist tuning it first.

In the starter kit (course-assets/harbordocs/) rrf and bm25 live alongside top_k in `search.py` — one retrieval module, all three offline-tested (the fusion math above is one of the tested assertions). Split them into your own fusion.py if you prefer; the code is identical.

• Why RRF wins in practice: no score normalization, no weights to tune, robust to one list being garbage for a given query — the good list dominates naturally.
• When hybrid pays: corpora full of identifiers and named entities (support, e-commerce, legal, logs). Run your labeled set: if the losses cluster on exact-term queries, hybrid is your fix. If your corpus is uniform prose, the gain may be a point or two — measure before adding the moving part.
• Query routing, the cheap version: a regex for obvious identifiers (HL-\d+) can skip fusion entirely and go keyword-first. Boring, effective, explainable.