Skip to content

Fix swapped dense/sparse search types; add RRF option for hybrid search - #386

Merged
fabnemEPFL merged 2 commits into
EPFLiGHT:mainfrom
AlexPotocnik:fix/retriever-hybrid-ranker
Nov 19, 2026
Merged

fabnemEPFL merged 2 commits into
EPFLiGHT:mainfrom
AlexPotocnik:fix/retriever-hybrid-ranker

Conversation

@AlexPotocnik

Copy link
Copy Markdown
Collaborator

Summary

Two fixes to how Retriever.retrieve combines the dense and sparse searches.

  1. search_type="dense" and "sparse" were swapped. Single-mode searches ran a hybrid search with WeightedRanker(w, 1 - w), where w came from {"dense": 0, "sparse": 1}. But w is the weight of the dense request, so "dense" ranked by the sparse scores and "sparse" by the dense ones. Each single mode now runs one plain Milvus search on its own field.
  2. Hybrid search can now fuse by rank (RRF). New RetrieverConfig options hybrid_ranker ("weighted", the default and current behaviour, or "rrf") and rrf_k (default 60), using Milvus's RRFRanker. No re-indexing needed.

Why RRF

WeightedRanker normalizes each metric with a fixed function before weighting: cosine → (1 + s) / 2, inner product → 0.5 + arctan(s) / π. SPLADE inner products are typically well above 1 (often 10–20), so they all map to about 0.95–0.99, while dense cosines (~0.3–0.5) map to about 0.65–0.75. At the default 0.5/0.5, a chunk found only by SPLADE then scores at least ~0.48 and one found only by the dense model at most ~0.38: a dense-only hit can never enter the top k, and hybrid search returns SPLADE's ranking. RRFRanker fuses by rank, so both searches count regardless of their score scales.

Behaviour changes

  • Hybrid search: none by default. hybrid_ranker defaults to "weighted", so existing configs return the same results.
  • Single-mode searches now return the intended model's ranking. Their scores are now the raw Milvus scores (cosine in [-1, 1], SPLADE inner product unbounded) instead of the normalized ones, so a min_score tuned on the old single-mode scores filters differently. Happy to return normalized scores instead if you prefer.
  • With hybrid_ranker: rrf, scores (and so min_score) are at most 2 / (rrf_k + 1) ≈ 0.033. This is noted in the config comment.

Proposal: RRF as the default

I kept weighted as the default so this PR changes nobody's results. Given the above, I'd suggest making rrf the default, if you agree, in a follow-up PR that would:

  • switch the default to rrf;
  • pin hybrid_ranker: weighted in the existing example and evaluation configs (e.g. examples/who/) so their reported numbers still reproduce;
  • mention the change and the min_score scale in the release notes / version bump.

Tests

  • test_single_search_type_ranks_by_its_own_model: each single search type returns its own model's top result (fails on the previous code: "dense" returned the sparse top result and vice versa).
  • test_rrf_hybrid_scores_are_reciprocal_ranks: with hybrid_ranker: rrf, scores equal the reciprocal-rank sums of the dense and sparse rankings.
  • tests/test_retriever.py and tests/test_retriever_api.py pass (35/35); ruff clean; no new pyright errors.

search_type="dense" and "sparse" ran a hybrid search with
WeightedRanker(w, 1 - w) and w from {"dense": 0, "sparse": 1}, but w is
the weight of the dense request: "dense" ranked by the sparse scores and
"sparse" by the dense ones. Both now run a single search on their own
field. Hybrid search is unchanged.

Adds a test checking each single search type returns its own model's top
result (it fails on the previous code).
Hybrid search fuses dense and sparse results with Milvus's WeightedRanker,
which normalizes each metric with a fixed function: COSINE (1 + s) / 2,
IP 0.5 + arctan(s) / pi. SPLADE inner products (~10-20) all map to ~0.98
while dense cosines map to ~0.7, so at 0.5/0.5 a dense-only hit never
reaches the top k and hybrid search returns the sparse ranking.

New RetrieverConfig options hybrid_ranker ("weighted", the default and
current behaviour, or "rrf") and rrf_k (60), using Milvus's RRFRanker,
which fuses by rank. No re-indexing needed. Test checks RRF scores are
the reciprocal-rank sums of the dense and sparse rankings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the retrieval-dispatch fixes and configurable hybrid-fusion changes.

No open issues 🎉

What changed in this PR

Corrects dense-only and sparse-only retrieval to search the intended Milvus field, and adds optional reciprocal rank fusion (RRF) for hybrid retrieval while preserving weighted fusion as the default.

Changes:

  • Replaces single-mode hybrid searches with direct searches, returning raw model scores.
  • Adds hybrid_ranker and rrf_k configuration options, with documentation of RRF’s score scale.
  • Adds regression tests for single-mode ranking, RRF scores, and default configuration.
File Description
tests/​test_retriever.py Adds ranking, reciprocal-rank score, and configuration-default tests.
src/​mmore/​rag/​retriever.py Fixes single-mode search dispatch and adds configurable hybrid fusion.
examples/​rag/​config.yaml Shows the default weighted ranker and documents the RRF alternative.

Review effort: Balanced

@fabnemEPFL
fabnemEPFL merged commit 349ef06 into EPFLiGHT:main Nov 19, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants