Skip to content
Veritas AI

Retrieval

The default mode is hybrid: dense vector search and sparse BM25 run in parallel, and their rankings are fused with Reciprocal Rank Fusion.

Why hybrid

Dense and sparse retrieval fail differently:

  • Dense vectors capture paraphrase and semantics, but dilute exact identifiers — SKUs, error codes, legal section numbers
  • BM25 nails exact terms, but has no notion of meaning

On keyword-heavy workloads, hybrid lifts recall 15–30% over dense-only.

RRF, not score mixing

Cosine similarities and BM25 scores live on incomparable scales — averaging them is meaningless. RRF fuses by rank position instead:

python
def rrf_score(rank: int, k: int = 60) -> float:
    return 1.0 / (k + rank)

# A chunk's final score is the sum of its RRF scores
# across the dense and sparse rankings.

The constant k dampens the head of each ranking; 60 is a robust default.

Reranking

The fused candidates (typically 20) pass through a cross-encoder that reads the query and each chunk together, then keeps the top 5. This adds 30–80ms and is usually the single biggest precision win in the pipeline. Disable it per-request with "rerank": false when latency matters more.

Tuning

| Knob | Default | Effect | | --- | --- | --- | | retrieval_mode | hybrid | dense / sparse / hybrid | | top_k | 5 | Chunks handed to the LLM | | fetch_k | 20 | Candidates before reranking | | rrf_k | 60 | Rank-fusion damping constant |

PreviousChunkingNext Generation