Q: How do I combine BM25 and vector search in Elasticsearch?

Combine BM25 and kNN vector search using reciprocal rank fusion via the retriever API with an rrf block containing a standard retriever and a knn retriever. RRF avoids score normalization problems since BM25 and cosine similarity scores are not comparable.

Q: Should synonyms be applied at index time or query time?

Query-time synonyms using synonym_graph filters are generally preferred because they can be updated without reindexing. Index-time synonyms require a full reindex for every change, while query-time synonym sets marked updateable can be refreshed in place.

Q: How do I measure search relevance with nDCG?

Use the Elasticsearch _rank_eval API with a judgment set of graded relevance ratings per query. Run it in CI against every query-template change, and fail the build when nDCG drops beyond the noise threshold, with per-query diffs attached.

Q: Why do exact SKU or model number searches fail?

Exact identifier searches fail when stemming or tokenization mangles the identifiers. Map SKUs as keyword fields with a lowercase normalizer, and route exact-looking queries to the keyword subfield instead of the stemmed text field.

Q: When should I use vector search instead of BM25?

Vector search alone misses exact terms like SKUs and rare keywords that BM25 handles well, while BM25 misses paraphrase and intent. Default to hybrid retrieval with rank fusion, and only adopt a single-mode setup after proving it against the judgment set.

Question 1

How do I fix zero-results queries in Elasticsearch?

Accepted Answer

Zero-results queries usually come from analyzer mismatches, missing synonyms, or overly strict minimum_should_match. Use the _analyze API to compare indexed terms against query-time tokens, align index and search analyzers, add query-time synonyms, and relax matching with patterns like 2<75%.

Question 2

How do I combine BM25 and vector search in Elasticsearch?

Accepted Answer

Combine BM25 and kNN vector search using reciprocal rank fusion via the retriever API with an rrf block containing a standard retriever and a knn retriever. RRF avoids score normalization problems since BM25 and cosine similarity scores are not comparable.

Question 3

Should synonyms be applied at index time or query time?

Accepted Answer

Query-time synonyms using synonym_graph filters are generally preferred because they can be updated without reindexing. Index-time synonyms require a full reindex for every change, while query-time synonym sets marked updateable can be refreshed in place.

Question 4

How do I measure search relevance with nDCG?

Accepted Answer

Use the Elasticsearch _rank_eval API with a judgment set of graded relevance ratings per query. Run it in CI against every query-template change, and fail the build when nDCG drops beyond the noise threshold, with per-query diffs attached.

Question 5

Why do exact SKU or model number searches fail?

Accepted Answer

Exact identifier searches fail when stemming or tokenization mangles the identifiers. Map SKUs as keyword fields with a lowercase normalizer, and route exact-looking queries to the keyword subfield instead of the stemmed text field.

Question 6

When should I use vector search instead of BM25?

Accepted Answer

Vector search alone misses exact terms like SKUs and rare keywords that BM25 handles well, while BM25 misses paraphrase and intent. Default to hybrid retrieval with rank fusion, and only adopt a single-mode setup after proving it against the judgment set.

agency-search-relevance-engineer

SYSTEM DOCUMENTATION & REQUIREMENTS

Frequently Asked Questions about agency-search-relevance-engineer

Related Skills

agency-search-relevance-engineer

esql-guide

Embedding Model Training

pytorch-lightning