# Enhancing Relevance of Embedding-based Retrieval at Walmart

**Authors:** Juexin Lin, Sachin Yadav, Feng Liu, Nicholas Rossi, Praveen Reddy Suram, Satya Chembolu, Prijith Chandran, Hrushikesh Mohapatra, Tony Lee, Alessandro Magnani, Ciya Liao  
**arXiv:** [2408.04884](https://arxiv.org/abs/2408.04884)  
**Date:** 2024-08-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2408.04884v1/

## Key points

- Research focuses on enhancing relevance of Embedding-based Neural Retrieval (EBR) at Walmart for product search
- Initial implementation of EBR system at Walmart showed promising results in improving relevance and add-to-cart rates
- Challenges included relevance degradation due to false positives/negatives in training data and difficulties in handling query misspellings
- Proposed approaches to strengthen EBR model capabilities, including:
- Introduction of Relevance Reward Model (RRM) based on human relevance feedback
- Techniques like typo-aware training and semi-positive generation employed to enhance performance
- Strategies aim to improve retrieval accuracy by addressing common issues encountered during product search queries
- Effectiveness of enhancements validated through offline relevance evaluation, online AB tests, and successful deployments in live production environments

## Summary

The research presented in this study focuses on enhancing the relevance of Embedding-based Neural Retrieval (EBR) at Walmart for product search. The initial implementation of the EBR system at Walmart showed promising results in improving relevance and add-to-cart rates. However, there were instances of relevance degradation due to factors such as false positives/negatives in training data and difficulties in handling query misspellings. To address these challenges, the researchers proposed several approaches to strengthen the capabilities of the EBR model in terms of retrieval relevance. One key contribution is the introduction of a Relevance Reward Model (RRM) based on human relevance feedback. This model helps filter out noise from training data and incorporates it into the EBR model through a multi-objective loss function. Additionally, techniques like typo-aware training and semi-positive generation were employed to enhance the performance of the EBR model further. These strategies aim to improve retrieval accuracy by addressing common issues encountered during product search queries. The effectiveness of these enhancements was validated through offline relevance evaluation, online AB tests, and successful deployments in live production environments. The study showcases how refining the EBR model can lead to significant improvements in retrieval relevance, ultimately enhancing the overall customer shopping experience at Walmart.