# Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-ranking

**Authors:** Qian Dong, Yiding Liu, Suqi Cheng, Shuaiqiang Wang, Zhicong Cheng, Shuzi Niu, Dawei Yin  
**arXiv:** [2204.11673](https://arxiv.org/abs/2204.11673)  
**Date:** 2022-04-25  
**Summary page:** https://www.summarizepaper.com/en/paper/2204.11673v1/

## Key points

- Passage re-ranking is a crucial task in information retrieval
- Pre-trained Language Models (PLMs) have shown potential in improving re-rankers
- Existing PLM-based re-rankers face challenges such as vocabulary mismatch and lack of domain-specific knowledge
- The proposed approach incorporates explicit knowledge from a knowledge graph into the re-ranking process
- A knowledge meta graph is introduced to distill reliable knowledge from the original graph
- PLM is used as the text encoder and a graph neural network is used as the knowledge encoder
- A novel knowledge injector facilitates dynamic interaction between the text encoder and knowledge encoder
- The method includes a GMN module within the knowledge injector to refine explicit knowledge based on learned text context features
- Experimental results demonstrate the effectiveness of the approach, especially for queries requiring domain-specific knowledge
- Our approach outperforms baselines that leverage external resources or incorporate domain-specific information

## Summary

Passage re-ranking is a crucial task in information retrieval and Pre-trained Language Models (PLMs) have shown great potential in improving the performance of re-rankers. However, existing PLM-based re-rankers often face challenges such as vocabulary mismatch and lack of domain-specific knowledge. To address these issues, we propose a novel approach that incorporates explicit knowledge from a knowledge graph into the re-ranking process. In our method, we leverage an existing but incomplete and noisy knowledge graph to enhance passage re-ranking. We introduce a knowledge meta graph that serves as a bridge between the query and passage by distilling reliable knowledge from the original graph. To align the embeddings of both text and knowledge in the latent space, we use PLM as the text encoder and employ a graph neural network over the knowledge meta graph as the knowledge encoder. To facilitate dynamic interaction between the text encoder and knowledge encoder, we design a novel knowledge injector. This injector allows for seamless integration of explicit knowledge into the re-ranking process. Inspired by CokeBERT, our method includes a GMN module within the knowledge injector to refine the context of explicit knowledge based on the learned text context features. Experimental results demonstrate that our approach is effective, particularly for queries that require in-depth domain knowledge. By incorporating explicit knowledge from a carefully curated knowledge graph, our method overcomes vocabulary mismatch and enhances domain-specific understanding in passage re-ranking tasks. In addition to our proposed approach, we also compare it with several baselines to evaluate its performance. These baselines include methods that focus on leveraging external resources or incorporating domain-specific information. Our approach outperforms these baselines, further highlighting its effectiveness in capturing relevant information for passage re-ranking tasks. Overall, our work demonstrates how incorporating explicit domain specific knowledge from a curated graph can significantly improve PLM based passage re ranking systems by addressing vocabulary mismatch and enhancing domain specific understanding; thus providing more accurate and relevant results for queries that require in depth understanding of their subject matter.