BERT-DRE: BERT with Deep Recursive Encoder for Natural Language Sentence Matching
AI-generated Key Points
- The paper presents a deep neural architecture for Natural Language Sentence Matching (NLSM) to identify semantic similarity in input text pairs.
- NLSM models predict a category or scale value for a pair of input texts to indicate their similarity or relationship.
- Two main steps are used: designing a model to obtain proper representation of the text and developing a matching mechanism to extract complex interactions.
- The authors collected a Persian Religious question matching dataset containing 18,000 samples with two questions, appropriate answers, and match or not-match labels for each question pair.
- BERT-DRE model was implemented by adding recursive encoder module to BERT and achieved an F1-score of 90.27% on the test data when trained and evaluated using annotated religious dataset making it the strongest model among those studied.
- Deep neural networks are preferred approach in NLSM field and researchers use convolutional and recurrent neural networks to determine semantic features and relationships among sentences.
- BERT-DRE outperforms its predecessor achieving an accuracy score of 90.29% when employing proposed architecture on same dataset.
Authors: Ehsan Tavan, Ali Rahmati, Maryam Najafi, Saeed Bibak, Zahed Rahmati
Abstract: This paper presents a deep neural architecture, for Natural Language Sentence Matching (NLSM) by adding a deep recursive encoder to BERT so called BERT with Deep Recursive Encoder (BERT-DRE). Our analysis of model behavior shows that BERT still does not capture the full complexity of text, so a deep recursive encoder is applied on top of BERT. Three Bi-LSTM layers with residual connection are used to design a recursive encoder and an attention module is used on top of this encoder. To obtain the final vector, a pooling layer consisting of average and maximum pooling is used. We experiment our model on four benchmarks, SNLI, FarsTail, MultiNLI, SciTail, and a novel Persian religious questions dataset. This paper focuses on improving the BERT results in the NLSM task. In this regard, comparisons between BERT-DRE and BERT are conducted, and it is shown that in all cases, BERT-DRE outperforms BERT. The BERT algorithm on the religious dataset achieved an accuracy of 89.70%, and BERT-DRE architectures improved to 90.29% using the same dataset.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
Welcome to our AI assistant! Here are some important things to keep in mind:
- The assistant will only answer questions related to this specific paper.
- Please note that this is not a bot for casual chatting.
- If you want the answer in a language other than the language you chose for navigating the website, simply add "TRANSLATE IN LANGUAGE L" at the end of your query (replace "LANGUAGE L" with the language of your choice).
- For example, you could ask "Can you extract the most important aspect of the paper? TRANSLATE IN SPANISH".
- If you want to keep the history of your questions/answers you should create an account.
Assess the quality of the AI-generated content by voting
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
Similar papers summarized with our AI tools
Navigate through even more similar papers through atree representation
Look for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.