Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks

Authors: Nils Reimers, Iryna Gurevych

Published at EMNLP 2019
License: CC BY-SA 4.0

Results of the summarizing process for the arXiv paper: 1908.10084v1

  • Comprehensive Summary
  • Key points
  • Layman's Summary
  • Blog article
The paper introduces Sentence-BERT (SBERT), a modification of the BERT network that utilizes siamese and triplet network structures to derive semantically meaningful sentence embeddings. SBERT allows for efficient semantic similarity search and unsupervised tasks like clustering without requiring both sentences to be fed into the network. The authors demonstrate that SBERT significantly reduces the computational overhead required for finding the most similar pair in a collection of 10,000 sentences from 65 hours with BERT to just 5 seconds while maintaining accuracy. They evaluate SBERT on various sentence-pair regression tasks and transfer learning tasks where it outperforms other state-of-the-art sentence embedding methods. Overall, SBERT enables BERT to be applied to new tasks that were previously not feasible and achieves improved performance in several NLP applications.
AI-generated keywords: SBERT BERT Semantic Similarity Sentence Embeddings NLP Applications

AI-generated Key Points

  • Sentence-BERT (SBERT) is a modification of the BERT network
  • SBERT utilizes siamese and triplet network structures to derive semantically meaningful sentence embeddings
  • SBERT allows for efficient semantic similarity search and unsupervised tasks like clustering without requiring both sentences to be fed into the network
  • SBERT significantly reduces computational overhead for finding similar pairs in a collection of 10,000 sentences from 65 hours with BERT to just 5 seconds while maintaining accuracy
  • SBERT outperforms other state-of-the-art sentence embedding methods in various sentence-pair regression tasks and transfer learning tasks
  • SBERT enables BERT to be applied to new tasks that were previously not feasible
  • SBERT achieves improved performance in several NLP applications.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Abstract: BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019) has set a new state-of-the-art performance on sentence-pair regression tasks like semantic textual similarity (STS). However, it requires that both sentences are fed into the network, which causes a massive computational overhead: Finding the most similar pair in a collection of 10,000 sentences requires about 50 million inference computations (~65 hours) with BERT. The construction of BERT makes it unsuitable for semantic similarity search as well as for unsupervised tasks like clustering. In this publication, we present Sentence-BERT (SBERT), a modification of the pretrained BERT network that use siamese and triplet network structures to derive semantically meaningful sentence embeddings that can be compared using cosine-similarity. This reduces the effort for finding the most similar pair from 65 hours with BERT / RoBERTa to about 5 seconds with SBERT, while maintaining the accuracy from BERT. We evaluate SBERT and SRoBERTa on common STS tasks and transfer learning tasks, where it outperforms other state-of-the-art sentence embeddings methods.

Submitted to arXiv on 27 Aug. 2019

Tools for this paper

Explain it differently 3 free rewrites left today
Export BibTeX · RIS (Zotero, Mendeley, EndNote) · Markdown (Notion, Obsidian) · Premium
Citation alerts Premium
Journal club Slides to present it Premium
For journalists Press kit and hype check Premium

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

Figures and plots

Ask about a figure by number or use its question button to request visual analysis. Up to two figures and four image panels per question. Values read from plots are approximate; check the original data.

AI assistant instructions?

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.