# Unsupervised Topic Segmentation of Meetings with BERT Embeddings

**Authors:** Alessandro Solbiati, Kevin Heffernan, Georgios Damaskinos, Shivani Poddar, Shubham Modi, Jacques Cali  
**arXiv:** [2106.12978](https://arxiv.org/abs/2106.12978)  
**Date:** 2021-06-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2106.12978v1/

## Key points

- Topic segmentation of meetings is challenging due to noisy meeting transcripts and lack of ground truth data.
- Meetings involve multiple participants with personalized language use, leading to transcript errors that make accurate topic segmentation difficult.
- Collecting labeled data for segmented meetings is complex and expensive as organizations are sensitive about their private meeting data.
- The proposed unsupervised approach uses pre-trained transformer models for topic segmentation, addressing the lack of ground truth data issue.
- A mechanism based on BERT embeddings and a new similarity score results in a 15.5% reduction in error rate compared to existing unsupervised methods.
- The study demonstrates a 26.6% reduction in error rate compared to current state-of-the-art supervised topic segmentation models trained on text datasets like Wikipedia.
- Utilizing pre-trained models like BERT and Sentence-BERT for sentence embeddings extraction helps filter out noisy speech data such as ASR miss-transcriptions and disfluencies from speakers.
- Employing a modified TextTiling method for topic segmentation without requiring labeled training data is part of the proposed approach.
- The unsupervised approach using pre-trained neural architectures shows significant improvements in topic segmentation accuracy for meeting transcripts compared to existing methods, effectively addressing challenges posed by noisy meeting data and lack of ground truth annotations.

## Summary

Topic segmentation of meetings is a challenging task due to the noisy nature of meeting transcripts and the lack of ground truth data. Meetings involve multiple participants with personalized language use, leading to transcript errors that make it difficult for even human annotators to accurately segment topics. Collecting labeled data for segmented meetings is complex and expensive as organizations are sensitive about their private meeting data. In this paper, we propose an unsupervised approach using pre-trained transformer models for topic segmentation of meetings. The lack of ground truth data hinders the benefits of advanced neural networks in comparison to other domains like written text. To address this issue, we introduce a mechanism based on BERT embeddings and a new similarity score that results in a 15.5% reduction in error rate compared to existing unsupervised methods. Our study also demonstrates a 26.6% reduction in error rate compared to current state-of-the-art supervised topic segmentation models trained on text datasets like Wikipedia. These models perform poorly due to differences between written text datasets and standard meeting transcripts datasets such as ICSI Meeting Corpus and AMI Meeting Corpus. The proposed approach involves utilizing pre-trained models like BERT and Sentence-BERT for sentence embeddings extraction, which helps filter out noisy speech data such as ASR miss-transcriptions and disfluencies from speakers. We also employ a modified TextTiling method for topic segmentation without requiring labeled training data. Overall, our unsupervised approach using pre-trained neural architectures shows significant improvements in topic segmentation accuracy for meeting transcripts compared to existing methods, effectively addressing the challenges posed by noisy meeting data and lack of ground truth annotations.