# Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches

**Authors:** Tim Schopf, Daniel Braun, Florian Matthes  
**arXiv:** [2211.16285](https://arxiv.org/abs/2211.16285)  
**Date:** 2022-11-29  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.16285v2/

## Key points

- Text classification of unseen classes is a significant challenge in Natural Language Processing.
- Two main types of methods used for this task are similarity-based approaches and zero-shot text classification approaches.
- Existing literature lacks consistent comparisons between these two types of methods.
- A systematic evaluation was conducted in the study on different similarity-based and zero-shot text classification methods for unseen classes.
- Benchmarking various state-of-the-art approaches was done on four text classification datasets, including a new dataset from the medical domain.
- Novel baselines using SimCSE and SBERT embeddings were proposed to improve results.
- A new similarity-based approach called Lbl2TransformerVec was introduced and outperformed existing state-of-the-art methods in unsupervised text classification tasks.
- Similarity-based approaches generally outperformed zero-shot methods across most cases in the experiments.
- Advanced embeddings like SimCSE or SBERT led to further improvements in similarity-based classification results compared to simpler text representations.

## Summary

In the field of Natural Language Processing, text classification of unseen classes poses a significant challenge. Researchers typically approach this task using two main types of methods: similarity-based approaches and zero-shot text classification approaches. However, existing literature lacks consistent comparisons between these approaches. To address this gap, a systematic evaluation was conducted in this study on different similarity-based and zero-shot text classification methods for unseen classes. The evaluation included benchmarking various state-of-the-art approaches on four text classification datasets, including a newly introduced dataset from the medical domain. The study also proposed novel baselines using SimCSE and SBERT embeddings to improve results. Additionally, a new similarity-based approach called Lbl2TransformerVec was introduced and outperformed existing state-of-the-art methods in unsupervised text classification tasks. The experiments revealed that similarity-based approaches generally outperformed zero-shot methods across most cases. Furthermore, utilizing advanced embeddings like SimCSE or SBERT instead of simpler text representations led to further improvements in similarity-based classification results. Overall, this comprehensive evaluation sheds light on the effectiveness of different text classification approaches for handling unseen classes and provides valuable insights for future research in the field of Natural Language Processing.