# Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information

**Authors:** Kun Zhao, Bohao Yang, Chenghua Lin, Wenge Rong, Aline Villavicencio, Xiaohui Cui  
**arXiv:** [2305.16967](https://arxiv.org/abs/2305.16967)  
**Date:** 2023-05-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.16967v1/

## Key points

- Authors address the challenge of the one-to-many issue in open-domain dialogues
- Proposed a novel learning-based automatic evaluation metric called CMN
- CMN utilizes Conditional Variational Autoencoders (CVAEs) with Next Sentence Prediction (NSP) and Mutual Information (MI)
- Demonstrated CMN's effectiveness compared to baseline methods through experiments on two dialogue datasets
- CMN can handle responses that deviate significantly from golden reference responses in terms of semantics

## Summary

In their paper titled "Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information," authors Kun Zhao, Bohao Yang, Chenghua Lin, Wenge Rong, Aline Villavicencio, and Xiaohui Cui address the challenge of the one-to-many issue in open-domain dialogues. This issue presents significant hurdles for automatic evaluation methods as there can be multiple suitable responses within a given conversational context. To overcome this challenge, the authors propose a novel learning-based automatic evaluation metric called CMN. The CMN metric utilizes Conditional Variational Autoencoders (CVAEs) augmented with a Next Sentence Prediction (NSP) objective and Mutual Information (MI) to model semantic similarity in the latent space. Through experiments on two open-domain dialogue datasets, the authors demonstrate CMN's effectiveness compared to baseline methods. Notably, CMN can handle responses that deviate significantly from golden reference responses in terms of semantics. Overall, this research contributes valuable insights into improving automatic evaluation methods for open-domain dialogues by incorporating advanced techniques such as CVAEs, NSP objectives, and MI calculations to evaluate semantic similarity within conversational contexts.