# A Deep Learning System for Sentiment Analysis of Service Calls

**Authors:** Yanan Jia, Sony SungChu  
**arXiv:** [2004.10320](https://arxiv.org/abs/2004.10320)  
**Date:** 2020-04-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2004.10320v1/

## Key points

- Sentiment analysis is crucial in advancing AI capabilities to understand human language and emotions
- Study focuses on sentiment states from well-trained Customer Service Representatives (CSRs) to make interactions between humans and AI more intelligent
- Sentiment analysis pipeline implemented for real-world multi-party conversations, specifically service calls
- Utilization of Voice Activity Detection (VAD) to filter out noise and long pauses, followed by Automatic Speech Recognition (ASR) and Automatic Speaker Diarization (ASD)
- Calls segmented into an average of 69 utterances with varying durations; speakers labeled as CSR or customer based on specific keywords within each utterance
- Guidelines established for annotating sentiments expressed during interactions, including negative tags for expressions of negative emotions or issues faced by CSRs
- Implicit sentiments conveyed through tone and context in both text and audio labels are considered in sentiment analysis
- Semi-supervised learning annotation pipeline implemented to enhance sentiment analysis process using deep learning systems
- Integration of voice recognition technologies with sentiment analysis techniques offers valuable insights into improving customer service interactions and enhancing AI capabilities

## Summary

In the realm of artificial intelligence (AI), sentiment analysis plays a crucial role in advancing the capabilities of machines to understand and replicate human language and emotions. By studying the formation and response of sentiment states from well-trained Customer Service Representatives (CSRs), interactions between humans and AI can become more intelligent. In this study, a sentiment analysis pipeline is implemented for real-world multi-party conversations, specifically service calls. To ensure data accuracy and relevance, Voice Activity Detection (VAD) is utilized to filter out noise and long pauses in calls, followed by Automatic Speech Recognition (ASR) and Automatic Speaker Diarization (ASD) to transcribe verbal statements, extract utterance start and end times, and identify speakers. Each call is segmented into an average of 69 utterances with varying durations. By labeling speakers as either CSR or customer based on specific keywords within each utterance, the data is further categorized for analysis. Guidelines are established for annotating sentiments expressed during interactions. Negative tags are assigned to expressions of negative emotions, attitudes, evaluations, or facts caused by other parties. For CSRs facing business process or system issues that prevent them from assisting customers effectively are also considered negative. Sentiment analysis also considers implicit sentiments conveyed through tone and context in both text and audio labels. Additionally, a semi-supervised learning annotation pipeline is implemented to enhance the sentiment analysis process further. This approach leverages deep learning systems to accurately analyze sentiments expressed during service calls. Overall,this detailed summary highlights the comprehensive approach taken in analyzing sentiments in service calls using a combination of technological tools and human interpretation. The integration of voice recognition technologies with sentiment analysis techniques offers valuable insights into improving customer service interactions and enhancing AI capabilities in understanding human emotions effectively.