# Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems

**Authors:** Svetlana Kiritchenko, Saif M. Mohammad  
**arXiv:** [1805.04508](https://arxiv.org/abs/1805.04508)  
**Date:** 2018-05-11  
**Summary page:** https://www.summarizepaper.com/en/paper/1805.04508v1/

## Key points

- Authors discuss the amplification and perpetuation of biases by machine learning systems
- Previous research has focused on individual systems, lacking benchmark datasets
- Introduce the Equity Evaluation Corpus (EEC) to uncover biases towards races and genders
- Analyze 219 sentiment analysis systems using EEC
- Findings reveal statistically significant bias in some systems towards certain races or genders
- Emphasize the importance of addressing biases for fair outcomes
- Make EEC freely available for further research and mitigation of biased behavior
- Study highlights the need for comprehensive evaluation of machine learning systems regarding biases.

## Summary

In their paper titled "Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems," authors Svetlana Kiritchenko and Saif M. Mohammad discuss the inadvertent amplification and perpetuation of inappropriate human biases by automatic machine learning systems. Previous research on examining biases has primarily focused on individual systems, and there is a lack of benchmark datasets for evaluating such biases. To address this gap, the authors introduce the Equity Evaluation Corpus (EEC), which comprises 8,640 carefully selected English sentences designed to uncover biases towards specific races and genders. The authors utilize the EEC to analyze 219 automatic sentiment analysis systems that participated in the SemEval-2018 Task 1 'Affect in Tweets' shared task. Their findings reveal that several of these systems exhibit statistically significant bias, consistently generating slightly higher sentiment intensity predictions for certain races or genders. The authors emphasize the importance of addressing these biases to ensure fair and unbiased outcomes. They make the EEC freely available to facilitate further research in this area, thus providing researchers with a benchmark dataset to assess and mitigate biased behavior in sentiment analysis systems. Overall, this study sheds light on the need for comprehensive evaluation of automatic machine learning systems regarding inappropriate biases.