# Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them

**Authors:** Hila Gonen, Yoav Goldberg  
**arXiv:** [1903.03862](https://arxiv.org/abs/1903.03862)  
**Date:** 2019-03-09  
**Summary page:** https://www.summarizepaper.com/en/paper/1903.03862v1/

## Key points

- Word embeddings are widely used in natural language processing (NLP)
- Word embeddings derived from text corpora often reflect gender biases present in society
- Researchers have developed methods for reducing gender bias in word embeddings
- Some debiasing techniques claim to significantly reduce gender bias, but the authors argue that they only provide a superficial removal of bias
- The authors conducted experiments on two debiasing methods and found that the gender bias information is still reflected in the distances between "gender-neutralized" words
- Existing bias removal techniques are insufficient and should not be trusted for providing truly gender-neutral modeling
- The study presents word lists used in previous research and discusses the accuracy results of their experiments
- A systematic bias remains even after debiasing
- Further research is needed to develop more robust techniques that can truly remove gender biases from NLP models.

## Summary

In the field of natural language processing (NLP), word embeddings are widely used for various tasks. However, it has been discovered that these word embeddings derived from text corpora often reflect gender biases present in society. This pervasive phenomenon has raised serious concerns and prompted researchers to develop methods for reducing this gender bias in word embeddings. Several recent works have proposed debiasing techniques that claim to significantly reduce gender bias in word embeddings, demonstrating convincing results. However, the authors of this study argue that these methods only provide a superficial removal of bias. While the bias is indeed reduced according to the provided definition, it is mostly hidden rather than completely removed. The authors conducted a series of experiments on two debiasing methods and found that the gender bias information is still reflected in the distances between "gender-neutralized" words in the debiased embeddings. This means that the bias can still be recovered from these embeddings. Based on their findings, they conclude that existing bias removal techniques are insufficient and should not be trusted for providing truly gender-neutral modeling. The study also provides additional context by presenting word lists used in previous research and discussing the accuracy results of their experiments. They highlight a systematic bias found in the embeddings which remains even after debiasing. Overall, this study challenges the effectiveness of current debiasing methods for word embeddings and emphasizes the need for further research to develop more robust techniques that can truly remove gender biases from NLP models.