# Women also Snowboard: Overcoming Bias in Captioning Models

**Authors:** Kaylee Burns, Lisa Anne Hendricks, Trevor Darrell, Anna Rohrbach  
**arXiv:** [1803.09797](https://arxiv.org/abs/1803.09797)  
**Date:** 2018-03-26  
**Summary page:** https://www.summarizepaper.com/en/paper/1803.09797v1/

## Key points

- Authors address biases in machine learning methods, specifically in image captioning models
- Image captioning models tend to amplify biases present in training data
- Proposed model called the Equalizer model ensures equal gender probability when gender evidence is occluded and provides confident predictions when evidence is present
- Model focuses on looking at a person rather than relying solely on contextual cues for gender-specific predictions
- Incorporates two losses: Appearance Confusion Loss and Confident Loss to mitigate bias in description dataset
- Outperforms prior work in describing images with people and mentioning their gender
- Matches ground truth ratio of sentences including women to sentences including men closely
- Model more frequently looks at people when predicting their gender, relying less on contextual cues
- Offers an effective approach for generating gender-specific caption words based on appearance or image context

## Summary

In their paper titled "Women also Snowboard: Overcoming Bias in Captioning Models," authors Kaylee Burns, Lisa Anne Hendricks, Trevor Darrell, and Anna Rohrbach address the issue of biases in machine learning methods, particularly in image captioning models. While some biases can be beneficial for learning, others can be harmful. The authors highlight that image captioning models tend to amplify biases present in the training data, leading to incorrect captions in domains where unbiased captions are desired. To tackle this problem, the authors propose a new model called the Equalizer model. This model ensures equal gender probability when gender evidence is occluded in a scene and provides confident predictions when gender evidence is present. By doing so, the model focuses on looking at a person rather than relying solely on contextual cues to make gender-specific predictions. The proposed model incorporates two losses: the Appearance Confusion Loss and the Confident Loss. These losses can be added to any description model to mitigate unwanted bias in a description dataset. The authors demonstrate that their model outperforms prior work when describing images with people and mentioning their gender. It also closely matches the ground truth ratio of sentences including women to sentences including men. Furthermore, unlike other approaches, the authors show that their model more frequently looks at people when predicting their gender. This indicates that it relies less on contextual cues and instead focuses on visual evidence from individuals. Overall, this research contributes to addressing biases in machine learning models by proposing an effective approach for generating gender-specific caption words based on appearance or image context. The proposed Equalizer model offers a promising solution for achieving unbiased image captions and improving accuracy in describing images with people while considering their gender.