# CIDEr: Consensus-based Image Description Evaluation

**Authors:** Ramakrishna Vedantam, C. Lawrence Zitnick, Devi Parikh  
**arXiv:** [1411.5726](https://arxiv.org/abs/1411.5726)  
**Date:** 2014-11-20  
**Summary page:** https://www.summarizepaper.com/en/paper/1411.5726v1/

## Key points

- Authors Vedantam, Zitnick, and Parikh address the challenge of automatically describing images with sentences in computer vision and natural language processing fields.
- They propose a novel paradigm for evaluating image descriptions based on human consensus, consisting of three key components: triplet-based method for collecting human annotations, automated metric to capture consensus effectively, and introduction of new datasets PASCAL-50S and ABSTRACT-50S.
- Their simple metric outperforms existing metrics in capturing human judgment of consensus across sentences from various sources.
- The study evaluates five state-of-the-art image description approaches using this protocol and establishes a benchmark for future comparisons in the field.
- The research advances evaluation methodologies for image descriptions and emphasizes the importance of incorporating human consensus into assessing description quality.

## Summary

In their paper titled "CIDEr: Consensus-based Image Description Evaluation," authors Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh address the challenge of automatically describing images with sentences in the fields of computer vision and natural language processing. They highlight recent advancements in object detection, attribute classification, action recognition, and other areas that have sparked a renewed interest in this domain. To tackle this challenge, the authors propose a novel paradigm for evaluating image descriptions based on human consensus. This paradigm comprises three key components: a new triplet-based method for collecting human annotations to measure consensus, an automated metric designed to capture consensus effectively, and the introduction of two new datasets - PASCAL-50S and ABSTRACT-50S - each containing 50 sentences describing individual images. The authors' simple metric demonstrates superior performance in capturing human judgment of consensus compared to existing metrics when applied across sentences generated by various sources. Furthermore, the study evaluates five state-of-the-art image description approaches using this innovative protocol and establishes a benchmark for future comparisons in the field. This research not only contributes to advancing evaluation methodologies for image descriptions but also sheds light on the importance of incorporating human consensus into assessing the quality of such descriptions. Through their work, Vedantam, Zitnick, and Parikh provide valuable insights that can guide future research endeavors in computer vision and natural language processing.