# Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

**Authors:** Amir Feder, Katherine A. Keith, Emaad Manzoor, Reid Pryzant, Dhanya Sridhar, Zach Wood-Doughty, Jacob Eisenstein, Justin Grimmer, Roi Reichart, Margaret E. Roberts, Brandon M. Stewart, Victor Veitch, Diyi Yang  
**arXiv:** [2109.00725](https://arxiv.org/abs/2109.00725)  
**Date:** 2021-09-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2109.00725v2/

## Key points

- Causal inference and natural language processing (NLP) are intersecting fields of research.
- Causality has been extensively studied in life and social sciences but not as much in NLP.
- There is an emerging area of interdisciplinary research combining causal inference and language processing.
- Existing research on causality in NLP is scattered across different domains, lacking unified definitions, benchmark datasets, and clear articulations of challenges and opportunities.
- This survey consolidates research from various academic areas and situates it within the broader NLP landscape.
- The paper discusses the statistical challenge of estimating causal effects with text, covering scenarios where text is used as an outcome, treatment, or to address confounding.
- Potential applications of causal inference in improving the robustness, fairness, and interpretability of NLP models are explored.
- The concern about unobserved confounding between explicitly considered variables when using partial causal models is highlighted.
- Using causal methodology forces practitioners to explicate their assumptions, which should be clearer in the NLP community for better scientific standards and understanding of language models.
- The survey provides insights into how recent advances in NLP modeling can help draw causal conclusions with text data while highlighting challenges and open questions.

## Summary

This survey paper explores the intersection of causal inference and natural language processing (NLP) and aims to provide a unified overview of causal inference for the NLP community. While causality has been extensively studied in the life and social sciences, it has not received the same level of attention in NLP, which has traditionally focused more on predictive tasks. However, there is an emerging area of interdisciplinary research that combines causal inference and language processing. The existing research on causality in NLP is scattered across different domains, lacking unified definitions, benchmark datasets, and clear articulations of challenges and opportunities specific to applying causal inference to textual data. This survey consolidates research from various academic areas and situates it within the broader NLP landscape. The paper introduces the statistical challenge of estimating causal effects with text, covering scenarios where text is used as an outcome, treatment, or to address confounding. It also explores potential applications of causal inference in improving the robustness, fairness, and interpretability of NLP models. One critical question for future work is how to safely use partial causal models that omit some variables and do not fully specify the causal relationships within text. The concern here is unobserved confounding between explicitly considered variables. The authors emphasize that using causal methodology forces practitioners to explicate their assumptions. They believe that the NLP community should be clearer about these assumptions and analyze their data using causal reasoning to improve scientific standards and gain a better understanding of language and the models built to process it. Overall, this survey provides insights into how recent advances in NLP modeling can help researchers draw causal conclusions with text data while highlighting challenges and open questions in this field.