# Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small

**Authors:** Kevin Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, Jacob Steinhardt  
**arXiv:** [2211.00593](https://arxiv.org/abs/2211.00593)  
**Date:** 2022-11-01  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.00593v1/

## Key points

- Research focuses on achieving mechanistic interpretability in machine learning models
- Specifically, focuses on GPT-2 small performing indirect object identification (IOI) task
- Previous studies have focused on simple behaviors in smaller models or broad descriptions of complex behaviors in larger models
- Researchers present an explanation for how GPT-2 small performs IOI by identifying 26 attention heads grouped into 7 main classes using interpretability approaches that rely on causal interventions
- Evaluation of explanation using three quantitative criteria: faithfulness, completeness, and minimality
- Feasibility of achieving mechanistic understanding of large machine learning models demonstrated
- Opportunities to scale understanding to larger models and more complex tasks
- Background information provided on IOI task, transformer architecture used in GPT-2 small, and technique for "knocking out" nodes in a model
- Contributes to advancing understanding of machine learning models and their internal mechanisms

## Summary

, , , , 

This research focuses on achieving mechanistic interpretability in machine learning models, specifically in the context of GPT-2 small performing a natural language task known as indirect object identification (IOI). The ultimate goal is to gain a deeper understanding of the model's behaviors by examining its internal components. Previous studies have either focused on simple behaviors in smaller models or provided broad descriptions of complex behaviors in larger models. To bridge this gap, the researchers present an explanation for how GPT-2 small performs IOI by identifying 26 attention heads grouped into 7 main classes using interpretability approaches that rely on causal interventions. This investigation is considered the largest attempt at reverse-engineering a natural behavior "in the wild" within a language model. The reliability of their explanation is evaluated using three quantitative criteria: faithfulness, completeness, and minimality. While these criteria support their explanation, they also highlight remaining gaps in understanding. This work provides evidence that it is feasible to achieve a mechanistic understanding of large machine learning models and opens up opportunities to scale this understanding to larger models and more complex tasks. In terms of background, the paper introduces the IOI task and provides an overview of the transformer architecture used in GPT-2 small. It also defines circuits more formally and describes a technique for "knocking out" nodes in a model. Overall, this research contributes to advancing our understanding of machine learning models and their internal mechanisms, paving the way for further exploration and improvement in this field.