# Finding Experts in Transformer Models

**Authors:** Xavier Suau, Luca Zappella, Nicholas Apostoloff  
**arXiv:** [2005.07647](https://arxiv.org/abs/2005.07647)  
**Date:** 2020-05-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2005.07647v1/

## Key points

- The research paper explores the presence of expert units in pre-trained Transformer Models (TM) and how they impact a model's performance.
- Expert units are defined as neurons that can classify a concept with a given average precision, where a concept is represented by a binary set of sentences containing the concept or not.
- The authors leverage the OneSec dataset to compile a dataset of diverse expert units in TM, creating 1641 concepts for analysis.
- The presence of expert units is correlated with the generalization power of TM, allowing ranking without requiring fine-tuning on downstream tasks.
- The overlap of top experts between concepts provides a sensible way to quantify concept co-learning that can be used for explainability of unknown concepts.
- Self-conditioning off-the-shelf pre-trained language models to generate text with a given concept is possible by forcing top experts to be active without retraining or using additional parameters.
- Ranking units by APm_c is an effective strategy for finding experts for concept c and proves that expert units are causal in LM settings.
- LMs dedicate very selective units to specific concepts.

## Summary

In their research paper titled "Finding Experts in Transformer Models," Xavier Suau, Luca Zappella, and Nicholas Apostoloff explore the presence of expert units in pre-trained Transformer Models (TM) and how they impact a model's performance. The authors define expert units as neurons that can classify a concept with a given average precision, where a concept is represented by a binary set of sentences containing the concept or not. To compile a dataset of diverse expert units in TM, the authors leverage the OneSec dataset and create 1641 concepts for analysis. The paper highlights several ways in which expert units are important. Firstly, the presence of expert units is correlated with the generalization power of TM, allowing ranking without requiring fine-tuning on downstream tasks. The authors propose an empirical method to decide how accurate such experts should be to evaluate generalization. Secondly, the overlap of top experts between concepts provides a sensible way to quantify concept co-learning that can be used for explainability of unknown concepts. Finally, the authors show how to self-condition off-the-shelf pre-trained language models to generate text with a given concept by forcing top experts to be active without retraining or using additional parameters. In addition to these findings, the authors also explore parallels between TMs' behavior and human brains' behavior and demonstrate through experiments that ranking units by APm_c is an effective strategy for finding experts for concept c and prove that expert units are causal in LM settings. Through Table 4 examples, they illustrate how LMs dedicate very selective units to specific concepts. Overall, this research sheds light on the importance of expert units in TMs and their potential applications beyond model evaluation for text generation and explainability purposes.