# Engineering Monosemanticity in Toy Models

**Authors:** Adam S. Jermyn, Nicholas Schiefer, Evan Hubinger  
**arXiv:** [2211.09169](https://arxiv.org/abs/2211.09169)  
**Date:** 2022-11-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.09169v1/

## Key points

- Monosemantic neurons in neural networks correspond to natural features in input data and are crucial for interpretability studies.
- Altering the local minimum found during training can increase a model's monosemantic nature without impacting loss.
- Models with more monosemantic loss minima have slight negative biases, which can be leveraged to create highly monosemantic models.
- Increasing the number of neurons per layer enhances monosemanticity but comes with higher computational costs.
- The study provides insights into how to engineer monosemanticity effectively in neural networks, opening up new avenues for research.

## Summary

In the paper "Engineering Monosemanticity in Toy Models," authors Adam S. Jermyn, Nicholas Schiefer, and Evan Hubinger explore the concept of monosemantic neurons in neural networks. These neurons correspond to natural features in the input data and are crucial for interpretability studies due to their clear understanding. The researchers conducted preliminary experiments to engineer monosemanticity in toy models and discovered that altering the local minimum found during training can increase the model's monosemantic nature without impacting loss. They observed that more monosemantic loss minima have slight negative biases, which they leveraged to create highly monosemantic models. Furthermore, the authors were able to mechanistically interpret these models, including residual polysemantic neurons, revealing a simple yet surprising algorithm. They also found that increasing the number of neurons per layer enhances monosemanticity in models but comes with higher computational costs. These findings open up new avenues for exploring and engineering monosemanticity in neural networks. The paper was primarily written and illustrated by Adam S. Jermyn, with contributions from Nicholas Schiefer and Evan Hubinger. The authors acknowledge Chris Olah for his encouragement and valuable suggestions, as well as other colleagues for their discussions on various aspects of the project. Training details included using the LAMB optimizer with batch sizes optimized for GPU usage. Overall, this study sheds light on the importance of monosemantic neurons in neural networks and provides insights into how they can be engineered effectively. The findings pave the way for future research directions aimed at further enhancing interpretability and performance in machine learning models.