# Sparse Autoencoders Can Interpret Randomly Initialized Transformers

**Authors:** Thomas Heap, Tim Lawson, Lucy Farnik, Laurence Aitchison  
**arXiv:** [2501.17727](https://arxiv.org/abs/2501.17727)  
**Date:** 2025-01-29  
**Summary page:** https://www.summarizepaper.com/en/paper/2501.17727v1/

## Key points

- Researchers explored the application of sparse autoencoders (SAEs) in interpreting randomly initialized transformers
- Concept of auto-interpretability involves using feature explanations to identify specific concepts within text for classification tasks
- 'Fuzzing' scoring method and simulation scoring were used to evaluate feature explanations
- Both random and trained transformers yield similarly interpretable SAE latents when analyzed using an open-source auto-interpretability pipeline
- SAE quality metrics are comparable between random and trained transformers across different model sizes and layers
- Auto-interpretability scores were notably consistent between trained and randomized transformer models compared to control conditions, indicating potential of SAEs in interpreting transformer models regardless of their initialization method

## Summary

In the study "Sparse Autoencoders Can Interpret Randomly Initialized Transformers," researchers Thomas Heap, Tim Lawson, Lucy Farnik, and Laurence Aitchison explore the application of sparse autoencoders (SAEs) in interpreting randomly initialized transformers. The authors delve into the concept of auto-interpretability, which involves using feature explanations to identify specific concepts within text for classification tasks. They employed the 'fuzzing' scoring method to evaluate feature explanations by prompting a language model to distinguish correctly delimited examples of tokens with non-zero and zero activation values for a given latent explanation. Furthermore, simulation scoring based on the correlation between simulated and observed activations was utilized for evaluation. The researchers found that both random and trained transformers yield similarly interpretable SAE latents when analyzed using an open-source auto-interpretability pipeline. Additionally, they observed that SAE quality metrics are comparable between random and trained transformers across different model sizes and layers. By randomly sampling 100 features from each trained SAE model variant and layer, the researchers generated auto-interpretability scores using an implementation based on previous work by Paulo et al. The Meta-Llama-3.1-70B-Instruct-AWQ-INT4 model was employed for generating explanations and predictions, showcasing larger capacity compared to previous models used in similar studies. Overall, the study revealed that auto-interpretability scores were notably consistent between trained and randomized transformer models compared to control conditions. These findings shed light on the potential of SAEs in interpreting transformer models regardless of their initialization method, raising intriguing questions about mechanistic interpretability in natural language processing applications.