# Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis

**Authors:** Akarsh Kumar, Jeff Clune, Joel Lehman, Kenneth O. Stanley  
**arXiv:** [2505.11581](https://arxiv.org/abs/2505.11581)  
**Date:** 2025-05-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2505.11581v1/

## Key points

- Growing interest in scaling up existing systems for improved performance in artificial intelligence
- Debate on whether better performance always means better internal representations
- Comparison between neural networks evolved through open-ended search processes and those trained using stochastic gradient descent (SGD)
- Visualization of hidden neurons' functional behaviors as images to examine internal construction of output behavior
- Significant differences found in internal representations between networks trained with SGD and evolved networks
- Networks trained with SGD exhibit disorganization, potentially degrading core model capacities like generalization and creativity
- Evolved networks tend towards a unified factored representation (UFR), which could be crucial for future representation learning
- Acknowledgment of contributions from various individuals and research projects, including insights on vector rotation and discussions on Mixture of Experts
- Highlighting previous studies on elegant representations found in Picbreeder CPPNs and support from funding sources like NSF GRFP Fellowship and Canada CIFAR AI Chairs program
- Discussion on findings related to neural algorithms in models like GPT-3 and GPT-4, showcasing evidence of FER in certain contexts
- References to studies on shortcut learning and heuristic-based approaches in model training, emphasizing the importance of understanding and mitigating FER for improved model performance

## Summary

In the field of artificial intelligence, there is a growing interest in the idea that scaling up existing systems can lead to improved performance. However, the question arises: does better performance necessarily mean better internal representations? While some believe that it does, this position paper challenges that assumption. The paper compares neural networks evolved through an open-ended search process to networks trained using stochastic gradient descent (SGD) on the task of generating a single image. This setup allows for easy visualization of each hidden neuron's functional behavior as an image, revealing how the network's output behavior is constructed internally. The results show that while both types of networks produce the same output behavior, their internal representations differ significantly. has seen a surge in interest regarding  existing systems to improve performance. However,  have become a topic of debate - does better performance always equate to better internal representations? This paper challenges this notion by comparing neural networks evolved through open-ended search processes with those trained using stochastic gradient descent (SGD). By visualizing hidden neurons' functional behaviors as images and examining their internal construction of output behavior, it reveals significant differences between the two types of networks. Networks trained with SGD exhibit a form of disorganization termed , which may degrade core model capacities such as generalization and creativity. On the other hand, evolved networks tend towards a unified factored representation (UFR), which could be crucial for future representation learning. The paper acknowledges contributions from various individuals and research projects, including insights on vector rotation and discussions on Mixture of Experts. The paper also highlights previous studies on elegant representations found in Picbreeder CPPNs and mentions support from funding sources like NSF GRFP Fellowship and Canada CIFAR AI Chairs program. Additionally, the paper discusses findings related to neural algorithms in models like GPT-3 and GPT-4, showcasing evidence of FER in certain contexts. It also references studies on shortcut learning and heuristic-based approaches in model training, emphasizing the importance of understanding and mitigating FER for improved model performance. In summary, this detailed analysis delves into the challenges posed by  in neural networks and emphasizes the significance of addressing this issue for advancing representation learning in .