# Testing the Tools of Systems Neuroscience on Artificial Neural Networks

**Authors:** Grace W. Lindsay  
**arXiv:** [2202.07035](https://arxiv.org/abs/2202.07035)  
**Date:** 2022-02-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2202.07035v1/

## Key points

- Neuroscientists face challenges in understanding the complex workings of the brain
- Analyzing neural activity during tasks can provide insights into neural circuit computations
- Lack of empirical evidence regarding effectiveness of common analysis tools
- Suggests testing analysis tools on artificial neural networks (ANNs)
- ANNs are popular models for cognitive functions and offer perfect observability and manipulability
- Proposes roadmap for testing tools on ANNs and provides a list of suitable tools for evaluation
- Using ANNs as reflection tool can accelerate progress in brain research
- Need reliable tools for understanding the brain, not just finding stories within data
- Define desired understanding and establish evaluation framework for analysis methods
- Iteratively test analysis methods on ANNs, refine them based on insights gained, and use refined tools to guide experimental design
- Example of using demixed PCA (dPCA) to analyze neural activity in an ANN trained to classify images with noise added
- Testing common analysis tools on ANNs promotes transparency and provides framework for evaluating explanations derived from these tools

## Summary

, , , , 

Neuroscientists face significant challenges in understanding the complex workings of the brain. The brain is a highly evolved system that operates on multiple temporal and spatial scales, making it difficult to comprehend. Analyzing neural activity recorded during various tasks can provide insights into how neural circuits perform computations. However, there is a lack of empirical evidence regarding the effectiveness of common analysis tools in quickly identifying relevant phenomena. To address this issue, the author argues for explicit testing of these analysis tools using artificial neural networks (ANNs) as a suitable testing ground. ANNs have gained popularity as models for various cognitive functions due to their similarities with biological neural networks and their ability to be trained for complex tasks. Additionally, ANNs offer the advantage of perfect observability and manipulability, making them ideal for evaluating tools used in systems and cognitive neuroscience. The author proposes a roadmap for testing these tools on ANNs and provides a list of suitable tools for evaluation. By using ANNs as a reflection tool, researchers can determine the extent to which these analysis methods contribute to a productive understanding of neural systems. This approach has the potential to accelerate progress in brain research. Understanding the brain requires more than just finding stories within high-dimensional nonlinear systems; it necessitates reliable tools that yield accurate explanations. Without well-honed analysis methods, scientists risk wasting valuable time and resources by compounding errors. Previous reflections on the state of methods in systems biology and neuroscience raise concerns about relying solely on informal measures of progress. To assess the quality and depth of explanations provided by these tools, neuroscientists need to define what kind of understanding they are aiming for and establish an evaluation framework. Typically, researchers seek abstracted descriptions or algorithmic-level understandings that capture how neural systems process information to achieve computational goals. These simplified mechanistic stories allow scientists to think through the system and make new experimental predictions while disregarding certain complexities present in data. The proposed approach involves iteratively testing analysis methods on ANNs, refining them based on the insights gained, and using these refined tools to guide experimental design. This process creates a feedback loop that enhances both the development of analysis methods and the confirmation of insights they provide. An example of this iterative procedure is presented, where an ANN trained to classify images of hand-written digits with different types of noise added is analyzed using demixed PCA (dPCA). dPCA aims to reduce population activity into interpretable components related to task parameters. By applying dPCA to neural activity at a specific network layer, researchers can obtain principal components ranked by variance explained and attribute each component's variance proportion to specific task parameters. In conclusion, explicitly testing common analysis tools on ANNs can help determine their effectiveness in understanding neural systems. This approach promotes transparency and provides a framework for evaluating the quality and depth of explanations derived from these tools. By leveraging ANNs as models for brain function, researchers can expedite progress in studying the complexities of the brain and gain valuable insights into how neural circuits implement computations.