# Neural Architecture Search: Insights from 1000 Papers

**Authors:** Colin White, Mahmoud Safari, Rhea Sukthanker, Binxin Ru, Thomas Elsken, Arber Zela, Debadeepta Dey, Frank Hutter  
**arXiv:** [2301.08727](https://arxiv.org/abs/2301.08727)  
**Date:** 2023-01-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2301.08727v1/

## Key points

- Deep learning has made significant advancements in various fields such as computer vision, natural language understanding, speech recognition, and reinforcement learning
- Designing specialized neural architectures manually can be complex and time-consuming
- Neural Architecture Search (NAS) automates the process of designing neural architectures for specific tasks
- NAS has surpassed human-designed architectures on many tasks
- Over 1000 papers on NAS have been released since 2020
- This survey provides a comprehensive guide to NAS, including a taxonomy of search spaces, algorithms, and speedup techniques used in NAS
- The survey also discusses valuable resources such as benchmarks, best practices, other surveys, and open source libraries related to NAS
- NAS is significant in advancing deep learning and offers promising opportunities for further advancements in machine learning.

## Summary

In the past decade, deep learning has made significant advancements in various fields such as computer vision, natural language understanding, speech recognition, and reinforcement learning. These breakthroughs have been enabled by specialized and high-performing neural architectures. However, designing these architectures manually can be a complex and time-consuming task. To address this challenge, researchers have turned to Neural Architecture Search (NAS), which automates the process of designing neural architectures for specific tasks. NAS has emerged as a crucial next step in automating machine learning and has already surpassed human-designed architectures on many tasks. In recent years, research in NAS has progressed rapidly with over 1000 papers released since 2020. To provide a comprehensive guide to NAS, this survey offers an organized overview of the field. It presents a taxonomy of search spaces, algorithms and speedup techniques used in NAS. Additionally it discusses valuable resources such as benchmarks, best practices, other surveys and open source libraries that can aid researchers and practitioners in their exploration of NAS. Overall this survey highlights the significance of NAS in advancing deep learning and provides insights from a vast body of research. By automating the design process of neural architectures for specific tasks NAS offers promising opportunities for further advancements in machine learning.