# NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance

**Authors:** Raphael T. Husistein, Markus Reiher, Marco Eckhoff  
**arXiv:** [2408.08776](https://arxiv.org/abs/2408.08776)  
**Date:** 2024-08-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2408.08776v2/

## Key points

- Authors Raphael T. Husistein, Markus Reiher, and Marco Eckhoff focus on artificial neural networks in their paper "NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance."
- Challenges in constructing high-performing neural networks are highlighted due to the laborious nature and substantial computing power required.
- Introduction of "Near," a training-free pre-estimator that automatically selects optimal network architectures using zero-cost proxies.
- Traditional Neural Architecture Search (NAS) methods involve training multiple networks, but "Near" estimates network expressivity scores without training data by utilizing activation rank within matrices.
- Effectiveness of "Near" demonstrated through strong correlation with model accuracy on benchmark datasets like NAS-Bench-101 and NATS-Bench-SSS/TSS.
- Proposal of estimating optimal layer sizes in multi-layer perceptrons using "Near" scores and informing decisions on hyperparameters like activation functions and weight initialization schemes.
- Study emphasizes the potential of "Near" as a valuable tool for automated model selection processes in machine learning research and application domains.

## Summary

In their paper titled "NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance," authors Raphael T. Husistein, Markus Reiher, and Marco Eckhoff delve into the realm of artificial neural networks and their significance as cutting-edge machine learning models across various applications such as natural language processing and image recognition. The authors highlight the challenges associated with constructing high-performing neural networks, emphasizing the laborious nature of this task and the substantial computing power required. To address these challenges, the authors introduce <keyword>Near</keyword>, a training-free pre-estimator that leverages zero-cost proxies to automatically select optimal network architectures from a pool of potential candidates. Traditional methods for <keyword>Neural Architecture Search (NAS)</keyword> involve training multiple neural networks to identify the best architecture, which can be time-consuming and computationally expensive. However, <keyword>Near</keyword> offers a promising alternative by utilizing activation rank within pre- and post-activation matrices to estimate network expressivity scores without requiring any training data. The effectiveness of <keyword>Near</keyword> is demonstrated through its strong correlation with model accuracy on benchmark datasets such as NAS-Bench-101 and NATS-Bench-SSS/TSS. Additionally, the authors propose a straightforward approach for estimating optimal layer sizes in multi-layer perceptrons using <keyword>Near</keyword> scores. They also showcase how <keyword>Near</keyword> can inform decisions regarding hyperparameters like activation functions and weight initialization schemes in neural networks. Overall, this study sheds light on the potential of <keyword>Near</keyword> as a training-free pre-estimator for machine learning model performance. Its findings contribute to advancing automated model selection processes in machine learning research and application domains. With <keyword>Near</keyword>, researchers and practitioners can optimize neural network architectures with minimal computational overhead, making it a valuable tool in the field of artificial intelligence.