# TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second

**Authors:** Noah Hollmann, Samuel Müller, Katharina Eggensperger, Frank Hutter  
**arXiv:** [2207.01848](https://arxiv.org/abs/2207.01848)  
**Date:** 2022-07-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2207.01848v5/

## Key points

- TabPFN is a Prior-Data Fitted Network (PFN) that approximates Bayesian inference on synthetic datasets drawn from the team's prior.
- Developed by Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter.
- The network requires no hyperparameter tuning and is competitive with state-of-the-art classification methods.
- The network accepts training and test samples as a set-valued input to yield predictions for the entire test set in a single forward pass.
- The prior incorporates ideas from causal reasoning and entails a large space of structural causal models with a preference for simple structures.
- Validated on 18 datasets in the OpenML-CC18 suite containing up to 1,000 training data points, up to 100 purely numerical features without missing values, and up to 10 classes.
- Outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with up to 70 times speedup. When using a GPU, this speedup increases to 3200 times.
- Validated results on an additional 67 small numerical datasets from OpenML.
- Provides all their code, the trained TabPFN model, an interactive browser demo, and a Colab notebook at https://github.com/automl/TabPFN.
- Offers an efficient solution for small tabular classification problems that require minimal setup time while delivering high-quality results in less than a second without any hyperparameter tuning.

## Summary

TabPFN is a Prior-Data Fitted Network (PFN) that approximates Bayesian inference on synthetic datasets drawn from the team's prior. Developed by Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter, this trained Transformer requires no hyperparameter tuning and is competitive with state-of-the-art classification methods. The network is fully entailed in the weights of the system and accepts training and test samples as a set-valued input to yield predictions for the entire test set in a single forward pass. This prior incorporates ideas from causal reasoning and entails a large space of structural causal models with a preference for simple structures. The team validated their approach on 18 datasets in the OpenML-CC18 suite containing up to 1,000 training data points, up to 100 purely numerical features without missing values, and up to 10 classes. They showed that TabPFN outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with up to 70 times speedup. When using a GPU, this speedup increases to 3200 times. The team also validated their results on an additional 67 small numerical datasets from OpenML. They provide all their code, the trained TabPFN model, an interactive browser demo, and a Colab notebook at https://github.com/automl/TabPFN. Overall, TabPFN offers an efficient solution for small tabular classification problems that require minimal setup time while delivering high-quality results in less than a second without any hyperparameter tuning.