# Sample, estimate, aggregate: A recipe for causal discovery foundation models

**Authors:** Menghua Wu, Yujia Bao, Regina Barzilay, Tommi Jaakkola  
**arXiv:** [2402.01929](https://arxiv.org/abs/2402.01929)  
**Date:** 2024-02-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.01929v1/

## Key points

- Causal discovery is essential for scientific research and policy decisions
- Existing algorithms are slow, data hungry, and brittle
- A new approach inspired by foundation models has been proposed
- Pretraining a deep learning model to analyze predictions from classical algorithms on smaller subsets of variables
- Efficiency in computing outputs for small problems, insights into marginal data structure, and consistent structural outputs across datasets are key aspects of the method
- Achieves state-of-the-art performance on synthetic and realistic datasets with robust generalization capabilities
- Significantly improved inference speeds compared to existing models
- Outperforms traditional continuous optimization methods with only around 500 data samples for acceptable performance on graphs with 100 nodes
- The Sample, Estimate, Aggregate (SEA) framework offers promising advancements in causal discovery research

## Summary

, , , , 

In the field of causal discovery, the task of inferring causal relationships from data is crucial for advancing scientific research and informing policy decisions. Existing algorithms are limited by their per-dataset nature, making them slow, data hungry, and prone to brittleness. To address these challenges, a new approach inspired by foundation models has been proposed. This novel framework involves pretraining a deep learning model to analyze predictions generated by classical algorithms applied to smaller subsets of variables. The key insight behind this method lies in the efficiency of computing outputs from classical algorithms for small problems, their ability to provide valuable insights into marginal data structure, and the consistency of their structural outputs across different datasets. By leveraging these strengths, the proposed framework achieves state-of-the-art performance on both synthetic and realistic datasets. Importantly, it demonstrates robust generalization capabilities to data generating mechanisms not encountered during training. One notable advantage of this approach is its significantly improved inference speeds compared to existing models. The method offers orders of magnitude faster computation times while maintaining high levels of accuracy and reliability in causal structure inference. Experimental results show that the model outperforms traditional continuous optimization methods by requiring only around 500 data samples for acceptable performance on graphs with 100 nodes. Overall, the Sample, Estimate, Aggregate (SEA) framework represents a promising advancement in causal discovery research. By combining deep learning techniques with classical algorithms in a novel way, this approach opens up new possibilities for accelerating scientific discoveries and facilitating evidence-based decision-making processes.