# Foundation Models for Generalist Geospatial Artificial Intelligence

**Authors:** Johannes Jakubik, Sujit Roy, C. E. Phillips, Paolo Fraccaro, Denys Godwin, Bianca Zadrozny, Daniela Szwarcman, Carlos Gomes, Gabby Nyirjesy, Blair Edwards, Daiki Kimura, Naomi Simumba, Linsong Chu, S. Karthik Mukkavilli, Devyani Lambhate, Kamal Das, Ranjini Bangalore, Dario Oliveira, Michal Muszynski, Kumar Ankur  
**arXiv:** [2310.18660](https://arxiv.org/abs/2310.18660)  
**Date:** 2023-10-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.18660v1/

## Key points

- Significant progress in the development of highly adaptable and reusable AI models is poised to revolutionize Earth science and remote sensing.
- Introduction of a groundbreaking framework for efficient pre-training and fine-tuning of foundational models on extensive geospatial data, leading to the creation of Prithvi, a transformer-based geospatial foundational model.
- Prithvi pre-trained on over 1TB of multispectral satellite imagery from the Harmonized Landsat-Sentinel 2 (HLS) dataset.
- Effectiveness of the framework demonstrated through successful fine-tuning of Prithvi for various Earth observation tasks including multi-temporal cloud gap imputation, flood mapping, wildfire scar segmentation, and multi-temporal crop segmentation.
- Pre-trained model accelerates fine-tuning process compared to randomly initialized weights and outperforms existing models like conditional GAN in multi-temporal cloud imputation by up to 5pp (or 5.7%) in structural similarity index.
- Study shows that labeled data can be significantly reduced without compromising model accuracy due to scarcity in Earth observation data.
- Pre-trained 100 million parameter model and corresponding fine-tuning workflows made publicly available as open-source contributions through Hugging Face for global Earth sciences community benefit.

## Summary

Significant progress in the development of highly adaptable and reusable Artificial Intelligence (AI) models is poised to revolutionize Earth science and remote sensing. This paper introduces a groundbreaking framework for the efficient pre-training and fine-tuning of foundational models on extensive geospatial data, leading to the creation of Prithvi, a transformer-based geospatial foundational model pre-trained on over 1TB of multispectral satellite imagery from the Harmonized Landsat-Sentinel 2 (HLS) dataset. The study showcases the effectiveness of this framework by successfully fine-tuning Prithvi for a variety of Earth observation tasks previously unexplored by foundation models. These tasks include multi-temporal cloud gap imputation, flood mapping, wildfire scar segmentation, and multi-temporal crop segmentation. The experiments demonstrate that the pre-trained model accelerates the fine-tuning process compared to randomly initialized weights and outperforms existing models like a conditional GAN in multi-temporal cloud imputation by up to 5pp (or 5.7%) in the structural similarity index. Moreover, due to the scarcity of labeled data in Earth observation, the study gradually reduces available labeled data for refining the model to assess data efficiency. Surprisingly, it shows that data can be significantly reduced without compromising model accuracy. The pre-trained 100 million parameter model and its corresponding fine-tuning workflows have been made publicly available as open-source contributions through Hugging Face to benefit the global Earth sciences community.