# Bayesian Optimization of Catalysts With In-context Learning

**Authors:** Mayk Caldas Ramos, Shane S. Michtavy, Marc D. Porosoff, Andrew D. White  
**arXiv:** [2304.05341](https://arxiv.org/abs/2304.05341)  
**Date:** 2023-04-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.05341v1/

## Key points

- The paper presents a novel approach to optimize catalysts and molecules using natural language processing (NLP) and large language models (LLMs).
- A prompting system is introduced that enables regression with uncertainty for in-context learning with frozen LLM models such as GPT-3, GPT-3.5, and GPT-4.
- The proposed method incorporates uncertainty which enables Bayesian optimization for catalyst or molecule optimization using NLP.
- This eliminates the need for training or simulation, making it a more efficient and cost-effective approach.
- In-context learning can improve past a model context window as data is gathered via example selection allowing the model to scale better.
- Although their method does not outperform all baselines, it requires zero training, feature selection and minimal computing while maintaining satisfactory performance.
- Gaussian Process Regression on text embeddings is found to be strong at Bayesian optimization.
- The code used in this study is available on their GitHub repository.
- Overall, this paper presents an innovative approach to optimize catalysts and molecules using NLP and LLMs without requiring extensive training or simulation. It has potential applications in various fields such as chemistry and materials science where optimizing catalysts is crucial for improving processes' efficiency and reducing costs.

## Summary

The paper titled "Bayesian Optimization of Catalysts with In-context Learning" presents a novel approach to optimize catalysts and molecules using natural language processing (NLP) and large language models (LLMs). The authors introduce a prompting system that enables regression with uncertainty for in-context learning with frozen LLM models such as GPT-3, GPT-3.5, and GPT-4. This allows predictions without the need for feature selection or architecture tuning. The proposed method incorporates uncertainty which enables Bayesian optimization for catalyst or molecule optimization using NLP. This eliminates the need for training or simulation, making it a more efficient and cost-effective approach. The study shows that in-context learning can improve past a model context window as data is gathered via example selection allowing the model to scale better. Although their method does not outperform all baselines, it requires zero training, feature selection and minimal computing while maintaining satisfactory performance. Furthermore, Gaussian Process Regression on text embeddings is found to be strong at Bayesian optimization. The code used in this study is available on their GitHub repository. Overall, this paper presents an innovative approach to optimize catalysts and molecules using NLP and LLMs without requiring extensive training or simulation. It has potential applications in various fields such as chemistry and materials science where optimizing catalysts is crucial for improving processes' efficiency and reducing costs.