# DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents

**Authors:** Peter Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Oyvind Tafjord, Peter Clark  
**arXiv:** [2406.06769](https://arxiv.org/abs/2406.06769)  
**Date:** 2024-06-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.06769v1/

## Key points

- DISCOVERYWORLD is a revolutionary platform designed to accelerate progress in scientific domains by evaluating AI agents' capacity for end-to-end scientific reasoning.
- It offers a diverse range of challenges and tasks covering topics such as radioisotope dating, rocket science, and proteomics.
- The platform features 120 different challenge tasks across eight topics with varying levels of difficulty and parametric variations.
- Tasks require agents to formulate hypotheses, design experiments, analyze results, and draw conclusions based on their findings.
- DISCOVERYWORLD provides three automatic metrics focusing on task completion, task-relevant actions taken, and the discovered explanatory knowledge.
- Strong baseline agents that have performed well in previous environments struggle with most tasks in DISCOVERYWORLD, indicating unique challenges related to scientific discovery processes.
- Human evaluation plays a crucial role in assessing agent performance across different thematic areas within the realm of scientific research.
- The platform serves as an innovative tool for accelerating the development and assessment of generalizable discovery skills in AI agents.

## Summary

DISCOVERYWORLD is a revolutionary platform designed to accelerate progress in scientific domains by evaluating AI agents' capacity for end-to-end scientific reasoning. It offers a diverse range of challenges and tasks, covering topics such as radioisotope dating, rocket science, and proteomics. With its simulated text-based environment and optional 2D visual overlay, DISCOVERYWORLD provides a cost-effective and accessible tool for testing agents' capabilities. The platform features 120 different challenge tasks across eight topics with varying levels of difficulty and parametric variations. These tasks require agents to formulate hypotheses, design experiments, analyze results, and draw conclusions based on their findings. To assess performance, DISCOVERYWORLD provides three automatic metrics focusing on task completion, task-relevant actions taken, and the discovered explanatory knowledge. Interestingly, even strong baseline agents that have performed well in previous environments struggle with most tasks in DISCOVERYWORLD. This suggests that the platform presents unique challenges related to scientific discovery processes that may not be adequately captured in other settings. In addition to automated metrics, human evaluation plays a crucial role in assessing agent performance across different thematic areas within the realm of scientific research. Each theme presents specific challenges that require agents to apply their problem-solving skills effectively. In summary,serves as an innovative tool for accelerating the development and assessment of generalizable discovery skills in AI agents. Its detailed descriptions provide additional context on implementation details and operational costs while offering valuable insights into agent performance through automated metrics and human evaluations across diverse thematic areas within the realm of scientific research.