# Evaluating Language-Model Agents on Realistic Autonomous Tasks

**Authors:** Megan Kinniment, Lucas Jun Koba Sato, Haoxing Du, Brian Goodrich, Max Hasin, Lawrence Chan, Luke Harold Miles, Tao R. Lin, Hjalmar Wijk, Joel Burget, Aaron Ho, Elizabeth Barnes, Paul Christiano  
**arXiv:** [2312.11671](https://arxiv.org/abs/2312.11671)  
**Date:** 2023-12-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.11671v1/

## Key points

- Language model agents investigated for acquiring resources, self-replicating, and adapting to new challenges
- ARA (autonomous replication and adaptation) capabilities could have unpredictable consequences
- Importance of measuring and forecasting ARA for security, monitoring, and alignment purposes
- Placing limits on ARA becomes challenging once achieved by a system
- Four example agents constructed combining language models with real-world action tools
- Agents tested on 12 tasks relevant to ARA, struggling with more challenging ones
- Evaluations alone cannot rule out future agents possessing ARA capabilities
- Pretraining evaluations needed to provide assurance against future iterations with ARA abilities
- Fine-tuning existing models without targeting ARA could lead to more competent agents
- Further research and evaluation necessary to understand and mitigate risks associated with autonomous replication and adaptation in language model agents.

## Summary

In this report, the authors investigate the capabilities of language model agents in acquiring resources, self-replicating, and adapting to new challenges. They refer to this set of abilities as "autonomous replication and adaptation" (ARA). The authors believe that systems with ARA capabilities could have far-reaching and unpredictable consequences. Therefore, they emphasize the importance of measuring and forecasting ARA for security, monitoring, and alignment purposes. Additionally, once a system achieves ARA, it becomes significantly more challenging to place limits on its capabilities. To evaluate the potential for ARA in language model agents, the authors construct four example agents that combine language models with tools enabling them to take actions in the real world. These agents are then tested on 12 tasks relevant to ARA. The results show that while these language model agents can complete some of the easier tasks, they struggle with more challenging ones. However, these evaluations alone cannot rule out the possibility that near-future agents will possess ARA capabilities. The authors highlight that without intermediate evaluations during pretraining, it is difficult to provide assurance that future iterations of language models (such as those with 100x effective compute scaleup) will not exhibit ARA abilities. Furthermore, they suggest that even fine-tuning existing models without directly targeting ARA could lead to significantly more competent agents. Overall, this report underscores the need for further research and evaluation to understand and mitigate potential risks associated with autonomous replication and adaptation in language model agents.