# AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs

**Authors:** Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, Kin Hei Lee, Zihan Wang, Ka Yiu Lee, Guchun Zhang, Kun Shao, Linyi Yang, Jun Wang  
**arXiv:** [2508.16153](https://arxiv.org/abs/2508.16153)  
**Date:** 2025-08-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2508.16153v1/

## Key points

- Introduces a novel learning paradigm for adaptive Large Language Model (LLM) agents that eliminates the need for fine-tuning
- Enables low-cost continual adaptation through memory-based online reinforcement learning
- Formulated as a Memory-augmented Markov Decision Process (M-MDP) with a neural case-selection policy
- Agent model, AgentFly, achieves top performance on various datasets representing different research challenges
- Outperforms existing methods on open-domain QA datasets
- Explores integration of external tools into language agents for multi-hop tool calls in long-horizon tasks
- Proposes Agentic Reinforcement Learning as a training paradigm to enable dynamic interactions with external tool environments
- Incorporates case-based reasoning into planning to facilitate strategic tool calls and improve performance in web research tasks

## Summary

, , , , 


In this paper, we introduce a novel learning paradigm for adaptive Large Language Model (LLM) agents that eliminates the need for fine-tuning. Our method enables low-cost continual adaptation through memory-based online reinforcement learning, formalized as a Memory-augmented Markov Decision Process (M-MDP) with a neural case-selection policy. Past experiences are stored in an episodic memory, and the policy is continually updated based on environmental feedback. Our agent model, AgentFly, achieves top performance on various datasets representing different research challenges. We evaluate our approach on four datasets: GAIA for long-horizon planning, DeepResearcher for real-time web-based research, SimpleQA for factual accuracy, and HLE for exploration at the frontier of human knowledge. Performance comparisons show that AgentFly outperforms existing methods on open-domain QA datasets. We also explore the integration of external tools into language agents in the context of multi-hop tool calls for long-horizon tasks. We propose Agentic Reinforcement Learning as a training paradigm to enable dynamic interactions with external tool environments. By incorporating case-based reasoning into planning, strategic tool calls are facilitated leading to consistently strong performance. Experimental results on the Deep Researcher dataset demonstrate that AgentFly augmented with MCP tools achieves significant improvements in F1 scores compared to baseline methods like CoT + RAG. This highlights the effectiveness of real-time online retrieval tools in enhancing agent performance in web research tasks. Overall, our study presents a scalable and efficient approach for developing generalist LLM agents capable of continuous learning without gradient updates. Our findings contribute to advancing machine learning towards open-ended skill acquisition and deep research scenarios.