Sakana Fugu Technical Report

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Close Not relevant

   Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents

   Yashar Talebirad, Amirhossein Nadiri · 2023 · Artificial Intelligence · preprint, not peer reviewed · cited 509 times · AI summary available

   Abstract

   In this paper, we present a novel framework for enhancing the capabilities of large language models (LLMs) by leveraging the power of multi-agent systems. Our framework introduces a collaborative environment where multiple intelligent agent components, each with distinctive attributes and roles, work together to handle complex tasks more efficiently and effectively. We demonstrate the practicality and versatility of our framework through case studies in artificial general intelligence (AGI), specifically focusing on the Auto-GPT and BabyAGI models. We also examine the "Gorilla" model, which integrates external APIs into the LLM. Our framework addresses limitations and challenges such as loop…

2. Close Not relevant

   AgentSquare: Automatic LLM Agent Search in Modular Design Space

   Yu Shang, Yu Li, Keyu Zhao et al. · 2024 · Computation and Language · cited 188 times · Not summarized yet

   Abstract

   Recent advancements in Large Language Models (LLMs) have led to a rapid growth of agentic systems capable of handling a wide range of complex tasks. However, current research largely relies on manual, task-specific design, limiting their adaptability to novel tasks. In this paper, we introduce a new research problem: Modularized LLM Agent Search (MoLAS). We propose a modular design space that abstracts existing LLM agent designs into four fundamental modules with uniform IO interface: Planning, Reasoning, Tool Use, and Memory. Building on this design space, we present a novel LLM agent search framework called AgentSquare, which introduces two core mechanisms, i.e., module evolution and recom…

3. Close Not relevant

   Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Models

   Chen Yang, Guangyue Peng, Jiaying Zhu et al. · 2025 · Computation and Language · preprint, not peer reviewed · cited 6 times · AI summary available

   Abstract

   We present Nanbeige4-3B, a family of small-scale but high-performing language models. Pretrained on 23T high-quality tokens and finetuned on over 30 million diverse instructions, we extend the boundary of the scaling law for small language models. In pre-training, we design a Fine-Grained Warmup-Stable-Decay (FG-WSD) training scheduler, which progressively refines data mixtures across stages to boost model performance. In post-training, to improve the quality of the SFT data, we design a joint mechanism that integrates deliberative generation refinement and chain-of-thought reconstruction, yielding substantial gains on complex tasks. Following SFT, we employ our flagship reasoning model to d…

4. Close Not relevant

   AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs

   Huichi Zhou, Yihang Chen, Siyuan Guo et al. · 2025 · Machine Learning · preprint, not peer reviewed · cited 119 times · AI summary available

   Abstract

   In this paper, we introduce a novel learning paradigm for adaptive Large Language Model (LLM) agents that eliminates the need for fine-tuning the underlying LLMs. Existing approaches are often either rigid, relying on static, handcrafted reflection workflows, or computationally intensive, requiring gradient updates of LLM model parameters. In contrast, our method enables low-cost continual adaptation via memory-based online reinforcement learning. We formalise this as a Memory-augmented Markov Decision Process (M-MDP), equipped with a neural case-selection policy to guide action decisions. Past experiences are stored in an episodic memory, either differentiable or non-parametric. The policy …

5. Close Not relevant

   Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents

   Pranav Putta, Edmund Mills, Naman Garg et al. · 2024 · Artificial Intelligence · preprint, not peer reviewed · cited 237 times · AI summary available

   Abstract

   Large Language Models (LLMs) have shown remarkable capabilities in natural language tasks requiring complex reasoning, yet their application in agentic, multi-step reasoning within interactive environments remains a difficult challenge. Traditional supervised pre-training on static datasets falls short in enabling autonomous agent capabilities needed to perform complex decision-making in dynamic settings like web navigation. Previous attempts to bridge this ga-through supervised fine-tuning on curated expert demonstrations-often suffer from compounding errors and limited exploration data, resulting in sub-optimal policy outcomes. To overcome these challenges, we propose a framework that comb…

6. Close Not relevant

   Long-form factuality in large language models

   Jerry Wei, Chengrun Yang, Xinying Song et al. · 2024 · Computation and Language · cited 190 times · AI summary available

   Abstract

   Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics. To benchmark a model's long-form factuality in open domains, we first use GPT-4 to generate LongFact, a prompt set comprising thousands of questions spanning 38 topics. We then propose that LLM agents can be used as automated evaluators for long-form factuality through a method which we call Search-Augmented Factuality Evaluator (SAFE). SAFE utilizes an LLM to break down a long-form response into a set of individual facts and to evaluate the accuracy of each fact using a multi-step reasoning process comprising sending search queries to Google Search a…

7. Close Not relevant

   From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence

   Jian Yang, Wei Zhang, Shark Liu et al. · 2025 · Software Engineering · preprint, not peer reviewed · cited 19 times · AI summary available

   Abstract

   Large language models (LLMs) have fundamentally transformed automated software development by enabling direct translation of natural language descriptions into functional code, driving commercial adoption through tools like Github Copilot (Microsoft), Cursor (Anysphere), Trae (ByteDance), and Claude Code (Anthropic). While the field has evolved dramatically from rule-based systems to Transformer-based architectures, achieving performance improvements from single-digit to over 95\% success rates on benchmarks like HumanEval. In this work, we provide a comprehensive synthesis and practical guide (a series of analytic and probing experiments) about code LLMs, systematically examining the comple…

8. Close Not relevant

   The Rise and Potential of Large Language Model Based Agents: A Survey

   Zhiheng Xi, Wenxiang Chen, Xin Guo et al. · 2023 · Artificial Intelligence · preprint, not peer reviewed · cited 2,186 times · AI summary available

   Abstract

   For a long time, humanity has pursued artificial intelligence (AI) equivalent to or surpassing the human level, with AI agents considered a promising vehicle for this pursuit. AI agents are artificial entities that sense their environment, make decisions, and take actions. Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks. Actually, what the community lacks is a general and powerful model to serve as a starting point for designing AI agents that can adapt to diverse scenarios. Due to the versatile capabilities they demonstrate, large language …

Prior work it builds on

1. Related Used in depth by this paper Not relevant

   Learning to Orchestrate Agents in Natural Language with the Conductor

   Stefan Nielsen, Edoardo Cetin, Peter S Schwendeman et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 31 times · Not summarized yet

   Abstract

   Powerful large language models (LLMs) from different providers have been expensively trained and finetuned to specialize across varying domains. In this work, we introduce a new kind of Conductor model trained with reinforcement learning to automatically discover powerful coordination strategies among LLMs. Our Conductor learns not only to design targeted communication topologies for effective agent-to-agent collaboration, but also to prompt engineer focused instructions to the LLMs to maximally leverage their individual capabilities. We show that, by learning optimal coordination strategies over pools of powerful worker LLMs, a 7B Conductor achieves significant performance gains beyond any …

2. Close Not relevant

   Improving Factuality and Reasoning in Language Models through Multiagent Debate

   Yilun Du, Shuang Li, A. Torralba et al. · 2023 · International Conference on Machine Learning · cited 2,569 times · Not summarized yet

   Abstract

   Large language models (LLMs) have demonstrated remarkable capabilities in language generation, understanding, and few-shot learning in recent years. An extensive body of work has explored how their performance may be further improved through the tools of prompting, ranging from verification, self-consistency, or intermediate scratchpads. In this paper, we present a complementary approach to improve language responses where multiple language model instances propose and debate their individual responses and reasoning processes over multiple rounds to arrive at a common final answer. Our findings indicate that this approach significantly enhances mathematical and strategic reasoning across a nu…

3. Related Not relevant

   ReAct: Synergizing Reasoning and Acting in Language Models

   Shunyu Yao, Jeffrey Zhao, Dian Yu et al. · 2022 · International Conference on Learning Representations · cited 13,293 times · Not summarized yet

   Abstract

   While large language models (LLMs) have demonstrated impressive capabilities across tasks in language understanding and interactive decision making, their abilities for reasoning (e.g. chain-of-thought prompting) and acting (e.g. action plan generation) have primarily been studied as separate topics. In this paper, we explore the use of LLMs to generate both reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two: reasoning traces help the model induce, track, and update action plans as well as handle exceptions, while actions allow it to interface with external sources, such as knowledge bases or environments, to gather additional in…

4. Related Used in depth by this paper Not relevant

   LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

   Naman Jain, King Han, Alex Gu et al. · 2024 · International Conference on Learning Representations · cited 2,545 times · Not summarized yet

   Abstract

   Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry. However, as new and improved LLMs are developed, existing evaluation benchmarks (e.g., HumanEval, MBPP) are no longer sufficient for assessing their capabilities. In this work, we propose LiveCodeBench, a comprehensive and contamination-free evaluation of LLMs for code, which continuously collects new problems over time from contests across three competition platforms, namely LeetCode, AtCoder, and CodeForces. Notably, our benchmark also focuses on a broader range of code related capabilities, such as self-repair, code execution…

5. Related Not relevant

   TRINITY: An Evolved LLM Coordinator

   Jinglue Xu, Qi Sun, Peter S Schwendeman et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 6 times · Not summarized yet

   Abstract

   Combining diverse foundation models is promising, but weight-merging is limited by mismatched architectures and closed APIs. Trinity addresses this with a lightweight coordinator that orchestrates collaboration among large language models (LLMs). The coordinator, comprising a compact language model (approximately $0.6$B parameters) and a lightweight head (approximately $10$K parameters), is optimized with an evolutionary strategy for efficient and adaptive delegation. Trinity processes queries over multiple turns, where at each turn the coordinator assigns one of three roles (Thinker, Worker, or Verifier) to a selected LLM, effectively offloading complex skill acquisition from the coordinato…

Papers that cite it

1. Related Uses this paper in depth Not relevant

   Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics

   Pavel Snopov, German Magai · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   Abstract

   Recent advances in AI for Mathematics have focused largely on autoformalization and theorem proving, leaving the role of Computer Algebra Systems (CAS) in agentic LLM workflows underexplored. We propose a ReAct-style agentic setup that combines LLM reasoning with verifiable feedback from SageMath, together with Context7 for the up-to-date documentation. We evaluate this agentic setup across frontier models for solving research-level mathematical problems from the RealMath benchmark in a setting that emulates a computational-mathematics research loop. We also propose a refinement to the RealMath benchmark by introducing a multi-step post-processing procedure and a multi-stage validation pipel…

2. Related Not relevant

   Ensembling LLMs for AI-Augmented Cybersecurity Software Requirements Generation

   Santiago Perez-Acuna, Y. Martín, J. Yelmo · 2026 · Not summarized yet

   Abstract

   Translating high-level controls from security standards into concrete, system-specific requirements is central to cybersecurity requirements engineering. Large language models (LLMs) can accelerate this labor-intensive, recall-sensitive task, but any single run is unreliable: it misses valid safeguards while introducing plausible hallucinations, and outputs shift across runs and models. We reframe this variability as a resource: rather than selecting one output, we study post-generation ensembling, aggregating stochastic runs with information-retrieval data-fusion operators. We propose two strategies: Uniform fusion rewards mere cross-run agreement, whereas Naive-Bayes fusion weights each ru…

3. Related Not relevant

   Recursive Self-Improvement through Multi-Agent Self-Supervision

   Hyunin Lee, Jinglue Xu, Jeffrey Seely et al. · 2026 · cited 5 times · Not summarized yet

   Abstract

   Recursive self-improvement (RSI) of a model on non-verifiable tasks, such as open-ended research, faces a supervision bottleneck when its outputs exceed what even human experts can reliably assess, leaving the model itself (optimizee) as the best available optimizer and evaluator. However, a single model instance struggles to critique and improve its own complex reasoning under this homogeneous loop. To address this, we propose Multi-Agent Self-Supervision (MASS), an RSI method that alternates between evolutionary workflow optimization and supervised fine-tuning on self-generated trajectories. Guided by early findings that multi-agent topologies excel at complex reasoning, MASS prompts a sin…

4. Related Not relevant

   Multi-Agent Debate and Visual Information Extraction for SeePhys Pro: A 1st-Place Technical Report from ICML 2026 AI4Math Track 3 Challenge

   Jiseok Kwak, Suhyeon Jo, Taewoo Kim et al. · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   Abstract

   This technical report presents our approach to Challenge Track~3: SeePhys Pro at the 3rd AI for Math Workshop, where the task is to answer college-level physics questions whose statement and figure may be given partly or entirely as an image. Visual physics problems become substantially harder for large language models when the decisive information resides in a figure rather than in the text, and this modality gap widens as more of the problem migrates into the image. We address the task with a two-stage framework: a visual information extraction stage that re-expresses figure content as solver-readable text to close the modality gap, and a reasoning stage that orchestrates three heterogeneo…