Stream of Search (SoS): Learning to Search in Language

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Proche Pas pertinent

   Automating Thought of Search: A Journey Towards Soundness and Completeness (Student Abstract)

   Daniel Cao, Michael Katz, Harsha Kokel et al. · 2025 · AAAI Conference on Artificial Intelligence · cité 3 fois · Pas encore résumé

   Résumé scientifique

   Large language models (LLMs) now turn their attention to search. Recently, Thought of Search (ToS) proposed defining the search space with code, having an LLM produce that code. ToS requires a human in the loop, collaboratively producing a sound successor function and goal test, achieving impressive 100% accuracy on all the tested datasets. In this work, we automate ToS (AutoToS), completely taking the human out of the loop of solving planning problems. AutoToS guides the language model step by step towards the generation of sound and complete search components, through feedback from both generic and domain specific unit tests. We achieve 100% accuracy, with minimal feedback iterations, usin…

2. Proche Pas pertinent

   Automating Thought of Search: A Journey Towards Soundness and Completeness

   Daniel Cao, Michael Katz, Harsha Kokel et al. · 2024 · Artificial Intelligence · cité 8 fois · Résumé IA disponible

   Résumé scientifique

   Planning remains one of the last standing bastions for large language models (LLMs), which now turn their attention to search. Most of the literature uses the language models as world models to define the search space, forgoing soundness for the sake of flexibility. A recent work, Thought of Search (ToS), proposed defining the search space with code, having the language models produce that code. ToS requires a human in the loop, collaboratively producing a sound successor function and goal test. The result, however, is worth the effort: all the tested datasets were solved with 100% accuracy. At the same time LLMs have demonstrated significant progress in code generation and refinement for co…

3. Proche Cite cet article Utilise cet article en profondeur Pas pertinent

   Learning to Better Search with Language Models via Guided Reinforced Self-Training

   Seungyong Moon, Bumsoo Park, H. Song · 2024 · Neural Information Processing Systems · cité 3 fois · Pas encore résumé

   Résumé scientifique

   While language models have shown remarkable performance across diverse tasks, they still encounter challenges in complex reasoning scenarios. Recent research suggests that language models trained on linearized search traces toward solutions, rather than solely on the final solutions, exhibit improved generalization, despite the search traces being potentially noisy or suboptimal. However, relying on such imperfect traces can result in inefficient use of test-time compute. To address this, we propose guided reinforced self-training (Guided-ReST), a fine-tuning algorithm designed to improve the model's capability for effective search during inference. The key insight behind Guided-ReST is that…

4. Proche Pas pertinent

   SEM: Reinforcement Learning for Search-Efficient Large Language Models

   Zeyang Sha, Shiwen Cui, Weiqiang Wang · 2025 · Computation and Language · preprint, non relu par les pairs · cité 10 fois · Pas encore résumé

   Résumé scientifique

   Recent advancements in Large Language Models(LLMs) have demonstrated their capabilities not only in reasoning but also in invoking external tools, particularly search engines. However, teaching models to discern when to invoke search and when to rely on their internal knowledge remains a significant challenge. Existing reinforcement learning approaches often lead to redundant search behaviors, resulting in inefficiencies and over-cost. In this paper, we propose SEM, a novel post-training reinforcement learning framework that explicitly trains LLMs to optimize search usage. By constructing a balanced dataset combining MuSiQue and MMLU, we create scenarios where the model must learn to disting…

5. Proche Pas pertinent

   Large Search Model: Redefining Search Stack in the Era of LLMs

   Liang Wang, Nan Yang, Xiaolong Huang et al. · 2023 · Information Retrieval · cité 32 fois · Résumé IA disponible

   Résumé scientifique

   Modern search engines are built on a stack of different components, including query understanding, retrieval, multi-stage ranking, and question answering, among others. These components are often optimized and deployed independently. In this paper, we introduce a novel conceptual framework called large search model, which redefines the conventional search stack by unifying search tasks with one large language model (LLM). All tasks are formulated as autoregressive text generation problems, allowing for the customization of tasks through the use of natural language prompts. This proposed framework capitalizes on the strong language understanding and reasoning capabilities of LLMs, offering th…

6. Proche Cite cet article Pas pertinent

   Language Models can Self-Improve at State-Value Estimation for Better Search

   Ethan Mendes, Alan Ritter · 2025 · Neural Information Processing Systems · cité 5 fois · Pas encore résumé

   Résumé scientifique

   Collecting ground-truth rewards or human demonstrations for multi-step reasoning tasks is often prohibitively expensive, particularly in interactive domains such as web tasks. We introduce Self-Taught Lookahead (STL), a reward-free framework that improves language model-based value functions by reasoning explicitly about state transitions. STL can be viewed as a chain-of-thought analogue of the value iteration algorithm: instead of regressing directly on numeric values, a value LLM is trained to simulate a step of lookahead in natural language - predicting the next action, resulting state, and rationale for its value, thereby refining value estimates without any labeled data. This self-super…

7. Proche Pas pertinent

   Tree Search for Language Model Agents

   Jing Yu Koh, Stephen McAleer, Daniel Fried et al. · 2024 · Artificial Intelligence · cité 166 fois · Résumé IA disponible

   Résumé scientifique

   Autonomous agents powered by language models (LMs) have demonstrated promise in their ability to perform decision-making tasks such as web automation. However, a key limitation remains: LMs, primarily optimized for natural language understanding and generation, struggle with multi-step reasoning, planning, and using environmental feedback when attempting to solve realistic computer tasks. Towards addressing this, we propose an inference-time search algorithm for LM agents to explicitly perform exploration and multi-step planning in interactive web environments. Our approach is a form of best-first tree search that operates within the actual environment space, and is complementary with most e…

8. Proche Pas pertinent

   A Survey of Deep Code Search

   Yutao Xie, Jiayi Lin, Hande Dong et al. · 2023 · Software Engineering · cité 37 fois · Résumé IA disponible

   Résumé scientifique

   Code writing is repetitive and predictable, inspiring us to develop various code intelligence techniques. This survey focuses on code search, that is, to retrieve code that matches a given query by effectively capturing the semantic similarity between the query and code. Deep learning, being able to extract complex semantics information, has achieved great success in this field. Recently, various deep learning methods, such as graph neural networks and pretraining models, have been applied to code search with significant progress. Deep learning is now the leading paradigm for code search. In this survey, we provide a comprehensive overview of deep learning-based code search. We review the ex…

Les travaux antérieurs sur lesquels il s’appuie

1. Apparenté Une méthode qu’il utilise Pas pertinent

   Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models

   Bilgehan Sel, Ahmad Al-Tawaha, Vanshaj Khattar et al. · 2023 · Computation and Language · cité 122 fois · Résumé IA disponible

   Résumé scientifique

   Current literature, aiming to surpass the "Chain-of-Thought" approach, often resorts to an external modus operandi involving halting, modifying, and then resuming the generation process to boost Large Language Models' (LLMs) reasoning capacities. This mode escalates the number of query requests, leading to increased costs, memory, and computational overheads. Addressing this, we propose the Algorithm of Thoughts -- a novel strategy that propels LLMs through algorithmic reasoning pathways, pioneering a new mode of in-context learning. By employing algorithmic examples, we exploit the innate recurrence dynamics of LLMs, expanding their idea exploration with merely one or a few queries. Our tec…

2. Apparenté Pas pertinent

   Strategic Reasoning with Language Models

   Kanishk Gandhi, Dorsa Sadigh, Noah D. Goodman · 2023 · arXiv.org · preprint, non relu par les pairs · cité 74 fois · Pas encore résumé

   Résumé scientifique

   Strategic reasoning enables agents to cooperate, communicate, and compete with other agents in diverse situations. Existing approaches to solving strategic games rely on extensive training, yielding strategies that do not generalize to new scenarios or games without retraining. Large Language Models (LLMs), with their ability to comprehend and generate complex, context-rich language, could prove powerful as tools for strategic gameplay. This paper introduces an approach that uses pretrained LLMs with few-shot chain-of-thought examples to enable strategic reasoning for AI agents. Our approach uses systematically generated demonstrations of reasoning about states, values, and beliefs to prompt…

3. Apparenté Pas pertinent

   Search-based structured prediction

   Hal Daumé, John Langford, D. Marcu · 2009 · Machine-mediated learning · cité 604 fois · Pas encore résumé

   Résumé scientifique

   We present Searn, an algorithm for integrating search and learning to solve complex structured prediction problems such as those that occur in natural language, speech, computational biology, and vision. Searn is a meta-algorithm that transforms these complex problems into simple classification problems to which any binary classifier may be applied. Unlike current algorithms for structured learning that require decomposition of both the loss function and the feature functions over the predicted structure, Searn is able to learn prediction functions for any loss function and any class of features. Moreover, Searn comes with a strong, natural theoretical guarantee: good performance on the deri…

4. Apparenté Une méthode qu’il utilise Pas pertinent

   Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping

   Lucas Lehnert, Sainbayar Sukhbaatar, Di-Jia Su et al. · 2024 · arXiv.org · preprint, non relu par les pairs · cité 114 fois · Pas encore résumé

   Résumé scientifique

   While Transformers have enabled tremendous progress in various application settings, such architectures still trail behind traditional symbolic planners for solving complex decision making tasks. In this work, we demonstrate how to train Transformers to solve complex planning tasks. This is accomplished by training an encoder-decoder Transformer model to predict the search dynamics of the $A^*$ search algorithm. We fine tune this model to obtain a Searchformer, a Transformer model that optimally solves previously unseen Sokoban puzzles 93.7% of the time, while using up to 26.8% fewer search steps than the $A^*$ implementation that was used for training initially. In our training method, $A^*…

5. Apparenté Pas pertinent

   Sequence Level Training with Recurrent Neural Networks

   M. Ranzato, S. Chopra, M. Auli et al. · 2015 · International Conference on Learning Representations · cité 1 891 fois · Pas encore résumé

   Résumé scientifique

   Many natural language processing applications use language models to generate text. These models are typically trained to predict the next word in a sequence, given the previous words and some context such as an image. However, at test time the model is expected to generate the entire sequence from scratch. This discrepancy makes generation brittle, as errors may accumulate along the way. We address this issue by proposing a novel sequence level training algorithm that directly optimizes the metric used at test time, such as BLEU or ROUGE. On three different tasks, our approach outperforms several strong baselines for greedy generation. The method is also competitive when these baselines emp…

Les articles qui le citent

1. Apparenté Utilise cet article en profondeur Pas pertinent

   Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search

   Maohao Shen, Guang-Tao Zeng, Zhenting Qi et al. · 2025 · International Conference on Machine Learning · cité 60 fois · Pas encore résumé

   Résumé scientifique

   Large language models (LLMs) have demonstrated remarkable reasoning capabilities across diverse domains. Recent studies have shown that increasing test-time computation enhances LLMs' reasoning capabilities. This typically involves extensive sampling at inference time guided by an external LLM verifier, resulting in a two-player system. Despite external guidance, the effectiveness of this system demonstrates the potential of a single LLM to tackle complex tasks. Thus, we pose a new research problem: Can we internalize the searching capabilities to fundamentally enhance the reasoning abilities of a single LLM? This work explores an orthogonal direction focusing on post-training LLMs for autor…

2. Apparenté Utilise cet article en profondeur Pas pertinent

   To Backtrack or Not to Backtrack: When Sequential Search Limits Model Reasoning

   Tian Qin, David Alvarez-Melis, Samy Jelassi et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 29 fois · Pas encore résumé

   Résumé scientifique

   Recent advancements in large language models (LLMs) have significantly improved their reasoning abilities, particularly through techniques involving search and backtracking. Backtracking naturally scales test-time compute by enabling sequential, linearized exploration via long chain-of-thought (CoT) generation. However, this is not the only strategy for scaling test time-compute: parallel sampling with best-of-N selection provides an alternative that generates diverse solutions simultaneously. Despite the growing adoption of sequential search, its advantages over parallel sampling-especially under a fixed compute budget-remain poorly understood. In this paper, we systematically compare these…

3. Apparenté Utilise cet article en profondeur Pas pertinent

   A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?

   Qiyuan Zhang, Fuyuan Lyu, Zexu Sun et al. · 2025 · cité 211 fois · Pas encore résumé

   Résumé scientifique

   As enthusiasm for scaling computation (data and parameters) in the pretraining era gradually diminished, test-time scaling (TTS), also referred to as ``test-time computing'' has emerged as a prominent research focus. Recent studies demonstrate that TTS can further elicit the problem-solving capabilities of large language models (LLMs), enabling significant breakthroughs not only in specialized reasoning tasks, such as mathematics and coding, but also in general tasks like open-ended Q&A. However, despite the explosion of recent efforts in this area, there remains an urgent need for a comprehensive survey offering a systemic understanding. To fill this gap, we propose a unified, multidimensio…

4. Apparenté Utilise cet article en profondeur Pas pertinent

   Step Back to Leap Forward: Self-Backtracking for Symbolic Reasoning and Planning in Language Models

   Xiao-Wen Yang, Xuan Zhu, Ding-Chu Zhang et al. · 2026 · AAAI Conference on Artificial Intelligence · cité 1 fois · Pas encore résumé

   Résumé scientifique

   Although autoregressive language models demonstrated remarkable performance across various tasks, their effectiveness in symbolic reasoning and decision-making scenarios remains constrained. Recent research indicates that training language models to emulate symbolic search algorithms (e.g. depth-first search or A* algorithm) can yield strong improvements in their symbolic reasoning and planning capabilities. However, existing methods only achieve superficial imitation of symbolic search trajectories, as their generation processes lack explicit backtracking mechanisms. This limitation prevents models from truly mastering symbolic search, often resulting in rigid and redundant outputs with poo…

5. Apparenté Utilise cet article en profondeur Pas pertinent

   Self-Play Search Distillation for Large Language Model Reasoning

   Lorenzo Molfetta, Wai-Chung Kwan, Giacomo Frisoni et al. · 2026 · Pas encore résumé

   Résumé scientifique

   Improving reasoning abilities in Large Language Models (LLMs) requires high-quality data that exposes difficult decisions, competing alternatives, and their consequences. Data scarcity is driven by the low quality of synthetic data and the cost of human labeling. We introduce Self-Play Search Distillation (SPSD), a framework for generating superhuman synthetic data via self-play of MuZero-like networks trained on board games. SPSD uses executable environments to turn search into structured reasoning problems. At each state, the expert identifies a preferred decision, plausible alternatives, plausible opponent replies, and value estimates. By converting the self-play search records into super…