SDPO: Segment-Level Direct Preference Optimization for Social Agents

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Close Not relevant

   Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents

   Pranav Putta, Edmund Mills, Naman Garg et al. · 2024 · Artificial Intelligence · AI summary available

   Abstract

   Large Language Models (LLMs) have shown remarkable capabilities in natural language tasks requiring complex reasoning, yet their application in agentic, multi-step reasoning within interactive environments remains a difficult challenge. Traditional supervised pre-training on static datasets falls short in enabling autonomous agent capabilities needed to perform complex decision-making in dynamic settings like web navigation. Previous attempts to bridge this ga-through supervised fine-tuning on curated expert demonstrations-often suffer from compounding errors and limited exploration data, resulting in sub-optimal policy outcomes. To overcome these challenges, we propose a framework that comb…

2. Close Not relevant

   MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework

   Sirui Hong, Mingchen Zhuge, Jonathan Chen et al. · 2023 · Artificial Intelligence · AI summary available

   Abstract

   Remarkable progress has been made on automated problem solving through societies of agents based on large language models (LLMs). Existing LLM-based multi-agent systems can already solve simple dialogue tasks. Solutions to more complex tasks, however, are complicated through logic inconsistencies due to cascading hallucinations caused by naively chaining LLMs. Here we introduce MetaGPT, an innovative meta-programming framework incorporating efficient human workflows into LLM-based multi-agent collaborations. MetaGPT encodes Standardized Operating Procedures (SOPs) into prompt sequences for more streamlined workflows, thus allowing agents with human-like domain expertise to verify intermediat…

3. Related Not relevant

   InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models

   Lichang Chen, Jiuhai Chen, Tom Goldstein et al. · 2023 · Artificial Intelligence · AI summary available

   Abstract

   Large language models~(LLMs) are instruction followers, but it can be challenging to find the best instruction for different situations, especially for black-box LLMs on which backpropagation is forbidden. Instead of directly optimizing the discrete instruction, we optimize a low-dimensional soft prompt applied to an open-source LLM to generate the instruction for the black-box LLM. On each iteration of the proposed method, which we call InstructZero, a soft prompt is converted into an instruction using the open-source LLM, which is then submitted to the black-box LLM for zero-shot evaluation, and the performance is sent to Bayesian optimization to produce new soft prompts improving the zero…

4. Related Not relevant

   DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Search

   Murong Yue, Wenlin Yao, Haitao Mi et al. · 2024 · Artificial Intelligence · Not summarized yet

   Abstract

   Enhancing the capability of large language models (LLMs) in reasoning has gained significant attention in recent years. Previous studies have demonstrated the effectiveness of various prompting strategies in aiding LLMs in reasoning (called "reasoning actions"), such as step-by-step thinking, reflecting before answering, solving with programs, and their combinations. However, these approaches often applied static, predefined reasoning actions uniformly to all questions, without considering the specific characteristics of each question or the capability of the task-solving LLM. In this paper, we propose DOTS, an approach enabling LLMs to reason dynamically via optimal reasoning trajectory sea…

5. Related Not relevant

   Data Interpreter: An LLM Agent For Data Science

   Sirui Hong, Yizhang Lin, Bangbang Liu et al. · 2024 · Artificial Intelligence · AI summary available

   Abstract

   Large Language Model (LLM)-based agents have demonstrated remarkable effectiveness. However, their performance can be compromised in data science scenarios that require real-time data adjustment, expertise in optimization due to complex dependencies among various tasks, and the ability to identify logical errors for precise reasoning. In this study, we introduce the Data Interpreter, a solution designed to solve with code that emphasizes three pivotal techniques to augment problem-solving in data science: 1) dynamic planning with hierarchical graph structures for real-time data adaptability;2) tool integration dynamically to enhance code proficiency during execution, enriching the requisite …

6. Related Not relevant

   AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents

   Ke Yang, Yao Liu, Sapana Chaudhary et al. · 2024 · Artificial Intelligence · AI summary available

   Abstract

   Autonomy via agents using large language models (LLMs) for personalized, standardized tasks boosts human efficiency. Automating web tasks (like booking hotels within a budget) is increasingly sought after. Fulfilling practical needs, the web agent also serves as an important proof-of-concept example for various agent grounding scenarios, with its success promising advancements in many future applications. Prior research often handcrafts web agent strategies (e.g., prompting templates, multi-agent systems, search methods, etc.) and the corresponding in-context examples, which may not generalize well across all real-world scenarios. On the other hand, there has been limited study on the misali…

7. Related Not relevant

   TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems

   Yilun Kong, Jingqing Ruan, Yihong Chen et al. · 2023 · Artificial Intelligence · Not summarized yet

   Abstract

   Large Language Models (LLMs) have demonstrated proficiency in addressing tasks that necessitate a combination of task planning and the usage of external tools that require a blend of task planning and the utilization of external tools, such as APIs. However, real-world complex systems present three prevalent challenges concerning task planning and tool usage: (1) The real system usually has a vast array of APIs, so it is impossible to feed the descriptions of all APIs to the prompt of LLMs as the token length is limited; (2) the real system is designed for handling complex tasks, and the base LLMs can hardly plan a correct sub-task order and API-calling order for such tasks; (3) Similar sema…

8. Related Not relevant

   A General Theoretical Paradigm to Understand Learning from Human Preferences

   Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot et al. · 2023 · Artificial Intelligence · Not summarized yet

   Abstract

   The prevalent deployment of learning from human preferences through reinforcement learning (RLHF) relies on two important approximations: the first assumes that pairwise preferences can be substituted with pointwise rewards. The second assumes that a reward model trained on these pointwise rewards can generalize from collected data to out-of-distribution data sampled by the policy. Recently, Direct Preference Optimisation (DPO) has been proposed as an approach that bypasses the second approximation and learn directly a policy from collected data without the reward modelling stage. However, this method still heavily relies on the first approximation. In this paper we try to gain a deeper theo…