Résumé IA · arXiv: 2309.11998v4 · cs.CL
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Posez des questions sur cet article à notre assistant IA
Comment discuter avec cet article ? Guide illustré
Vous pouvez aussi discutez avec plusieurs papiers à la fois ici.
Gardez cette conversation — un compte gratuit conserve vos questions et réponses, vos collections d’articles et vos alertes.
Continuer avec Google Créer un compte gratuitOutils pour cet article
Outils avancés Premium
Pour suivre un domaine de près : alertes de citations, diapositives de journal club, comparaison des versions et envoi vers Zotero. Découvrir Premium
Les articles autour de celui-ci
Les voir sur une carteClassés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
ShareChat: A Dataset of Chatbot Conversations in the WildYueru Yan, Tuc Nguyen, Bo Su et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 17 fois · Pas encore résumé
Résumé scientifique
By evaluating Large Language Models (LLMs) through uniform, text-only interfaces, current academic benchmarks obscure how the unique designs and affordances of distinct commercial platforms shape real-world user behavior and system performance. To bridge this gap, we present ShareChat, the first large-scale corpus of 142,808 conversations (660,293 turns) collected from publicly shared URLs on ChatGPT, Perplexity, Grok, Gemini, and Claude. ShareChat preserves native platform affordances, including citations, thinking traces, and code artifacts, across 95 languages and the period from April 2023 to October 2025, complementing existing corpora that homogenize these interactions. To demonstrate …
-
BotChat: Evaluating LLMs' Capabilities of Having Multi-Turn DialoguesHaodong Duan, Jueqi Wei, Chonghua Wang et al. · 2023 · Computation and Language · preprint, non relu par les pairs · cité 48 fois · Résumé IA disponible
Résumé scientifique
Interacting with human via high-quality multi-turn dialogues is a key feature of large language models (LLMs). However, human-based evaluation of such capability involves intensive manual labor. This report provides a preliminary evaluation of existing large language models for human-style multi-turn chatting, through an LLM-based approach. We start from real-world human dialogues and keep the very first utterances as the ChatSEED. Then we prompt LLMs to generate a full multi-turn dialogue (tens of utterances) based on the ChatSEED, utterance by utterance. Finally, we adopt state-of-the-art LLMs (GPT-4, \etc) as the judge to evaluate the generated dialogues. With different evaluation protoco…
-
Datasets for Large Language Models: A Comprehensive SurveyYang Liu, Jiahuan Cao, Chongyu Liu et al. · 2024 · Computation and Language · cité 119 fois · Résumé IA disponible
Résumé scientifique
This paper embarks on an exploration into the Large Language Model (LLM) datasets, which play a crucial role in the remarkable advancements of LLMs. The datasets serve as the foundational infrastructure analogous to a root system that sustains and nurtures the development of LLMs. Consequently, examination of these datasets emerges as a critical topic in research. In order to address the current lack of a comprehensive overview and thorough analysis of LLM datasets, and to gain insights into their current status and future trends, this survey consolidates and categorizes the fundamental aspects of LLM datasets from five perspectives: (1) Pre-training Corpora; (2) Instruction Fine-tuning Data…
-
Decoding Human-LLM Collaboration in Coding: An Empirical Study of Multi-Turn Conversations in the WildBinquan Zhang, Li Zhang, Hao-Yuan Zhang et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 4 fois · Pas encore résumé
Résumé scientifique
Large language models (LLMs) are increasingly acting as dynamic conversational interfaces, supporting multi-turn interactions that mimic human-like conversation and facilitate complex tasks like coding. While datasets such as LMSYS-Chat-1M and WildChat capture real-world user-LLM conversations, few studies systematically explore the mechanisms of human-LLM collaboration in coding scenarios. What tortuous paths do users experience during the interaction process? How well do the LLMs follow instructions? Are users satisfied? In this paper, we conduct an empirical analysis on human-LLM coding collaboration using LMSYS-Chat-1M and WildChat datasets to explore the human-LLM collaboration mechanis…
-
Building Emotional Support Chatbots in the Era of LLMsZhong-Hua Zheng, Li-Zi Liao, Yang Deng et al. · 2023 · arXiv.org · preprint, non relu par les pairs · cité 97 fois · Pas encore résumé
Résumé scientifique
The integration of emotional support into various conversational scenarios presents profound societal benefits, such as social interactions, mental health counseling, and customer service. However, there are unsolved challenges that hinder real-world applications in this field, including limited data availability and the absence of well-accepted model training paradigms. This work endeavors to navigate these challenges by harnessing the capabilities of Large Language Models (LLMs). We introduce an innovative methodology that synthesizes human insights with the computational prowess of LLMs to curate an extensive emotional support dialogue dataset. Our approach is initiated with a meticulousl…
-
LLMOps in Action: A Framework for Designing, Deploying, and Governing Advanced ChatbotsPradheepan Raghavan, D. Tamburri, Stefano Fossati et al. · 2026 · IEEE Transactions on Knowledge and Data Engineering · Pas encore résumé
Résumé scientifique
This paper presents a systematic literature review (SLR) focused on the implementation of chatbots using Large Language Models (LLMs), aimed at providing insights into the architectures, frameworks, best practices, and evaluation metrics that are shaping the field. By analyzing 39 primary studies, the review addresses six key research questions, exploring common architectures such as client-server and Retrieval-Augmented Generation (RAG), and identifying frequently utilized models, including the GPT family, BERT, and open-source models like LLaMA. The paper evaluates the performance of these models across various domains, emphasizing the impact of fine-tuning, prompt engineering, and embeddi…
-
Linguistic anthropomorphism in conversational AI: theoretical foundations and empirical evidence from LMSYS chat 1MJ. L. Blanco-Guzmán, P. Palos-Sánchez, Dan-Andrei Sitar-Tǎut et al. · 2026 · Online information review (Print) · Pas encore résumé
Résumé scientifique
This study investigates linguistic anthropomorphism in large language models (LLMs) by analyzing over one million conversational turns generated by 25 state-of-the-art models, including GPT-4, Claude, Vicuna, Llama-2, and others. The research examines anthropomorphic language markers across eight major languages and explores variation in anthropomorphic language use by model and language. Using the Large Model Systems (LMSYS) Chat 1M dataset from Hugging Face, the study examines anthropomorphic language markers such as first-person pronouns, explicit empathy statements, and emojis across eight major languages. The methodology combines the detection of predefined anthropomorphic and empatheti…
-
llm-japanese-dataset v0: Construction of Japanese Chat Dataset for Large Language Models and its MethodologyMasanori Hirano, Masahiro Suzuki, Hiroki Sakaji · 2023 · Computation and Language · cité 9 fois · Pas encore résumé
Résumé scientifique
This study constructed a Japanese chat dataset for tuning large language models (LLMs), which consist of about 8.4 million records. Recently, LLMs have been developed and gaining popularity. However, high-performing LLMs are usually mainly for English. There are two ways to support languages other than English by those LLMs: constructing LLMs from scratch or tuning existing models. However, in both ways, datasets are necessary parts. In this study, we focused on supporting Japanese in those LLMs and making a dataset for training or tuning LLMs in Japanese. The dataset we constructed consisted of various tasks, such as translation and knowledge tasks. In our experiment, we tuned an existing L…
Les travaux antérieurs sur lesquels il s’appuie
-
The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue SystemsRyan Lowe, Nissan Pow, I. Serban et al. · 2015 · SIGDIAL Conference · cité 1 128 fois · Pas encore résumé
Résumé scientifique
This paper introduces the Ubuntu Dialogue Corpus, a dataset containing almost 1 million multi-turn dialogues, with a total of over 7 million utterances and 100 million words. This provides a unique resource for research into building dialogue managers based on neural language models that can make use of large amounts of unlabeled data. The dataset has both the multi-turn property of conversations in the Dialog State Tracking Challenge datasets, and the unstructured nature of interactions from microblog services such as Twitter. We also describe two neural learning architectures suitable for analyzing this dataset, and provide benchmark performance on the task of selecting the best next respo…
-
MultiWOZ - A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue ModellingPaweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng et al. · 2018 · Conference on Empirical Methods in Natural Language Processing · cité 1 666 fois · Pas encore résumé
Résumé scientifique
Even though machine learning has become the major scene in dialogue research community, the real breakthrough has been blocked by the scale of data available.To address this fundamental obstacle, we introduce the Multi-Domain Wizard-of-Oz dataset (MultiWOZ), a fully-labeled collection of human-human written conversations spanning over multiple domains and topics.At a size of 10k dialogues, it is at least one order of magnitude larger than all previous annotated task-oriented corpora.The contribution of this work apart from the open-sourced dataset is two-fold:firstly, a detailed description of the data collection procedure along with a summary of data structure and analysis is provided. The …
-
A Survey on Evaluation of Large Language ModelsYupeng Chang, Xu Wang, Jindong Wang et al. · 2023 · Computation and Language · cité 3 999 fois · Résumé IA disponible
Résumé scientifique
Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance in various applications. As LLMs continue to play a vital role in both research and daily use, their evaluation becomes increasingly critical, not only at the task level, but also at the society level for better understanding of their potential risks. Over the past years, significant efforts have been made to examine LLMs from various perspectives. This paper presents a comprehensive review of these evaluation methods for LLMs, focusing on three key dimensions: what to evaluate, where to evaluate, and how to evaluate. Firstly, we provide an overview from the …
-
OpenAssistant Conversations - Democratizing Large Language Model AlignmentAndreas Kopf, Yannic Kilcher, Dimitri von Rütte et al. · 2023 · Neural Information Processing Systems · cité 983 fois · Pas encore résumé
Résumé scientifique
Aligning large language models (LLMs) with human preferences has proven to drastically improve usability and has driven rapid adoption as demonstrated by ChatGPT. Alignment techniques such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) greatly reduce the required skill and domain knowledge to effectively harness the capabilities of LLMs, increasing their accessibility and utility across various domains. However, state-of-the-art alignment techniques like RLHF rely on high-quality human feedback data, which is expensive to create and often remains proprietary. In an effort to democratize research on large-scale alignment, we release OpenAssistant Convers…
-
Enhancing Chat Language Models by Scaling High-quality Instructional ConversationsNing Ding, Yulin Chen, Bo-Kai Xu et al. · 2023 · Conference on Empirical Methods in Natural Language Processing · cité 1 075 fois · Pas encore résumé
Résumé scientifique
Fine-tuning on instruction data has been widely validated as an effective practice for implementing chat language models like ChatGPT. Scaling the diversity and quality of such data, although straightforward, stands a great chance of leading to improved performance. This paper aims to improve the upper bound of open-source models further. We first provide a systematically designed, diverse, informative, large-scale dataset of instructional conversations, UltraChat, which does not involve human queries. Our objective is to capture the breadth of interactions that a human might have with an AI assistant and employs a comprehensive framework to generate multi-turn conversation iteratively. Ultr…
Les articles qui le citent
-
Investigating Thematic Patterns and User Preferences in LLM Interactions using BERTopicAbhay Bhandarkar, Gaurav Mishra, Khushi Juchani et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 2 fois · Pas encore résumé
Résumé scientifique
This study applies BERTopic, a transformer-based topic modeling technique, to the lmsys-chat-1m dataset, a multilingual conversational corpus built from head-to-head evaluations of large language models (LLMs). Each user prompt is paired with two anonymized LLM responses and a human preference label, used to assess user evaluation of competing model outputs. The main objective is uncovering thematic patterns in these conversations and examining their relation to user preferences, particularly if certain LLMs are consistently preferred within specific topics. A robust preprocessing pipeline was designed for multilingual variation, balancing dialogue turns, and cleaning noisy or redacted data.…
-
Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System PromptsSunnie S. Y. Kim, M. Bowler, Leon A. Gatys · 2026 · arXiv.org · preprint, non relu par les pairs · Pas encore résumé
Résumé scientifique
Large language models (LLMs) exhibit a wide range of human-like behaviors, from expressing thoughts and emotions, to engaging in relationship-building with users, to refusing requests and maintaining boundaries. Despite their prevalence, researchers and practitioners lack methods and empirical insights to make informed decisions about when and what types of human-like behaviors LLMs should exhibit. To fill this gap, we present a multi-dimensional analysis of the prevalence, potential effects, and controllability of these behaviors using LLM-as-a-judge and human evaluation. Across 21,000 multi-turn conversations from four widely used models (gpt-4o, gpt-4.1-mini, claude-sonnet-4.6, gemini-2.5…
-
Dataset Signatures in Human-LLM Interactions and User ModelingJoseph Suh, Serina Chang · 2026 · Pas encore résumé
Résumé scientifique
Human--LLM interaction datasets shape our understanding of AI use and provide a foundation for downstream research, including training and evaluation of user models. In recent years, a growing number of datasets have sought to capture a representative picture of human--LLM interactions. But how different are the pictures these datasets provide, and what do those differences mean for research built on them? We study these questions across seven conversation datasets, spanning in-the-wild chat logs and human preference data. We begin by revisiting the dataset classification experiment of Torralba&Efros and find that neural network classifiers identify the source of a conversation from user mes…
-
AI-Wrapped: Participatory, Privacy-Preserving Measurement of Longitudinal LLM Use In-the-WildCathy Mengying Fang, Sheer Karny, Chayapatr Archiwaranguprok et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 5 fois · Pas encore résumé
Résumé scientifique
Alignment research on large language models (LLMs) increasingly depends on understanding how these systems are used in everyday contexts. Yet naturalistic interaction data is difficult to access due to privacy constraints and platform control. We present AI-Wrapped, a prototype workflow for collecting naturalistic LLM chatbot usage data while providing participants with an immediate"wrapped"-style report on their usage statistics, top topics, and behavioral patterns. We report findings from an initial deployment with 82 U.S.-based adults across 48,495 conversations from their 2025 chat histories. Participants used LLMs for both instrumental and reflective purposes and had topics with emotion…
-
@GrokSet: multi-party Human-LLM Interactions in Social MediaMatteo Migliarini, Berat Ercevik, Oluwagbemike Olowe et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 1 fois · Pas encore résumé
Résumé scientifique
Large Language Models (LLMs) are increasingly deployed as active participants on public social media platforms, yet their behavior in these unconstrained social environments remains largely unstudied. Existing datasets, drawn primarily from private chat interfaces, lack the multi-party dynamics and public visibility crucial for understanding real-world performance. To address this gap, we introduce @GrokSet, a large-scale dataset of over 1 million tweets involving the @Grok LLM on X. Our analysis reveals a distinct functional shift: rather than serving as a general assistant, the LLM is frequently invoked as an authoritative arbiter in high-stakes, polarizing political debates. However, we o…
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 2 novembre 2026.
Le résumé précédent a été créé il y a plus d'un an et peut être réexécuté (si nécessaire) en cliquant sur le bouton Exécuter ci-dessous.
Avertissement : Notre outil de synthèse basé sur l'IA et l'assistant virtuel fournis sur ce site Web peuvent ne pas toujours fournir des résumés complets ou des réponses exactes. Nous vous encourageons à examiner attentivement et à évaluer le contenu généré pour vous assurer de sa qualité et de sa pertinence par rapport à vos besoins.