Artificial Disfluency Detection, Uh No, Disfluency Generation for the Masses

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Apparenté Pas pertinent

   Active Retrieval Augmented Generation

   Zhengbao Jiang, Frank F. Xu, Luyu Gao et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augmenting LMs by retrieving information from external knowledge resources is one promising solution. Most existing retrieval-augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. This is limiting, however, in more general scenarios involving generation of long texts, where continually gathering information throughout the generation process is essential. There have been some past efforts to retrieve information multiple times while generating outputs, which mostly re…

2. Apparenté Pas pertinent

   A Survey on LLM-generated Text Detection: Necessity, Methods, and Future Directions

   Junchao Wu, Shu Yang, Runzhe Zhan et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   The powerful ability to understand, follow, and generate complex language emerging from large language models (LLMs) makes LLM-generated text flood many areas of our daily lives at an incredible speed and is widely accepted by humans. As LLMs continue to expand, there is an imperative need to develop detectors that can detect LLM-generated text. This is crucial to mitigate potential misuse of LLMs and safeguard realms like artistic expression and social networks from harmful influence of LLM-generated content. The LLM-generated text detection aims to discern if a piece of text was produced by an LLM, which is essentially a binary classification task. The detector techniques have witnessed no…

3. Apparenté Pas pertinent

   APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decoding

   Mingdao Liu, Aohan Zeng, Bowen Wang et al. · 2024 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   The massive adoption of large language models (LLMs) demands efficient deployment strategies. However, the auto-regressive decoding process, which is fundamental to how most LLMs generate text, poses challenges to achieve efficient serving. In this work, we introduce a parallel auto-regressive generation method. By instruct-tuning on general domain data that contains hierarchical structures, we enable LLMs to independently plan their generation process and perform auto-parallel auto-regressive (APAR) generation, significantly reducing the number of generation steps. APAR alone can achieve up to 2x speed-up, and when combined with speculative decoding, the speed-up can reach up to 4x. In addi…

4. Apparenté Pas pertinent

   Filler Word Detection and Classification: A Dataset and Benchmark

   Ge Zhu, Juan-Pablo Caceres, Justin Salamon · 2022 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Filler words such as `uh' or `um' are sounds or words people use to signal they are pausing to think. Finding and removing filler words from recordings is a common and tedious task in media editing. Automatically detecting and classifying filler words could greatly aid in this task, but few studies have been published on this problem. A key reason is the absence of a dataset with annotated filler words for training and evaluation. In this work, we present a novel speech dataset, PodcastFillers, with 35K annotated filler words and 50K annotations of other sounds that commonly occur in podcasts such as breaths, laughter, and word repetitions. We propose a pipeline that leverages VAD and ASR to…

5. Apparenté Pas pertinent

   DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation

   Shansan Gong, Ruixiang Zhang, Huangjie Zheng et al. · 2025 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Diffusion large language models (dLLMs) are compelling alternatives to autoregressive (AR) models because their denoising models operate over the entire sequence. The global planning and iterative refinement features of dLLMs are particularly useful for code generation. However, current training and inference mechanisms for dLLMs in coding are still under-explored. To demystify the decoding behavior of dLLMs and unlock their potential for coding, we systematically investigate their denoising processes and reinforcement learning (RL) methods. We train a 7B dLLM, \textbf{DiffuCoder}, on 130B tokens of code. Using this model as a testbed, we analyze its decoding behavior, revealing how it diffe…

6. Apparenté Pas pertinent

   RAG-DDR: Optimizing Retrieval-Augmented Generation Using Differentiable Data Rewards

   Xinze Li, Sen Mei, Zhenghao Liu et al. · 2024 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Retrieval-Augmented Generation (RAG) has proven its effectiveness in mitigating hallucinations in Large Language Models (LLMs) by retrieving knowledge from external resources. To adapt LLMs for RAG pipelines, current approaches use instruction tuning to optimize LLMs, improving their ability to utilize retrieved knowledge. This supervised fine-tuning (SFT) approach focuses on equipping LLMs to handle diverse RAG tasks using different instructions. However, it trains RAG modules to overfit training signals and overlooks the varying data preferences among agents within the RAG system. In this paper, we propose a Differentiable Data Rewards (DDR) method, which end-to-end trains RAG systems by a…

7. Apparenté Pas pertinent

   RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models

   Yuanhao Wu, Juno Zhu, Siliang Xu et al. · 2023 · Calcul et langage · Pas encore résumé

   Résumé scientifique

   Retrieval-augmented generation (RAG) has become a main technique for alleviating hallucinations in large language models (LLMs). Despite the integration of RAG, LLMs may still present unsupported or contradictory claims to the retrieved contents. In order to develop effective hallucination prevention strategies under RAG, it is important to create benchmark datasets that can measure the extent of hallucination. This paper presents RAGTruth, a corpus tailored for analyzing word-level hallucinations in various domains and tasks within the standard RAG frameworks for LLM applications. RAGTruth comprises nearly 18,000 naturally generated responses from diverse LLMs using RAG. These responses hav…

8. Apparenté Pas pertinent

   DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows

   Ajay Patel, Colin Raffel, Chris Callison-Burch · 2024 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Large language models (LLMs) have become a dominant and important tool for NLP researchers in a wide range of tasks. Today, many researchers use LLMs in synthetic data generation, task evaluation, fine-tuning, distillation, and other model-in-the-loop research workflows. However, challenges arise when using these models that stem from their scale, their closed source nature, and the lack of standardized tooling for these new and emerging workflows. The rapid rise to prominence of these models and these unique challenges has had immediate adverse impacts on open science and on the reproducibility of work that uses them. In this paper, we introduce DataDreamer, an open source Python library th…