Filler Word Detection and Classification: A Dataset and Benchmark

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Apparenté Pas pertinent

   Artificial Disfluency Detection, Uh No, Disfluency Generation for the Masses

   T. Passali, T. Mavropoulos, G. Tsoumakas et al. · 2022 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Existing approaches for disfluency detection typically require the existence of large annotated datasets. However, current datasets for this task are limited, suffer from class imbalance, and lack some types of disfluencies that can be encountered in real-world scenarios. This work proposes LARD, a method for automatically generating artificial disfluencies from fluent text. LARD can simulate all the different types of disfluencies (repetitions, replacements and restarts) based on the reparandum/interregnum annotation scheme. In addition, it incorporates contextual embeddings into the disfluency generation to produce realistic context-aware artificial disfluencies. Since the proposed method …

2. Apparenté Pas pertinent

   Efficient Training of Language Models to Fill in the Middle

   Mohammad Bavarian, Heewoo Jun, Nikolas Tezak et al. · 2022 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   We show that autoregressive language models can learn to infill text after we apply a straightforward transformation to the dataset, which simply moves a span of text from the middle of a document to its end. While this data augmentation has garnered much interest in recent years, we provide extensive evidence that training models with a large fraction of data transformed in this way does not harm the original left-to-right generative capability, as measured by perplexity and sampling evaluations across a wide range of scales. Given the usefulness, simplicity, and efficiency of training models to fill-in-the-middle (FIM), we suggest that future autoregressive language models be trained with …

3. Apparenté Pas pertinent

   Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics

   Siddhant Arora, Zhiyun Lu, Chung-Cheng Chiu et al. · 2025 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   The recent wave of audio foundation models (FMs) could provide new capabilities for conversational modeling. However, there have been limited efforts to evaluate these audio FMs comprehensively on their ability to have natural and interactive conversations. To engage in meaningful conversation with the end user, we would want the FMs to additionally perform a fluent succession of turns without too much overlapping speech or long stretches of silence. Inspired by this, we ask whether the recently proposed audio FMs can understand, predict, and perform turn-taking events? To answer this, we propose a novel evaluation protocol that can assess spoken dialog system's turn-taking capabilities usin…

4. Apparenté Pas pertinent

   Podcast Summary Assessment: A Resource for Evaluating Summary Assessment Methods

   Potsawee Manakul, Mark J. F. Gales · 2022 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Automatic summary assessment is useful for both machine-generated and human-produced summaries. Automatically evaluating the summary text given the document enables, for example, summary generation system development and detection of inappropriate summaries. Summary assessment can be run in a number of modes: ranking summary generation systems; ranking summaries of a particular document; and estimating the quality of a document-summary pair on an absolute scale. Existing datasets with annotation for summary assessment are usually based on news summarization datasets such as CNN/DailyMail or XSum. In this work, we describe a new dataset, the podcast summary assessment corpus, a collection of …

5. Apparenté Pas pertinent

   Learning From Free-Text Human Feedback -- Collect New Datasets Or Extend Existing Ones?

   Dominic Petrak, Nafise Sadat Moosavi, Ye Tian et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Learning from free-text human feedback is essential for dialog systems, but annotated data is scarce and usually covers only a small fraction of error types known in conversational AI. Instead of collecting and annotating new datasets from scratch, recent advances in synthetic dialog generation could be used to augment existing dialog datasets with the necessary annotations. However, to assess the feasibility of such an effort, it is important to know the types and frequency of free-text human feedback included in these datasets. In this work, we investigate this question for a variety of commonly used dialog datasets, including MultiWoZ, SGD, BABI, PersonaChat, Wizards-of-Wikipedia, and the…

6. Apparenté Pas pertinent

   XferBench: a Data-Driven Benchmark for Emergent Language

   Brendon Boldt, David Mortensen · 2024 · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human · Résumé IA disponible

   Résumé scientifique

   In this paper, we introduce a benchmark for evaluating the overall quality of emergent languages using data-driven methods. Specifically, we interpret the notion of the "quality" of an emergent language as its similarity to human language within a deep learning framework. We measure this by using the emergent language as pretraining data for a downstream NLP tasks in human language -- the better the downstream performance, the better the emergent language. We implement this benchmark as an easy-to-use Python package that only requires a text file of utterances from the emergent language to be evaluated. Finally, we empirically test the benchmark's validity using human, synthetic, and emergen…

7. Apparenté Pas pertinent

   Proficiency assessment of L2 spoken English using wav2vec 2.0

   Stefano Bannò, Marco Matassoni · 2022 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   The increasing demand for learning English as a second language has led to a growing interest in methods for automatically assessing spoken language proficiency. Most approaches use hand-crafted features, but their efficacy relies on their particular underlying assumptions and they risk discarding potentially salient information about proficiency. Other approaches rely on transcriptions produced by ASR systems which may not provide a faithful rendition of a learner's utterance in specific scenarios (e.g., non-native children's spontaneous speech). Furthermore, transcriptions do not yield any information about relevant aspects such as intonation, rhythm or prosody. In this paper, we investiga…

8. Apparenté Pas pertinent

   SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models

   Xin Zhang, Dong Zhang, Shimin Li et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Current speech large language models build upon discrete speech representations, which can be categorized into semantic tokens and acoustic tokens. However, existing speech tokens are not specifically designed for speech language modeling. To assess the suitability of speech tokens for building speech language models, we established the first benchmark, SLMTokBench. Our results indicate that neither semantic nor acoustic tokens are ideal for this purpose. Therefore, we propose SpeechTokenizer, a unified speech tokenizer for speech large language models. SpeechTokenizer adopts the Encoder-Decoder architecture with residual vector quantization (RVQ). Unifying semantic and acoustic tokens, Spee…