ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Proche Pas pertinent

   NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data

   Sergei Bogdanov, Alexandre Constantin, Timothée Bernard et al. · 2024 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Large Language Models (LLMs) have shown impressive abilities in data annotation, opening the way for new approaches to solve classic NLP problems. In this paper, we show how to use LLMs to create NuNER, a compact language representation model specialized in the Named Entity Recognition (NER) task. NuNER can be fine-tuned to solve downstream NER problems in a data-efficient way, outperforming similar-sized foundation models in the few-shot regime and competing with much larger LLMs. We find that the size and entity-type diversity of the pre-training dataset are key to achieving good performance. We view NuNER as a member of the broader family of task-specific foundation models, recently unloc…

2. Proche Pas pertinent

   Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models

   Wenhao Yu, Hongming Zhang, Xiaoman Pan et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Retrieval-augmented language models (RALMs) represent a substantial advancement in the capabilities of large language models, notably in reducing factual hallucination by leveraging external knowledge sources. However, the reliability of the retrieved information is not always guaranteed. The retrieval of irrelevant data can lead to misguided responses, and potentially causing the model to overlook its inherent knowledge, even when it possesses adequate information to address the query. Moreover, standard RALMs often struggle to assess whether they possess adequate knowledge, both intrinsic and retrieved, to provide an accurate answer. In situations where knowledge is lacking, these systems …

3. Proche Pas pertinent

   Conflict-Aware Soft Prompting for Retrieval-Augmented Generation

   Eunseong Choi, June Park, Hyeri Lee et al. · 2025 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Retrieval-augmented generation (RAG) enhances the capabilities of large language models (LLMs) by incorporating external knowledge into their input prompts. However, when the retrieved context contradicts the LLM's parametric knowledge, it often fails to resolve the conflict between incorrect external context and correct parametric knowledge, known as context-memory conflict. To tackle this problem, we introduce Conflict-Aware REtrieval-Augmented Generation (CARE), consisting of a context assessor and a base LLM. The context assessor encodes compact memory token embeddings from raw context tokens. Through grounded/adversarial soft prompting, the context assessor is trained to discern unrelia…

4. Proche Pas pertinent

   Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity

   Soyeong Jeong, Jinheon Baek, Sukmin Cho et al. · 2024 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Retrieval-Augmented Large Language Models (LLMs), which incorporate the non-parametric knowledge from external knowledge bases into LLMs, have emerged as a promising approach to enhancing response accuracy in several tasks, such as Question-Answering (QA). However, even though there are various approaches dealing with queries of different complexities, they either handle simple queries with unnecessary computational overhead or fail to adequately address complex multi-step queries; yet, not all user requests fall into only one of the simple or complex categories. In this work, we propose a novel adaptive QA framework, that can dynamically select the most suitable strategy for (retrieval-augm…

5. Proche Pas pertinent

   An Empirical Survey of Data Augmentation for Limited Data Learning in NLP

   Jiaao Chen, Derek Tam, Colin Raffel et al. · 2021 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   NLP has achieved great progress in the past decade through the use of neural models and large labeled datasets. The dependence on abundant data prevents NLP models from being applied to low-resource settings or novel tasks where significant time, money, or expertise is required to label massive amounts of textual data. Recently, data augmentation methods have been explored as a means of improving data efficiency in NLP. To date, there has been no systematic empirical overview of data augmentation for NLP in the limited labeled data setting, making it difficult to understand which methods work in which settings. In this paper, we provide an empirical survey of recent progress on data augmenta…

6. Proche Pas pertinent

   Active Retrieval Augmented Generation

   Zhengbao Jiang, Frank F. Xu, Luyu Gao et al. · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Despite the remarkable ability of large language models (LMs) to comprehend and generate language, they have a tendency to hallucinate and create factually inaccurate output. Augmenting LMs by retrieving information from external knowledge resources is one promising solution. Most existing retrieval augmented LMs employ a retrieve-and-generate setup that only retrieves information once based on the input. This is limiting, however, in more general scenarios involving generation of long texts, where continually gathering information throughout generation is essential. In this work, we provide a generalized view of active retrieval augmented generation, methods that actively decide when and wh…

7. Proche Pas pertinent

   LLM-powered Data Augmentation for Enhanced Crosslingual Performance

   Chenxi Whitehouse, Monojit Choudhury, Alham Fikri Aji · 2023 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   This paper aims to explore the potential of leveraging Large Language Models (LLMs) for data augmentation in crosslingual commonsense reasoning datasets, where the available training data is extremely limited. To achieve this, we employ several LLMs including Dolly-v2, StableVicuna, ChatGPT, and GPT-4 to augment three datasets: XCOPA, XWinograd, and XStoryCloze. Subsequently, we assess the effectiveness of fine-tuning smaller crosslingual models, mBERT and XLMR, using the synthesised data. We compare the performance of training with data generated in English and target languages, as well as translating the English-generated data into the target languages. Our experiments reveal the overall a…

8. Proche Pas pertinent

   Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledge

   Heydar Soudani, Evangelos Kanoulas, Faegheh Hasibi · 2024 · Calcul et langage · Résumé IA disponible

   Résumé scientifique

   Large language models (LLMs) memorize a vast amount of factual knowledge, exhibiting strong performance across diverse tasks and domains. However, it has been observed that the performance diminishes when dealing with less-popular or low-frequency concepts and entities, for example in domain specific applications. The two prominent approaches to enhance the performance of LLMs on low-frequent topics are: Retrieval Augmented Generation (RAG) and fine-tuning (FT) over synthetic data. This paper explores and evaluates the impact of RAG and FT on customizing LLMs in handling low-frequency entities on question answering task. Our findings indicate that FT significantly boosts the performance acro…