# ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER

**Authors:** Sreyan Ghosh, Utkarsh Tyagi, Manan Suri, Sonal Kumar, S Ramaneswaran, Dinesh Manocha  
**arXiv:** [2306.00928](https://arxiv.org/abs/2306.00928)  
**Date:** 2023-06-01  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.00928v1/

## Key points

- Complex Named Entity Recognition (NER) is a challenging task in low-context text
- Existing data augmentation techniques for low-resource complex NER suffer from context-entity mismatch
- ACLM (Attention-map aware keyword selection for Conditional Language Model fine-tuning) is a novel data augmentation approach based on conditional generation
- ACLM uses selective masking aided by attention maps to retain named entities and relevant keywords
- ACLM outperforms other neural baselines by a significant margin in monolingual, cross-lingual, and multilingual complex NER across various low-resource settings
- ACLM also demonstrates promising results in biomedical research and other domains suffering from data scarcity
- Mixner algorithm enhances diversity of augmentations by mixing two templates during generation phase
- Benefits of ACLM demonstrated qualitatively and quantitatively on the Multi-CoNER dataset
- ACLM effectively addresses context-entity mismatch problem and generates diverse, coherent, and high-quality augmentations
- ACLM outperforms previous methods in science and medicine domains with absolute gains ranging from 1% to 11%

## Summary

Complex Named Entity Recognition (NER) is a challenging task that involves detecting linguistically complex named entities in low-context text. The existing data augmentation techniques for low-resource complex NER often suffer from the context-entity mismatch issue, resulting in incoherent augmentations where complex named entities are placed in the wrong context. To address this problem, the authors propose ACLM (Attention-map aware keyword selection for Conditional Language Model fine-tuning), a novel data augmentation approach based on conditional generation. ACLM is built upon BART and is optimized on a text reconstruction or denoising task. It introduces selective masking aided by attention maps to retain the named entities and certain keywords in the input sentence that provide contextually relevant additional knowledge or hints about the named entities. By generating diverse and coherent augmentations while preserving the true word sense of complex entities, ACLM outperforms other neural baselines by a significant margin (1%-36%) in monolingual, cross-lingual, and multilingual complex NER across various low-resource settings. In addition to its effectiveness in complex NER, ACLM also demonstrates promising results when applied to other domains suffering from data scarcity such as biomedical research. Compared to prior methods, ACLM generates more effective and factual augmentations for these domains. The authors propose mixner, a novel algorithm that further enhances the diversity of augmentations by mixing two templates during the generation phase. This boosts the variety of generated samples and contributes to ACLM's overall performance. The benefits of ACLM are demonstrated both qualitatively and quantitatively on the Multi-CoNER dataset. Compared to previous methods, ACLM effectively addresses the context-entity mismatch problem and generates more diverse, coherent, and high-quality augmentations. Extensive experiments also show its applicability in science and medicine domains where it outperforms all baselines with absolute gains ranging from 1% to 11%. In summary, ACLM is a novel data augmentation framework specifically designed for low-resource complex NER. It effectively tackles the context-entity mismatch problem, generates diverse and coherent augmentations, and outperforms previous methods in various settings.