# SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents

**Authors:** Qi Zhang, Zhijia Chen, Huitong Pan, Cornelia Caragea, Longin Jan Latecki, Eduard Dragut  
**arXiv:** [2410.21155](https://arxiv.org/abs/2410.21155)  
**Date:** 2024-10-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.21155v1/

## Key points

- The field of scientific information extraction (SciIE) has focused on developing datasets for training and validating models.
- Existing datasets often do not capture the full complexity of scientific texts due to the high cost and intricacy of annotating such materials.
- Authors Qi Zhang, Zhijia Chen, Huitong Pan, Cornelia Caragea, Longin Jan Latecki, and Eduard Dragut introduced a new dataset named SciER targeting entities related to datasets, methods, and tasks in scientific documents.
- SciER comprises 106 fully annotated publications with over 24k entities and 12k relations meticulously identified.
- SciER stands out for its meticulous attention to detail in capturing intricate entity interactions within full-text articles through a fine-grained tag set for relations.
- An out-of-distribution test set is provided for a more realistic evaluation of models trained on the dataset.
- Comprehensive experiments were conducted using state-of-the-art supervised models and LLM-based baselines to highlight challenges and encourage innovation in the field of SciIE.

## Summary

The field of scientific information extraction (SciIE) has seen a significant focus on developing datasets for training and validating models. However, existing datasets often fall short in capturing the full complexity of scientific texts due to the high cost and intricacy of annotating such materials. To address this gap, authors Qi Zhang, Zhijia Chen, Huitong Pan, Cornelia Caragea, Longin Jan Latecki, and Eduard Dragut have introduced a new dataset named SciER that targets entities related to datasets, methods, and tasks found in scientific documents. This dataset comprises 106 fully annotated publications with over 24k entities and 12k relations meticulously identified. What sets SciER apart is its meticulous attention to detail in capturing the intricate use and interactions among entities within full-text articles. To achieve this level of granularity, the dataset incorporates a fine-grained tag set for relations. Additionally, an out-of-distribution test set is provided for a more realistic evaluation of models trained on the dataset. The team conducted comprehensive experiments using both state-of-the-art supervised models and their proposed LLM-based baselines. Through these experiments, they shed light on the challenges presented by their dataset while also encouraging researchers to develop innovative models that can further advance the field of SciIE. Overall,<Organization>,SciER represents a significant contribution to the field by providing researchers with a rich resource for training and evaluating entity and relation extraction models tailored specifically for scientific texts. This dataset not only expands the scope of available resources but also highlights the importance of capturing nuanced relationships within scientific literature for more accurate information extraction processes.