# Counts@IITK at SemEval-2021 Task 8: SciBERT Based Entity And Semantic Relation Extraction For Scientific Data

**Authors:** Akash Gangwar, Sabhay Jain, Shubham Sourav, Ashutosh Modi  
**arXiv:** [2104.01364](https://arxiv.org/abs/2104.01364)  
**Date:** 2021-04-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2104.01364v1/

## Key points

- System developed for SemEval 2021 Task 8 (MeasEval)
- Utilized SciBERT with [CLS] token embedding and a CRF layer
- Achieved an overall F1-overlap score of 0.432, ranking fifth on the leaderboard
- Implementation of the system is available on Github
- Background information on related work in entity extraction and relation extraction using LSTM CRF, BERT, and CRF layers
- Task setup for SemEval 2021 Task 8: articles manually annotated for quantities, measured entities, properties, qualifiers, and units
- Pre-processing steps using SciSpaCy to split paragraphs into sentences for input to the SciBERT model
- Training dataset included paragraphs with quantities, measured entities, properties, and qualifiers; evaluation set used separate paragraphs for testing

## Summary

This paper presents the system developed for SemEval 2021 Task 8 (MeasEval), which focuses on extracting and classifying spans and relations to identify quantities, attributes of quantities, and related information in scientific data. The submitted system utilized SciBERT with [CLS] token embedding and a CRF layer, achieving an overall F1-overlap score of 0.432 and ranking fifth on the leaderboard. The top-performing system on the leaderboard achieved an F1-overlap score of 0.519. The implementation of the system is available on Github. The paper provides background information on related work in entity extraction and relation extraction using models like LSTM CRF, BERT, and CRF layers. It also discusses the task setup for SemEval 2021 Task 8, which includes articles from various sub-domains manually annotated for quantities, measured entities, properties, qualifiers, and units. The system overview details the pre-processing steps using SciSpaCy to split paragraphs into sentences for input to the SciBERT model. The training dataset consisted of paragraphs with quantities, measured entities, properties, and qualifiers while the evaluation set included a separate set of paragraphs for testing. Overall,this paper contributes to semantic relation extraction in scientific data by participating in MeasEval Task 8 at SemEval 2021 and providing insights into system performance analysis.