# UPB at SemEval-2021 Task 8: Extracting Semantic Information on Measurements as Multi-Turn Question Answering

**Authors:** Andrei-Marius Avram, George-Eduard Zaharia, Dumitru-Clementin Cercel, Mihai Dascalu  
**arXiv:** [2104.04549](https://arxiv.org/abs/2104.04549)  
**Date:** 2021-04-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2104.04549v1/

## Key points

- Authors' approach to solving all five subtasks of the 8th task of MeasEval competition at SemEval-2021
- Cascade system with individual subsystems for first two subtasks and single subsystem for last three subtasks
- Identifying quantities using a pretrained language model with CRF layer
- Extracting measurement units and modifiers using Bidirectional LSTMs at character level
- Identifying measured entities, properties, qualifiers, and relations using multi-turn question answering approach with hand-crafted questions specific to each relation type
- Best performing model achieved an F1-score of 36.91% on test set
- Limitations highlighted regarding unit extraction and sensitivity to identified quantities' quality
- Discussion on related work in span identification, measurement unit identification, and relation extraction, including models like CRFs, LSTM cells with CRF, BERT+CRF, SpanBERT, and different neural network-based models
- Paper structured into sections discussing solutions for relation extraction, span identification, and measurement unit identification while outlining approaches taken for each subtask proposed by MeasEval competition
- Performance evaluation of systems presented along with error analysis
- Concluding remarks and suggestions for future improvements in this area of research

## Summary

The paper presents the authors' approach to solving all five subtasks of the 8th task of MeasEval competition at SemEval-2021. The competition aims to advance research in extracting semantic information on measurements from scientific texts. Their approach consists of a cascade system with individual subsystems for each problem in the first two subtasks and a single subsystem for jointly solving the last three subtasks. The first step is identifying quantities using a pretrained language model with a Conditional Random Fields (CRF) layer. Then, measurement units and modifiers are extracted using Bidirectional LSTMs at the character level. Finally, measured entities, properties, and qualifiers are identified along with their relations using a multi-turn question answering approach with hand-crafted questions specific to each relation type. The best performing model achieved an F1-score of 36.91% on the test set; however, limitations were also highlighted regarding unit extraction and sensitivity to identified quantities' quality. Related work in span identification, measurement unit identification, and relation extraction was discussed as well as various models used in previous studies such as CRFs, LSTM cells with CRF, BERT+CRF, SpanBERT, and different neural network-based models. The paper is structured into sections discussing solutions for relation extraction, span identification, and measurement unit identification while outlining approaches taken for each subtask proposed by MeasEval competition. A performance evaluation of their systems together with an error analysis is presented followed by concluding remarks and suggestions for future improvements in this area of research.