# Comparison of biomedical relationship extraction methods and models for knowledge graph creation

**Authors:** Nikola Milosevic, Wolfgang Thielemann  
**arXiv:** [2201.01647](https://arxiv.org/abs/2201.01647)  
**Date:** 2022-01-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.01647v4/

## Key points

- Biomedical research is expanding rapidly
- Knowledge graphs offer a framework for organizing and validating biomedical knowledge from literature
- Rule-based and machine learning-based methods are compared for relationship extraction from biomedical literature
- Transformer-based models perform well on small and unbalanced datasets
- PubMedBERT-based model achieves the highest F1-score of 0.92, followed closely by DistilBERT with an F1-score of 0.89
- BERT-based models outperform T5-based generative models in this context
- Researchers struggle to cope with the volume of biomedical literature and need tools to find relevant articles and validate claims
- Information retrieval approaches like PubMed and Quertle provide a list of relevant articles but do not validate hypotheses or claims
- Extracting named relationships from biomedical literature can contribute to building a large knowledge graph connecting entities through various relationships
- Future work involves creating a comprehensive biomedical knowledge graph for target identification, indication expansion, and drug discovery.

## Summary

Biomedical research is expanding rapidly, leading to an overwhelming amount of published literature. To effectively utilize this knowledge, it needs to be organized and validated. Knowledge graphs offer a framework for representing semantic knowledge from literature by extracting relationships between biomedical entities and normalizing them. This paper compares rule-based and machine learning-based methods for scalable relationship extraction from biomedical literature, including traditional machine learning methods (Naive Bayes, Random Forests) and modern deep learning transformers (DistilBERT, PubMedBERT, T5, SciFive). The study examines the resilience of these methods to unbalanced and small datasets. Experimental results show that transformer-based models perform well on both small and unbalanced datasets. The PubMedBERT-based model fine-tuned on balanced data achieves the highest F1-score of 0.92, while the DistilBERT-based model follows closely with an F1-score of 0.89. BERT-based models outperform T5-based generative models in this context. The introduction highlights the exponential growth of biomedical literature, with over 950,000 articles added to Medline in 2020 alone. Researchers struggle to cope with this volume of information and require tools to find relevant articles and validate claims and hypotheses. Information retrieval approaches have been developed for bio-medicine such as PubMed and Quertle but they only provide a list of relevant articles without validating hypotheses or claims. To validate hypotheses or claims researchers need to read through significant amounts of literature manually; however these hypotheses can often be summarized as relationships between concepts in simple sentences (e.g., "Aspirin treats pain"). Extracting these named relationships from biomedical literature can contribute to building a large knowledge graph where entities are connected through various relationships. The future work section discusses the long term task of creating a comprehensive biomedical knowledge graph for target identification indication expansion and drug discovery.