# Edge: Enriching Knowledge Graph Embeddings with External Text

**Authors:** Saed Rezayi, Handong Zhao, Sungchul Kim, Ryan A. Rossi, Nedim Lipka, Sheng Li  
**arXiv:** [2104.04909](https://arxiv.org/abs/2104.04909)  
**Date:** 2021-04-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2104.04909v1/

## Key points

- Authors propose a framework called EDGE for knowledge graph enrichment and embedding
- Goal is to address sparsity in knowledge graphs by integrating a learning component
- EDGE constructs a graph using external text based on semantic and structural similarities
- Aligns the constructed graph with the original knowledge graph in the same embedding space
- Infuses learning through graph alignment to ensure similar entities are close together and dissimilar entities are pushed apart
- Integration of information from an auxiliary textual source enhances the quality of low-dimensional embeddings
- Takes input of a knowledge graph (KG) and an external source of texts (T) to generate an augmented knowledge graph (aKG)
- Multi-criteria objective function is devised to align KG and aKG by minimizing distance between their embeddings
- Extensive experiments show that EDGE outperforms state-of-the-art models in link prediction and node classification tasks
- Contributions include proposing EDGE, introducing procedure for generating augmented knowledge graph, and novel embedding approach with multi-criteria objective function optimization
- Paper is organized into sections discussing gaps in existing literature, problem definition, evaluation through experiments, conclusion, and future directions
- Related work focuses on heterogeneous knowledge graphs while this work focuses on entity embedding learning

## Summary

In this paper, the authors propose a novel framework called EDGE for knowledge graph enrichment and embedding to address the issue of sparsity in knowledge graphs. The goal is to integrate a learning component into the process to improve the quality of representations generated by various methods. EDGE first constructs a graph using external text based on semantic and structural similarities and aligns it with the original knowledge graph in the same embedding space. By infusing learning in the knowledge distillation process through graph alignment, EDGE ensures that similar entities remain close together while dissimilar entities are pushed further apart. This integration of information from an auxiliary textual source helps enhance the quality of low-dimensional embeddings by introducing new features. The framework takes as input a knowledge graph (KG) and an external source of texts (T) and generates an augmented knowledge graph (aKG). The generation of aKG considers semantic and structural similarities among KG entities and ensures that all original entities are included. This facilitates the alignment process between KG and aKG in the embedding space. To align KG and aKG, a multi-criteria objective function is devised, which minimizes the distance between their embeddings. Textual nodes related to each target entity are rewarded while unrelated ones are penalized using negative sampling. Extensive experiments on four benchmark datasets demonstrate that EDGE outperforms state-of-the-art models in tasks such as link prediction and node classification. The evaluation results also confirm the generalizability of the model. The contributions of this work include: proposing EDGE, introducing a procedure to generate an augmented knowledge graph from external texts that is linked with the original knowledge graph; proposing a novel knowledge graph embedding approach that optimizes a multi-criteria objective function to align two knowledge graphs in a joint embedding space; demonstrating effectiveness and generalizability through evaluations on link prediction and node classification tasks on four datasets. The rest of the paper is organized as follows: Section 2 identifies gaps in existing literature; Section 3 provides problem definition with detailed explanation of proposed model; Section 4 evaluates model through experiments on link prediction and node classification presenting results along with ablation study; finally concluding work discussing future directions in Section 5. In terms of related work, many approaches focus on heterogeneous knowledge graphs with different types of edges whereas this work specifically considers only one type relation focusing solely on entity embedding learning; additionally mentioning its relation to Graph Neural Networks such as Graph Convolutional Networks (GCN).