# GRENADE: Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs

**Authors:** Yichuan Li, Kaize Ding, Kyumin Lee  
**arXiv:** [2310.15109](https://arxiv.org/abs/2310.15109)  
**Date:** 2023-10-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.15109v1/

## Key points

- Researchers in self-supervised representation learning on text-attributed graphs aim to develop expressive and generalizable representations for downstream tasks.
- Current methods face challenges in capturing complete structural context information or rely heavily on task-specific training labels, limiting their effectiveness and practical applicability.
- GRENADE is a cutting-edge solution that combines pre-trained language models and graph neural networks, utilizing graph-centric contrastive learning and graph-centric knowledge alignment.
- GRENADE effectively captures informative textual semantics and crucial structural context information within text-attributed graphs.
- Extensive experimentation has shown GRENADE's superiority over existing state-of-the-art methods in terms of performance and versatility.
- The implementation of GRENADE is accessible at https://github.com/bigheiniu/GRENADE, demonstrating its potential to revolutionize self-supervised representation learning on text-attributed graphs.

## Summary

In the field of self-supervised representation learning on text-attributed graphs, researchers are increasingly focused on developing expressive and generalizable representations for various downstream tasks. However, current methods face challenges in capturing complete structural context information or rely heavily on task-specific training labels, limiting their effectiveness and practical applicability. To address this issue, a cutting-edge solution called GRENADE has been introduced - a Graph-Centric Language Model for Self-Supervised Representation Learning on Text-Attributed Graphs. By combining pre-trained language models and graph neural networks and utilizing two specialized self-supervised learning algorithms - graph-centric contrastive learning and graph-centric knowledge alignment - GRENADE stands out as an innovative approach. These techniques enable GRENADE to effectively capture both informative textual semantics and crucial structural context information within text-attributed graphs. Extensive experimentation has demonstrated its superiority over existing state-of-the-art methods in terms of performance and versatility. The implementation of GRENADE is readily accessible at https://github.com/bigheiniu/GRENADE, showcasing its potential to revolutionize self-supervised representation learning on text-attributed graphs. This groundbreaking work was conducted by Yichuan Li, Kaize Ding, and Kyumin Lee and presented at EMNLP 2023.