# Graph-ToolFormer: To Empower LLMs with Graph Reasoning Ability via Prompt Augmented by ChatGPT

**Authors:** Jiawei Zhang  
**arXiv:** [2304.11116](https://arxiv.org/abs/2304.11116)  
**Date:** 2023-04-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.11116v3/

## Key points

- The paper aims to develop a large language model (LLM) with reasoning ability on complex graph data.
- Existing LLMs have flaws in multi-step logic reasoning, precise mathematical calculation, and perception of spatial and temporal factors in graph learning tasks.
- The authors propose the Graph-ToolFormer framework to address these challenges.
- The framework teaches LLMs to use external graph reasoning API tools through prompts augmented by ChatGPT.
- Various graph data reasoning tasks are investigated, including basic graph property reasoning tasks and advanced reasoning tasks on real-world graph data.
- Potential future research opportunities for incorporating other advanced graph reasoning tasks into the Graph-ToolFormer framework are discussed.
- Section 4.3.3 specifically focuses on advanced graph reasoning tasks using real-world graph data with more complex structures.
- Examples of specific reasoning tasks are provided, such as academic paper topic reasoning, protein function reasoning, sequential product recommendation reasoning, social community reasoning, and semantics reasoning on knowledge graphs.
- The authors demonstrate how ChatGPT can generate output including API calls for graph reasoning based on input statements and queries.
- Input-output prompt pair datasets are generated using API calls to compare return results with true values in statements.
- Outputs with unrunnable or incorrect API calls are filtered from the dataset overall.
- The paper presents a framework for empowering LLMs with graph reasonability using ChatGPT to annotate APIs generating prompts.

## Summary

This paper aims to develop a large language model (LLM) with reasoning ability on complex graph data. Existing LLMs have flaws in performing multi-step logic reasoning, precise mathematical calculation, and perception of spatial and temporal factors in graph learning tasks. To address these challenges, the authors propose the Graph-ToolFormer framework, inspired by ChatGPT and Toolformer models. This framework teaches LLMs to use external graph reasoning API tools through prompts augmented by ChatGPT. The authors investigate various graph data reasoning tasks, including basic graph property reasoning tasks and advanced reasoning tasks on real-world graph data such as bibliographic networks, protein molecules, sequential recommender systems, social networks, and knowledge graphs. The paper also discusses potential future research opportunities for incorporating other advanced graph reasoning tasks into the Graph-ToolFormer framework. It highlights the flexibility of the framework to host various graph reasoning application tasks with LLMs as the general interface. In Section 4.3.3, the authors specifically focus on advanced graph reasoning tasks using real-world graph data with more complex structures. These include academic paper topic reasoning on bibliographic networks, protein function reasoning based on protein graph structures, sequential product recommendation reasoning based on recommender systems, social community reasoning from online social networks, and semantics reasoning on knowledge graphs. For example, in the case of academic paper topic reasoning on bibliographic networks, the authors discuss how topics of a paper can be inferred not only from its own textual descriptions but also from other papers cited by or citing it. They represent the bibliographic network as G = (V , E) and load it via an API call <API>GL("bibliographic-network") → G</API>. They provide an input statement about a paper in the Cora bibliographic network with a topic of "Neural Networks" and use ChatGPT to insert appropriate API calls for this statement. The authors demonstrate how ChatGPT can generate the output including the API call for graph reasoning based on the input statement and query. They also show rephrased versions of the input statement and how ChatGPT can annotate API calls for these variations. The authors mention that this process is done for all node/graph instances studied in both basic graph property reasoning tasks and advanced graph reasoning tasks to generate input-output prompt pair datasets which are used to run API calls comparing return results with true values in statements; outputs with unrunnable or incorrect API calls being filtered from dataset overall concluding that this paper presents a framework for empowering LLMs with graph reasonability exploring various advanced ones using real world data demonstrating use of ChatGPT to annotate APIs generating prompts discussing potential future research opportunities in this field too.