# ProAI: Proactive Multi-Agent Conversational AI with Structured Knowledge Base for Psychiatric Diagnosis

**Authors:** Yuqi Wu, Guangya Wan, Jingjing Li, Shengming Zhao, Lingfeng Ma, Tianyi Ye, Ion Pop, Yanbo Zhang, Jie Chen  
**arXiv:** [2502.20689](https://arxiv.org/abs/2502.20689)  
**Date:** 2025-02-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.20689v1/

## Key points

- ProAI is a goal-oriented and proactive conversational AI framework designed to enhance diagnostic capabilities
- It takes a proactive approach by asking relevant questions and steering conversations towards specific objectives
- Simulation of patient interactions using an LLM agent with clinically informed symptomatology and behavior
- Evaluation of diagnostic accuracy using Critical Node Recall (CN-Recall) and Differential Diagnosis Accuracy (DDx-ACC) metrics
- User experience evaluation based on Helpfulness and Empathy dimensions
- Doctor evaluation based on Specialty and Precision metrics related to clinical quality, coherence, adherence to guidelines, accuracy, and specificity of diagnoses
- Future considerations include expanding evaluation to include a broader range of psychiatric disorders, conducting clinical trials with real patients, and automating knowledge graph construction for different diagnostic domains

## Summary

, , , , 

In this study, we introduce ProAI, a goal-oriented and proactive conversational AI framework designed to enhance the diagnostic capabilities of language model-driven conversational systems. Most existing conversational AI systems operate reactively, responding to user prompts without actively guiding the interaction. However, ProAI takes a proactive approach by asking relevant questions and steering conversations towards specific objectives. Drawing inspiration from Wu et al. (2023, 2024), we simulate patient interactions using an LLM agent that embodies specific mental disorders with clinically informed symptomatology and behavior. Through multi-round conversations, ProAI engages in diagnostic reasoning to accurately identify the patient's condition. The system's diagnostic accuracy is evaluated using Critical Node Recall (CN-Recall) and Differential Diagnosis Accuracy (DDx-ACC) metrics, which assess its thoroughness in assessing essential criteria nodes and ability to reach correct diagnostic conclusions while ruling out alternative conditions. User experience evaluation is crucial for ensuring a positive patient experience during clinical interviews. We measure two key dimensions - Helpfulness and Empathy - to evaluate the effectiveness of the agent's medical consultation and its ability to demonstrate understanding and build rapport with patients. Additionally, doctor evaluation ensures that diagnostic decisions are based on rigorous medical reasoning by assessing Specialty and Precision metrics related to clinical quality, coherence, adherence to guidelines, accuracy, and specificity of diagnoses. While ProAI demonstrates strong performance in mental health differential diagnosis, several limitations need consideration. Future work should expand evaluation to include a broader range of psychiatric disorders and tasks beyond mental health diagnosis. Clinical trials with real patients would further validate the system's practical utility. Automation of knowledge graph construction for different diagnostic domains could streamline system development. Overall, our study highlights the potential for more reliable, adaptive, and goal-driven AI diagnostic assistants by advancing LLMs beyond reactive dialogue systems. By combining different LLMs in a hybrid system approach like "Two Agents Mixed," we can achieve a better balance between accuracy and user experience in clinical AI systems through thoughtful design and strategic model selection.