# Leveraging Large Language Models for Mental Health Prediction via Online Text Data

**Authors:** Xuhai Xu, Bingshen Yao, Yuanzhe Dong, Hong Yu, James Hendler, Anind K. Dey, Dakuo Wang  
**arXiv:** [2307.14385](https://arxiv.org/abs/2307.14385)  
**Date:** 2023-07-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.14385v1/

## Key points

- Recent surge of Large Language Models (LLMs) including GPT-3.5/4, PaLM, FLAN-T5, and Alpaca showing promising potential for various applications
- Lack of research focusing on understanding and enhancing LLMs' capabilities in the mental health domain
- Comprehensive evaluation of multiple LLMs specifically for mental health prediction tasks using online text data
- Experiment results show that while LLMs show promise in mental health tasks, their performance is not yet comparable to task-specific NLP models
- Prompt design enhancement strategies effective for critical action prediction tasks like suicide prediction
- Instruction finetuning significantly improves the performance of LLMs across all mental health prediction tasks simultaneously
- Best-finetuned model developed in the study - Mental-Alpaca - outperforms larger GPT-3.5 model by 16.7% on balanced accuracy and performs comparably to state-of-the-art task-specific models
- Effectiveness of prompt design enhancements for mental health tasks and potential for further improvement through instruction finetuning highlighted
- Utilization of four diverse mental health datasets to define six different mental health prediction tasks ranging from binary stress prediction to five-level suicide risk prediction at both post-level and user-level data

## Summary

The recent surge of Large Language Models (LLMs), including GPT-3.5/4, PaLM, FLAN-T5, and Alpaca, has shown promising potential for various applications. However, there is a lack of research focusing on understanding and enhancing LLMs' capabilities in the mental health domain. This study presents a comprehensive evaluation of multiple LLMs specifically for mental health prediction tasks using online text data. The experiments cover zero-shot prompting, few-shot prompting, and instruction finetuning to assess the performance of LLMs on mental health tasks. Results show that while LLMs show promise in these tasks, their performance is not yet comparable to task-specific NLP models. <br>
Through detailed experiments and analysis, it was found that prompt design enhancement strategies are effective for critical action prediction tasks like suicide prediction. Furthermore,<br>
instruction finetuning significantly improves the performance of LLMs across all mental health prediction tasks simultaneously. The best-finetuned model developed in this study - Mental-Alpaca - outperforms the larger GPT-3.5 model by 16.7% on balanced accuracy and performs comparably to state-of-the-art task-specific models.<br>
This research highlights key takeaways such as the effectiveness of prompt design enhancements for mental health tasks and the potential for further improvement through instruction finetuning. It also provides actionable guidelines for future researchers, engineers, and practitioners looking to enhance LLMs with better knowledge in the mental health domain and excel in mental health prediction tasks.<br>
Additionally,<br>
the study utilized four diverse mental health datasets - Dreaddit, DepSeverity, SDCNL, and CSSRS-Suicide - to define six different mental health prediction tasks ranging from binary stress prediction to five-level suicide risk prediction at both post-level and user-level data. Overall, this research contributes valuable insights into leveraging large language models for mental health prediction using online text data and highlights areas for further exploration and improvement in this important domain.