# Machine Learning Approaches for Mental Illness Detection on Social Media: A Systematic Review of Biases and Methodological Challenges

**Authors:** Yuchen Cao, Jianglai Dai, Zhongyan Wang, Yeyubei Zhang, Xiaorui Shen, Yunchong Liu, Yexin Tian  
**arXiv:** [2410.16204](https://arxiv.org/abs/2410.16204)  
**Date:** 2024-10-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.16204v3/

## Key points

- Systematic review on the use of machine learning (ML) models for detecting depression through social media data
- Identified 47 relevant studies published after 2010
- Utilized Prediction model Risk Of Bias ASsessment Tool (PROBAST) to assess methodological quality and bias
- Significant biases found in studies, including heavy reliance on Twitter and English-language content, limiting diversity
- Non-probability sampling methods used in 80% of studies, affecting representativeness
- Only 23% of studies addressed linguistic nuances crucial for accurate sentiment analysis
- Risks identified: inconsistent hyperparameter tuning, inadequate data partitioning, class imbalance issues
- Future research recommendations: diversifying data sources, standardizing preprocessing methods, addressing class imbalance, enhancing reporting transparency

## Summary

This systematic review examines the use of machine learning (ML) models for detecting mental illness, specifically depression, through analysis of social media data. A comprehensive search of PubMed, IEEE Xplore, and Google Scholar identified 47 relevant studies published after 2010. The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was utilized to assess methodological quality and risk of bias in these studies. However, significant biases were found that may affect the reliability and generalizability of the ML models. These include a heavy reliance on Twitter (63.8%) and English-language content (over 90%), limiting diversity with a focus on users from the United States and Europe. Additionally, non-probability sampling methods were used in 80% of the studies which may limit representativeness. Only 23% of the studies explicitly addressed linguistic nuances such as negations which are crucial for accurate sentiment analysis. Inconsistent hyperparameter tuning (27.7%) and inadequate data partitioning (17%) were also identified as risks for overfitting. While most studies used appropriate evaluation metrics for imbalanced data (74.5%), some relied solely on accuracy without addressing class imbalance which could skew results. Reporting transparency varied across studies with many lacking critical methodological details. To address these challenges, future research should focus on diversifying data sources, standardizing preprocessing methods, ensuring consistent model development practices, addressing class imbalance issues, and enhancing reporting transparency. Through this review process involving title and abstract screening by two authors with expertise in machine learning and mental health research followed by full-text screening to ensure unbiased study selection; detailed information on study characteristics such as author names, publication details, study designs; machine learning models used; social media platforms analyzed; performance metrics measured; potential biases identified; limitations of the studies reviewed were extracted. Analytical methods were employed to systematically synthesize findings across different stages of the machine learning lifecycle in the selected studies including sampling techniques, data preprocessing methods, model construction, tuning, and evaluation based on quantitative metrics such as accuracy, precision, recall, F1 scores, and AUROCs. The importance of reporting transparency and completeness in scientific research was emphasized throughout the review process to ensure integrity, reproducibility, and reliability of findings reported in each study. By addressing these challenges and improving reporting standards, more robust and generalizable ML models can be developed for depression detection on social media platforms to contribute to improved mental health outcomes globally.