Social Media Images Can Predict Suicide Risk Using Interpretable Large Language-Vision Models
Published in The Journal of Clinical Psychiatry, 2023
Recommended citation: Yael Badian, Yaakov Ophir, Refael Tikochinski, Nitay Calderon, Anat Brunstein Klomek, Eyal Fruchter and Roi Reichart https://legacy.psychiatrist.com/jcp/social-media-images-can-predict-suicide-risk-using-interpretable-large-language-vision-models/
Abstract
Background: Suicide, a leading cause of death and a major public health concern, became an even more pressing matter since the emergence of social media two decades ago and, more recently, following the hardships that characterized the COVID-19 crisis. Contemporary studies therefore aim to predict signs of suicide risk from social media using highly advanced artificial intelligence (AI) methods. Indeed, these new AI-based studies managed to break a longstanding prediction ceiling in suicidology; however, they still have principal limitations that prevent their implementation in real-life settings. These include “black box” methodologies, inadequate outcome measures, and scarce research on non-verbal inputs, such as images (despite their popularity today).Objective: This study aims to address these limitations and present an interpretable prediction model of clinically valid suicide risk from images.
Methods: The data were extracted from a larger dataset from May through June 2018 that was used to predict suicide risk from textual postings. Specifically, the extracted data included a total of 177,220 images that were uploaded by 841 Facebook users who completed a gold-standard suicide scale. The images were represented with CLIP (Contrastive Language-Image Pre-training), a state-of-the-art deep-learning algorithm, which was utilized, unconventionally, to extract predefined interpretable features (eg, “photo of sad people”) that served as inputs to a simple logistic regression model.
Results: The results of this hybrid model that integrated theory-driven features with bottom-up methods indicated high prediction performance that surpassed common deep learning algorithms (area under the receiver operating characteristic curve [AUC] = 0.720, Cohen d = 0.82). Further analyses supported a theory-driven hypothesis that at-risk users would have images with increased negative emotions and decreased belongingness.
Conclusions: This study provides a first proof that publicly available images can be leveraged to predict validated suicide risk. It also provides simple and flexible strategies that could enhance the development of real-life monitoring tools for suicide.
bibtex
@article{badian2023social,
title={Social media images can predict suicide risk using interpretable large language-vision models},
author={Badian, Yael and Ophir, Yaakov and Tikochinski, Refael and Calderon, Nitay and Klomek, Anat Brunstein and Fruchter, Eyal and Reichart, Roi},
journal={The Journal of Clinical Psychiatry},
volume={85},
number={1},
pages={50516},
year={2023},
publisher={Physicians Postgraduate Press, Inc.}
}