Within and outside the military, machine learning approaches have played an increasing role in the classification of suicidal behavior given they can more easily accommodate complex relations among independent variables and outcomes.1–4 In this study using psychological survey data from the Military Suicide Research Consortium’s (MSRC) Common Data Elements (CDE)*, three primary machine learning approaches and a traditional statistical approach showed similar classification performance of suicide thoughts and behaviors based on CDE items measuring hopelessness, thwarted belongingness, anxiety sensitivity, PTSD, insomnia, alcohol and substance use.5
For psychological survey data (as opposed to electronic health record, social media, etc.), machine learning may not outperform traditional methods as seen in a recent study from Army STARRS Regular Army survey.4 This study did replicate the Army STARRS study showing no machine learning advantage when the data comes from validated psychological measures.4
By contrast, in classifying electronic health records, social media, and other data developed without specific theories or structured measures in which the number of predictors far exceeds the sample size, machine learning has outperformed traditional methods. In other studies, the validation methods for machine learning may be flawed8 or traditional methods may have value.7 Future research is needed to determine when machine learning vs. traditional methods are the best choice.
*(N = 5,977-6,058 across outcomes)
1. Burke, T. A., Ammerman, B. A. & Jacobucci, R. The use of machine learning in the study of suicidal and non-suicidal self-injurious thoughts and behaviors: A systematic review. J. Affect. Disord. 245, 869–884 (2019).
2. Kessler, R. C. et al. Predicting suicides after psychiatric hospitalization in US Army soldiers: The Army Study To Assess Risk and Resilience in Servicemembers (Army STARRS). JAMA Psychiatry 72, 49–57 (2015).
3. Kessler, R. C. et al. Predicting suicides after outpatient mental health visits in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). Mol. Psychiatry 22, 544–551 (2017).
4. Zuromski, K. L. et al. Assessment of a risk index for suicide attempts among US Army soldiers with suicide ideation. JAMA Netw. Open 2, (2019).
5. Littlefield, A. K., Cooke, J. T., Bagge, C., Glenn, C., Kleiman, E. M., Jacobucci, R., Millner, A. J., & Steinley, D. (2021). Machine learning to classify suicidal thoughts and behaviors: Implementation within the Common Data Elements used by the Military Suicide Research Consortium. Clinical Psychological Science. //doi.org/10.1177/2167702620961067**
6. Ringer, F. B. et al. Initial validation of brief measures of suicide risk factors: Common data elements used by the Military Suicide Research Consortium. Psychol. Assess. 30, 767–778 (2018).
7. Su, C., Aseltine, R., Doshi, R., Chen, K., Rogers, S. C., & Wang, F. (2020). Machine learning for suicide risk prediction in children and adolescents with electronic health records. Translational psychiatry, 10(1), 413. doi.org/10.1038/s41398-020-01100-0
8. Jacobucci R., Littlefield A. K,. Millner A.J., Kleiman E.M., Steinley D. (2021). Evidence of Inflated Prediction Performance: A Commentary on Machine Learning and Suicide Research. Clinical Psychological Science. 2021;9(1):129-134. doi:10.1177/2167702620954216
**Denotes publication from an MSRC-funded study.