4 - (SYM 61) Leveraging Clinician Suicide Risk Assessment to Predict Suicidal Behavior in a Large Healthcare System

Suicide is among the top ten leading causes of death (CDC, 2022). As > 75% of people who die by suicide saw a healthcare provider within the year prior to their death (Luoma et al., 2002), healthcare systems are a key setting in which to focus suicide risk detection efforts. The Joint Commission now recommends conducting standardized suicide risk screening and assessment in nearly all clinical settings. Although many studies point to the limitations of traditional suicide risk screening measures (e.g., suboptimal sensitivity, reliance on patient self-disclosure), data on the predictive accuracy of comprehensive clinician suicide risk assessment are limited. Improving the understanding of how well clinicians do at determining patients’ risk of suicide in routine care is key to informing myriad suicide risk detection efforts, including how to integrate newer statistical approaches (e.g., Barak-Corren et al., 2020) with clinician assessment (Simon et al., 2021). The SAFE-T (Suicide Assessment Five-Step Evaluation and Triage) is a widely used framework for suicide risk assessment that involves clinicians identifying risk/protective factors, assessing suicidal thoughts and behaviors (STBs), using clinical judgment to determine risk level, and delivering appropriate interventions. At Mass General Brigham (MGB), conducting and documenting (using a structured electronic health record [EHR] form) the SAFE-T is required at all outpatient behavioral health visits. We recently harvested data from 563,200 SAFE-Ts among 70,610 MGB patients from July 2019 to January 2021. Data from each of the five SAFE-T steps were transformed into binary or ordinal features to be used in predictive models. Two binary outcomes were ascertained from the EHR: suicide attempt and emergency department (ED) visit for STBs within 6 months of each index SAFE-T. Using time-based training and test splits, we fit a series of lasso and random forest models using different combinations of SAFE-T features to predict future suicide attempt and ED visit for STBs. Random forest models using clinician judgment of risk (None, Low, Moderate, or High) and recent STBs (as documented on the SAFE-T EHR form) to predict suicide attempt had the highest accuracy (AUC = 0.86 [95% CI: 0.84-0.88], sensitivity = 0.70, specificity = 0.84). Ongoing analyses combine SAFE-T with our validated EHR-based machine learning models to determine the synergistic effects of clinician and statistical prediction. Implications for precision suicide risk prediction and prevention efforts will be discussed.