Written by Developing risk models in healthcare is challenging – implementing them effectively more so. In this randomized clinical trial, we test two designs of a system prompting physicians to consider screening for suicide prevention driven by an automated model. We show two types of designs – “interruptive” which prompted physicians to pause and respond to a notification and “non-interruptive” which placed a small icon on the screen that could either be acted upon or ignored. Interruptive CDS was 3x more effective leading to documented screening than in the year prior to the trial.
There is much more to do including testing systems such as these at larger scale and refining them to prompt action beyond screening in general. These types of trials are critical to separate the hype around AI from their usefulness in clinical practice.
Thank you to Mark Kelly and Good Morning Nashville for your coverage of this study – we’re grateful to have had the opportunity to share this work with a wider Tennessee audience and to showcase suicide prevention efforts ongoing at Vanderbilt University Medical Center.
See below for the abstract, and check out the full paper here.
Key Points
Question In settings without universal suicide risk screening, is interruptive clinical decision support (CDS) with an on-screen pop-up more effective than non-interruptive CDS in prompting in-person risk assessment at the point of care for patients predicted by a statistical model to be at high risk of a suicide attempt?
Findings In this randomized clinical trial of 561 participants with 596 clinician encounters, interruptive CDS was significantly more effective at prompting in-person assessment than non-interruptive CDS and more effective compared with baseline documented screening rates the prior year.
Meaning These results suggest that well-powered large-scale trials randomizing interruptive CDS compared with standard of care are indicated to measure effectiveness in reducing suicidal thoughts and behaviors in the context of alert burden and capacity constraints.
Abstract
Importance Suicide prevention requires risk identification, intervention, and follow-up. Traditional risk identification relies on patient self-reporting, support network reporting, or face-to-face screening. Statistical risk models have been studied and some have been deployed to augment clinical judgment. Few have been tested in clinical practice via clinical decision support (CDS). Barriers to effective CDS include potential alert burden for a stigmatized clinical problem and lack of data on how best to integrate scalable risk models into clinical workflows.
Objective To evaluate the effectiveness of risk model–driven CDS on suicide risk assessment.
Design, Setting, and Participants This comparative effectiveness randomized clinical trial was performed from August 17, 2022, to February 16, 2023, in the Department of Neurology across the divisions of Neuro-Movement Disorders, Neuromuscular Disorders, and Behavioral and Cognitive Neurology at Vanderbilt University Medical Center, an academic medical center in the US Mid-South. Patients scheduled for routine care in those settings were randomized at visit check-in. Follow-up was completed March 16, 2023, and data were analyzed from April 11 to July 24, 2023. Analyses were based on intention to treat.
Interventions Interruptive vs noninterruptive CDS to prompt further suicide risk assessment using a real-time, validated statistical suicide attempt risk model. In the interruptive CDS, an alert window via on-screen pop-up and a patient panel icon were visible simultaneously. Dismissing the alert hid it with no effect on the patient panel icon. The noninterruptive CDS showed the patient panel icon without the pop-up alert. When present, the noninterruptive CDS displayed “elevated suicide risk score” in the patient summarization panel. Hovering over this icon resulted in a pop-up identical to the interruptive CDS.
Main Outcomes and Measures The main outcome was the decision to assess risk in person. Secondary outcomes included rates of suicidal ideation and attempts in both treatment arms and baseline rates of documented screening during the prior year. Manual medical record review of every trial encounter was used to determine whether suicide risk assessment was subsequently documented.
Results A total of 561 patients with 596 encounters were randomized to interruptive or noninterruptive CDS in a 1:1 ratio (mean [SD] age, 59.3 [16.5] years; 292 [52%] women). Adjusting for clinician cluster effects, interruptive CDS led to significantly higher numbers of decisions to screen (121 of 289 encounters [42%]) compared with noninterruptive CDS (12 of 307 encounters [4%]) (odds ratio, 17.70; 95% CI, 6.42-48.79; P < .001) and compared with the baseline rate the prior year (64 of 832 encounters [8%]). No documented episodes of suicidal ideation or attempts occurred in either arm.
Conclusions and Relevance In this randomized clinical trial of interruptive and non-interruptive CDS to prompt face-to-face suicide risk assessment, interruptive CDS led to higher numbers of decisions to screen with documented suicide risk assessments. Well-powered large-scale trials randomizing this type of CDS compared with standard of care are indicated to measure effectiveness in reducing suicidal self-harm.
Walsh Science 