(PS12-60) An Artificial Intelligence Human Avatar Based Mental Health Assessment: A Validation Study in Healthcare Workers

Background:

Healthcare workers are experiencing emotional burnout, mental health distress, and higher rates of suicide following the Covid-19 pandemic. Healthcare workers experience multiple barriers to obtaining mental health assessments and treatment, such as stigma, licensure concerns, and confidentiality. There is evidence that some healthcare workers may prefer to speak with an avatar rather than a real human about mental health symptoms to mitigate these concerns. Thus, we developed an application called “BeCalm” to conduct psychological assessments using artificial intelligence and user interaction with a digital, human-like avatar. This assessment covers areas such as: occupational burnout; work quality of life, general quality of life; sleep; anxiety; loneliness; mood; and also gauges interest in treatment and suggests appropriate self-help education and professional resources. This study aimed to test the feasibility, accessibility, and preliminary validity of the BeCalm application.

Method:

A total of 38 healthcare workers completed the BeCalm assessment followed by a clinical assessment battery of evidence-based measures assessing the same areas of mental health covered in the BeCalm application. This battery included: the interview-based Mini International Neuropsychiatric Interview, Professional Quality of Life Scale; Maslach Burnout Inventory, Insomnia Severity Index, Patient Health Questionnaire-9, UCLA Loneliness Scale, and Beck Depression Inventory. Participants also completed a self-report feedback form about their experience with the app. The average age of the participants was 31.87 (SD = 11.28) and 84% were biologically female. All participants who completed the study were employed at a hospital, in any role, regardless of patient contact.

Results:

Participants spent an average of 22.25 minutes in the application (SD = 7.52), with a maximum duration of 39.52 minutes and a minimum duration of 11.47 minutes. Feeling a connection with the avatar was the most frequently endorsed positive aspect of BeCalm, followed by the resources, summary, and psychoeducation provided at the end of the assessment. Strong convergent validity was found across most symptom domains between the responses to the self-report measures and the BeCalm app responses, such as burnout (r=.766, p < .001) depression (r=.690, p < .001), anxiety (r=.454, p=.004), satisfaction with work (r=.390, p=.015), general quality of life (r=.431, p=.007), sleep (r=.572, p < .001), and loneliness (r=.408, p < .001). Strong validity was also found with the interview-based measure on depression (r=.410, p=.011) and anxiety (r=.335, p=.040).

Discussion:

The BeCalm application showed both strong feasibility and acceptability and strong convergent validity with well-validated assessments of multiple domains of mental health. This application offers an enjoyable, useful, and scalable method of assessing the mental health of healthcare workers and may promote insight, behavior change, and possible treatment engagement in a population experiencing high levels of burnout, emotional distress, and an underutilization of services.