2 - (SYM 109) Engagement Metrics That Matter: Associations with Clinical and Wellness Outcomes Among Users of an Nlp-informed Relational Agent

Mental health problems continue to affect millions of Americans despite the availability of evidence-based interventions. The shortage of U.S. mental health providers is one reason that prevents access to those in need. Delivering evidence-based care via natural language processing (NLP)-informed relational agents may help address this barrier. Woebot for Mood and Anxiety (W-MA)1 is a mental health intervention guided by the relational agent, Woebot. Prior studies have found W-MA users to have the ability to form a bond with Woebot and to experience significant improvements in both mental health and wellness outcomes. Another study clustered users by different behavioral metrics (e.g., number of times opening the app, messages exchanged each week) and affective (e.g., working alliance Goal, Task, and Bond) and cognitive (enactment) measures of engagement. Of the three clusters of users identified, the “Efficient Engagers” had the lowest levels of behavioral engagement but higher Goal, Task, and enactment than users in other clusters (p< 0.05). “Efficient Engagers” had significantly greater reductions in both Patient Health Questionnaire-8 item (PHQ-8)-measured depressive symptoms (p=0.01) and Perceived Stress Scale (PSS)-measured stress (p=0.01) than “Typical Utilizers” at the end-of-intervention assessment and also had different patterns of age, gender identity, race/ethnicity, sexual orientation, education levels, and insurance coverage than other users (each comparison p< 0.05). Additional analyses that explored direct associations between the different engagement metrics used to cluster study participants with clinical and wellness outcomes revealed that cognitive and affective measures of engagement might be more strongly correlated with outcomes than those strictly behavioral in nature. These findings have important implications for what engagement metrics might matter “most” and how needs might differ based on user characteristics. 

DISCLAIMER

1Woebot for Mood and Anxiety (W-MA-02) is an investigational medical device. It has not been evaluated, cleared, or approved by the FDA.