1 - (SYM 54) Socrates 2.0: A Novel Generative Artificial Intelligence Tool to Facilitate Socratic Dialogue

Cognitive behavioral therapies are highly effective for treating a wide range of mental health concerns, including posttraumatic stress disorder (PTSD). Cognitive restructuring, a core mechanism of these therapies is often facilitated via Socratic dialogue and out-of-session skills practice using worksheets. Despite the demonstrated effectiveness of these mechanisms, adherence to out-of-session practice remains low. A potential reason may be the static and non-interactive nature of traditional worksheets. Large language models (LLMs) offer a solution for the development of more natural, engaging digital tools. In this present, we detail the development of Socrates 2.0, which was designed to complement cognitive behavioral therapy and engage users in Socratic dialogue surrounding unrealistic or unhelpful beliefs.

The initial version of the AI tool, Socrates 1.0, was a zero-shot, single-agent model designed to facilitate Socratic dialogue. LLM prompts were engineered in an iterative manner by mental health experts and AI engineers. Despite achieving foundational dialogue capabilities, it faced challenges such as verbosity, task forgetfulness, and looping, ultimately impacting the natural flow and progression of therapeutic conversation.

To overcome some of these issues, we developed Socrates 2.0, a multi-AI agent system comprising an AI therapist, AI supervisor, and AI external rater. The AI supervisor mirrors the 'bug-in-ear' real-time feedback of human supervisors, enhancing dialogue focus and AI therapist adaptability. The AI external rater was developed to evaluate the belief strength, aiding in dialogue progression and timely session conclusions. The addition of multiple AI agents dramatically reduced previous limitations, such infinite dialogue loops.

Socrates 2.0 underwent thorough testing across 500+ scenarios. It only engaged in inappropriate dialogues in under 1% of cases, with AI supervision promptly correcting issues. The tool's feasibility was supported by positive feedback from 6 therapists and a Community Advisory Board consisting of 6 individuals with lived experience of PTSD. Initial feedback focused on Socrates 2.0’s 24/7 accessibility and the ability to have high quality anonymous interactions anonymity. Both groups favored Socrates 2.0 over conventional worksheets and suggested future enhancements, including voice capabilities. Our presentation will conclude with a discussion of multi-AI agent models and their potential for enhancing therapeutic practices.