Skip to main content
ABCT 2024 58th Annual Convention Main banner
Psychology Tools Ad
Final Program


Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Presentation Icons
Recorded Session
Recorded Session
Ticketed Session
Ticketed Session
Back
ViewAttendees 6

Symposia

Technology/Digital Health

The promise of technological advancements in innovating various aspects of healthcare delivery

2 - (SYM 134) A Generative AI Application to Assess and Support Fidelity to Cognitive Processing Therapy

Friday, November 15, 2024
8:30 AM – 10:00 AM EST
Location: Congress B, Fourth Floor, Loews Hotel
Abstract Body

Research has demonstrated a link between fidelity to cognitive processing therapy and PTSD treatment outcomes. However, evidence suggests that fidelity may decrease over time without ongoing support. Assessing fidelity by reviewing session recordings requires substantial time and training and is not feasible in most clinical settings. Advances in natural language processing (NLP) led to reliable automated coding of some treatment modalities, which paved the way for more scalable fidelity assessment. However, this process requires an intensive process of training NLP models on hundreds if not thousands of fidelity-coded therapy sessions, and patients are often unaware that their session recordings are used to train models that may ultimately have commercial applications. With the advent of generative artificial intelligence applications (also known as Large Language Models—LLMs) such as GPT-4, it has become possible to train applications to code sessions through prompt engineering. In this presentation, we describe the process of training and validating fidelity assessment using LLMs in an ongoing study of asynchronous messaging-based CPT for PTSD. Eighty transcripts were coded by an LLM and two modes of comparison were used to assess model reliability. Preliminary findings demonstrate good reliability and agreement with human coding for treatment adherence. We will also describe ongoing work to assess competence and to provide support for improved treatment fidelity and additional generative AI approaches to supporting therapist fidelity. Best practices and implications for implementation of evidence-based treatments in routine care settings will also be discussed.