(PS12-87) The Treatment Effect Prediction Model for Cognitive Behavioral Therapy for Panic Disorder: Improving the Coefficient of Determination Using a Machine Learning Approach

Background: Cognitive-behavioral therapy (CBT) is empirically supported treatment for panic disorder (PD). The authors presented a predictive model for the effects of CBT for PD based on multiple regression analysis in previous study (Ogawa et al. 2018). The coefficient of determination remains at around 0.5 at most in the study, and further improvement of the prediction model is required. Machine learning is a cutting-edge and appropriate method that has been applied extensively in many domains to analyze complex data. However there hasn't been much testing of the machine learning approach in CBT research.

Objective: The present study aimed to create a treatment effect prediction model for CBT for PD using a new approach, machine learning.

Methods: The data used in Ogawa et al. (2018) mentioned above (collected through group CBT for 200 patients with PD) was reanalyzed via linear regression using a machine learning approach. The predicted outcome is the ten items (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, paranoid ideation, psychoticism, and global severity index.) of Symptom Checklist-90 Revised. The baseline characteristics are the five factors (neuroticism, extraversion, conscientiousness, openness, and agreeableness) of NEO Five Factor Index, the baseline value of each item of Symptom Checklist-90 Revised, age, age of onset, and gender. Furthermore, we attempted to construct a more accurate model through feature engineering. The study’s protocol was approved by the Ethics Committee of Nagoya City University Graduate School of Humanities and Social Sciences and written consent was obtained from the patients.

Results: The reanalysis using linear regression based on machine learning showed an increase in the coefficient of determination for nine out of ten items (somatization, obsessive-compulsive, interpersonal sensitivity, depression, anxiety, hostility, phobic anxiety, psychoticism, and global severity index). Furthermore, in the reanalysis using feature engineering, the coefficient of determination for each item further increased, with seven out of ten items reaching 0.50 or higher. In particular, the coefficient of determination for the somatization subscale increased from 0.49 (Ogawa et al. 2018) to 0.70.

Discussion: In general, a coefficient of determination of 0.50 or higher is considered good, and the machine learning algorithm, including feature engineering, was able to improve the accuracy of the treatment effect prediction model for CBT for PD. Clinically, it is suggested that psychiatric symptoms following CBT for PD may be predicted by entering each patient's baseline data, NEO Five Factor Index, age, gender, and other data into our prediction model.