A recent comparative effectiveness study in JAMA Psychiatry of CBT versus psychodynamic psychotherapy for depression was billed as a noninferiority trial.
One sentence in the results section changed the whole significance of the study.
The dodo bird verdict for the study is that everybody gets a booby prize.
The study is currently freely accessed at JAMA Psychiatry, although you may need to register for free to actually download the PDF.
Connolly Gibbons M, Gallop R, Thompson D, et al. Comparative Effectiveness of Cognitive Therapy and Dynamic Psychotherapy for Major Depressive Disorder in a Community Mental Health Setting: A Randomized Clinical Noninferiority Trial. JAMA Psychiatry. Published online August 03, 2016. doi:10.1001/jamapsychiatry.2016.1720.
The moderately sized study compared to active treatments without a nonspecific comparison/control group.
Results. Among the 237 patients (59 men [24.9%]; 178 women [75.1%]; mean [SD] age, 36.2 [12.1] years) treated by 20 therapists (19 women and 1 man; mean [SD] age, 40.0 [14.6] years), 118 were randomized to DT and 119 to CT. A mean (SD) difference between treatments was found in the change on the Hamilton Rating Scale for Depression of 0.86 (7.73) scale points (95% CI, −0.70 to 2.42; Cohen d, 0.11), indicating that DT was statistically not inferior to CT. A statistically significant main effect was found for time (F1,198 = 75.92; P = .001). No statistically significant differences were found between treatments on patient ratings of treatment credibility. Dynamic psychotherapy and CT were discriminated from each other on competence in supportive techniques (t120 = 2.48; P = .02), competence in expressive techniques (t120 = 4.78; P = .001), adherence to CT techniques (t115 = −7.07; P = .001), and competence in CT (t115 = −7.07; P = .001).
Conclusions and Relevance. This study suggests that DT is not inferior to CT on change in depression for the treatment of MDD in a community mental health setting. The 95% CI suggests that the effects of DT are equivalent to those of CT.
In case there is any ambiguity in the message the authors wanted to convey, they reiterated:
Question Is short-term dynamic psychotherapy not inferior to cognitive therapy in the treatment of major depressive disorder (MDD) in the community mental health setting?
Findings In this randomized noninferiority trial that included 237 adults, short-term dynamic psychotherapy was statistically significantly noninferior to cognitive therapy in decreasing depressive symptoms among patients receiving services for MDD in the community mental health setting.
Meaning Short-term dynamic psychotherapy and cognitive therapy may be effective in treating MDD in the community.
I examined the 40 tweets available on August 6, 2016 and found only one that went beyond parroting.
I I suspect that Robert Howard had discovered the one sentence in the results section that I noticed:
Nineteen patients (16.1%) in DT and 26 patients (21.8%) in the CT condition demonstrated response to treatment as measured by a 50% reduction on the HAM-D score across treatment (χ21 = 1.27; P = .32).
Most of the patients assigned to either group in this study failed to respond to treatment. Tipped off by this sentence, I looked for the degree of treatment exposure and found that most patients did not get exposed to sufficient intensity of treatment.
Sixty-three patients (26.6%) attended 1 or fewer sessions of psychotherapy; 122 (51.5%), 5 or fewer sessions; and 187 (78.9%), 11 or fewer sessions. We found no statistically significant difference between treatments in the number of sessions attended (t235 = 1.47; P = .14).
The title of the JAMA Psychiatry article noted that patients had been recruited from the community mental health center. I interpret this to suggest they were likely to be a low income group who were not previously prepared for psychotherapy.
Before anyone proposes that the solution is simply to offer more therapy, note that the patients were not attending enough sessions of a larger number (16) that were offered. My interpretation is that greater effort may be needed to get such patients to consistently show up for sessions.
My colleagues and I previously conducted an exceptionally well resourced study in in the same low income and socially disadvantaged Philadelphia population. Our intention was to reduce risk factors among recently pregnant, low income women for another low weight birth delivery. We demonstrated that we could recruit and retain these women, but it took an intensive, creative effort.
One of the risk factors that we addressed was depression and we offered antidepressant medication and free treatment at the world-renowned University of Pennsylvania Center For Cognitive Therapy. We provided free transportation and child care. Few women access sufficient therapy or receive sufficient dose of antidepressants. The therapists at the center complained that the women did not seem to have their life in order and did not seem ready for psychotherapy. Personally, I think that the therapist may not have been ready for such women and did not sufficiently engage them.
Back to the study under discussion, it was accompanied by an editorial that parroted the authors’ intended message in its title:
Abbass AA, Town JM. Bona Fide Psychotherapy Models Are Equally Effective for Major Depressive Disorder: Future Research Directions. JAMA Psychiatry. Published online August 03, 2016. doi:10.1001/jamapsychiatry.2016.1916.
But I noticed this in the text:
EFFECTIVENESS OF PSYCHOTHERAPY FOR MDD: GOOD BUT COULD BE BETTER?
Among other points, the study by Connolly Gibbons and colleagues raises the ongoing challenge facing all psychiatrists using pharmacotherapy and psychotherapy: how to improve rates of remission in real-world clinical samples. The study found that more than 80% of all participants did not respond to treatment (22% of patients receiving CBT and 16% of patients receiving STPP had response to treatment as measured by a 50% reduction in observer-rated depression). This high rate of nonresponse may be partly explained by inadequate treatment “dose” or number of sessions, clinical sample, therapist expertise, biomedical factors, and sociofamilial factors impeding outcomes
The JAMA Psychiatry article under discussion cited another, similar study conducted in the Netherlands, but did not elaborate on its findings:
Driessen E, Van HL, Don FJ, Peen J, Kool S, Westra D, Hendriksen M, Schoevers RA, Cuijpers P, Twisk JW, Dekker JJ. The efficacy of cognitive-behavioral therapy and psychodynamic therapy in the outpatient treatment of major depression: a randomized clinical trial. American Journal of Psychiatry. 2013 Sep 1.
Unlike the JAMA Psychiatry article, the abstract of the Dutch study qualified its finding of non-inferiority by noting that nether therapy did particularly well:
No statistically significant treatment differences were found for any of the outcome measures. The average posttreatment remission rate was 22.7%. Noninferiority was shown for posttreatment HAM-D and patient-rated depression scores but could not be demonstrated for posttreatment remission rates or any of the follow-up measures.
The findings extend the evidence base of psychodynamic therapy for depression but also indicate that time-limited treatment is insufficient for a substantial number of patients encountered in psychiatric outpatient clinics.
I suspect that both of these randomized trials will be cited as evidence of the Dodo Bird Verdict for psychotherapy for depression – everybody’s a winner and everybody gets a prize. However, in both the studies, the cognitive behavior therapy underperformed relative to the efficacy demonstrated in a larger body of studies. The literature for psychodynamic therapy is more limited and of low quality.
Still, I think the messages that when you move into more difficult populations, you can’t expect results obtained with more carefully selected, therapy-ready patient populations who were recruited to more typical studies. But this may reflect on the unrepresentativeness of patients in the larger literature.
Meanwhile, Psychiatrist Erick Turner and I have been having an exchange on Twitter concerning another noninferiority study.
Erick is referring to a perspective he shares with things I’ve been saying regularly about noninferiority trials. They typically don’t include a nonspecific comparison/control group. Without such a group, we can’t evaluate whether either of the active treatments are better than provision of nonspecific treatments with elements of support, positive expectation, and attention.
That is also a limitation of the current study, but by peeking into the actual results, we discover referral to neither of two active treatments left most patients free of depression.
What if there had been a credible attention/support condition in the present study? Would either of these two treatments that were “noninferior” to each other have a clinically significant advantage? What would be the implications, if not? would the report have made it into JAMA Psychiatry?