Why we just don’t do an RCT to prove that electroconvulsive therapy works.

Over the past couple of days, there’s been a lively discussion on Twitter of the evidence for electroconvulsive therapy. Twitter is not a good place to discuss complex issues, but I take it has a good place to prompt more fruitful discussions elsewhere. Presumably these discussions can be referred back to Twitter.

In the near future, I’ll have some blog posts about electroconvulsive therapy. But for now, I simply want to address the question that keeps coming up, “Why don’t we simply do an RCT to prove that electroconvulsive therapy works?”

  • After providing some personal background, I’ll argue that is very difficult to conduct a randomized clinical trial of ECT. And the vague question “why don’t we simply…?” is exceedingly naïve in terms of the resources that would be required, the difficulty accumulating them, and the likelihood that such a trial would not be ethical.
  • Doesn’t evidence-based medicine required randomized trials as the gold standard? In many circumstances, RCTs would be preferred, but not badly done, uninformative RCTs.
  • I don’t think that everyone in the Twitter discussion is there simply to exchange ideas. I have had to block some angry trolls who arrived mysteriously with little identification and few followers, apparently intent on simply disrupting the discussion. But beyond that small group, I think there’s a lot of emotion involved in discussions of a valuable but highly stigmatized treatment, electroconvulsive therapy.

My perspective on randomized controlled trials of ECT is shaped by experience in a number of settings.

1.Working on a specialized depression unit where ECT was a treatment option. As an Associate (then full) Professor of Psychology in Psychiatry at University of Michigan Medical Center, I was assigned to a specialized combined inpatient/outpatient unit for severely depressed patients. The unit had been started by Bernard “Barney” Carroll. He had already left the unit when I arrived, but had a bigger than life presence. What I saw what was occurring there, he earned my admiration for life.

The inpatients were deemed ‘treatment refractory” or “treatment resistant.” I learned quickly that these were not diagnostic categories appropriately applied to people, but simply often misleading labels applied to patients with particular  histories treatment that had been unsatisfactory.

Inpatients on this unit arrived on multiple medications and clinicians had to work to taper these medications while avoiding the dangers of suddenly stopping them.

In careful N =1 experiments, medication was reintroduced after a few weeks of the patients remaining medication free. In some cases, dramatic improvement was observed, often with use of the stigmatized MAOI class of medications. These medications can be extremely effective, but are avoided by clinicians because of fears about adverse effects when combined with common foods and drinks like smoked meats, certain cheeses, fava beans, or Chianti.

Some patients still did not respond, and ECT was considered as a treatment option. Protocols were in place and enforced. These protocols required careful specification of why this treatment was sought and whether informed consent had been received. It was recognized ECT was experimental treatment and should be delivered with appropriate discussion and informed consent.

Why would treatment be delivered that was not backed by evidence randomized trials? Because an individual patients, effects could be immediately recognized. Yet, collectively, these patients were quite heterogeneous and any credible randomized trial would have to be very large for randomization to overcome the heterogeneity or be stratified, with many cells difficult to fill.

I’m publicly challenging some of the people that are been in the discussion on Twitter to identify what treatment options they would propose to vulnerable, chronically depressed patients who probably could not function outside of inpatient settings.

I note that some of the participants in the Twitter discussion continue to advocate cognitive behavior therapy for psychosis as a front-line treatment, replacing medication. I believe that patients should be protected from these persons and their misinformation.

2. Involvement in a committee evaluating clinical trials recruiting advanced cancer patients. After leaving the University of Michigan, I joined an oversight committee at the Abramson Family Cancer Center of the University Pennsylvania. We were not an alternative committee for the protection of human subjects or Institutional Review Board (IRB) required for all research, but a supplementary committee developed to ensure that vulnerable cancer patients were not being unnecessarily burdened with requests for informed consent to treatments. We reviewed carefully what was being asked of the patients who participated in particular trials. We also examined that if the trial had no likely benefit to their personal patient outcomes, how it was justified and presented to them. We grappled a lot with a recognition that patients often participate in trials with therapeutic illusions. They believe the cancer center would not ask them to be in the treatment trial unless it would improve their outcomes. That is an true of all trials, with other trials having to be justified on the contribution the patient’s results would conceivably inform other patients treatment in the future.

3. Involvement on a national committee shutting down clinical trials that were not accruing sufficient numbers of patients. I also served on a committee in the National Cancer Institute Community Clinical Oncology Program (CCOP) that was basically shutting down trials that did not accrue sufficient patients. It was an unsettling experience. Many radiation oncology trials had been going on for years without accruing sufficient numbers of patients for meaningful results. Clinicians were understandably reluctant to commit their patients with advanced cancer to trials in which treatment assignment was randomized.

Yet, if studies do not accrue enough patients for an adequately powered, well designed trial, we failed patients enrolling them in a study that cannot produce meaningful results. In some case in these radiation therapy trials were evaluating treatments that were already discarded. It would’ve been futile to keep them going. In other cases, the accrual was so slow that reaching the prespecified number of patients was unlikely in the future. Such trials should have been ended to protect any more patients from participating in a trial that was not going to make a meaningful contribution to science and to release scarce admnistrative resources for use elsewhere.

4. Involvement in efforts to identify alternatives to the randomized controlled trial. I was impressed that there were a lot of areas of medicine where common practices were not based on results of randomized trials. With colleagues I reviewed the relevant literature about alternatives to randomized trial. Interestingly, some of this literature had been developed with respect to orthodontics. Many orthodontic procedures are not dictated by medical necessity, but by aesthetics and patient preferences. Orthodontists are particularly unwilling to allow their patients to be approached for a randomized trial in which clinicians do not control treatment assignment. They have strong beliefs about which approaches are effective, even in the absence of data, but they also worry about losing patients to clinicians who protect their patients from research with randomized assignment.

The article that we produced:*

TenHave TR, Coyne J, Salzer M, Katz I. Research to improve the quality of care for depression: alternatives to the simple randomized clinical trial. General Hospital Psychiatry. 2003 Apr 30;25(2):115-23.

Some basic principles of evidence-based treatment

3 legged stoolEvidence-based medicine does not “prove” that treatments “work”, it generates evidence that compares treatments to alternatives in terms of benefits and risks.

“Evidence” is only one leg of a three-legged stool, the other legs being patient preference and clinical experience. This statement is often abused to avoid introduction of best evidence into discussions, but the intention is to recognize that some clinical judgment and patient involvement is necessary in translating introducing “evidence” into clinical situations.

The highest form of evidence is a well conducted systematic review and meta-analysis that integrates findings of studies, with the expectation that the limitations of individual studies will be overcome.

Many, perhaps most systematic reviews are limited in their usefulness because of the quality of studies being integrated and the failure of other systematic reviews to be conducted and reported consistent with best practices.

Randomized controlled trials with appropriate comparison/control groups are the best designs to compare and contrast treatments.

Randomized controlled trials have some ethical requirements that must be met:

  • There has to be some sense of equipoise, with neither investigator nor patients believing they are being asked to accept randomization to an inferior treatment than patients otherwise would receive.
  • Patients must be informed of the burden imposed by consenting to enroll in a clinical trial.

The burden of a clinical trial can include:

Needing to show up for multiple assessments, including long after treatment is completed.

Making clinic visits just to meet the requirements of the trial

Side effects and adverse events associated with assessments or being assigned to a treatment that the patient would not have preferred.

Adding more demands to a  life already disrupted by necessary medical care.

  • Patients must not be denied treatment to which they would otherwise be entitled in asking them to freely consent to participation in a trial.
  • Patients must not be penalized for changing their mind after consenting to be in a trial.
  • Patients should not be invited to participate in trials there are unlikely to accrue sufficient numbers of participants to make a meaningful contribution to medicine and science. Given the burden and drawbacks of participating in a clinical trial, patients consenting to do so believing there is  benefit to other patients and to medicine and science. That benefit is not realized if trials are not completed in published in peer reviewed journals.
  • Patients need to be protected from investigators not involved in their clinical care approaching them during that clinical care
  • Appropriately, many clinicians will be reluctant to have their patients being approached, particularly when the patients are vulnerable and desperate.

Further resources

An excellent, but brief discussion of informed consent and responsibilities of investigators and clinicians in ensuring that consent from patients is truly informed.

An excellent but brief discussion of problems in patients not understanding equipoise or what’s involved consenting to participate in a randomized trial:

Lay Public’s Understanding of Equipoise and Randomization in Randomized Controlled Trials (RCTs)


Recognition of gaps between evidence gained from mental health research and clinical practice in the community together with changes in treatment patterns and patient/provider preferences for care have led to interest in enhancements in the designs and analyses of clinical and community trials of mental health interventions. Gaps between clinical trials and community care include differences in populations and treatment strategies. To bridge these gaps, we propose enhancing the simple randomized trial with several different designs with the immediate aims of improving patient recruitment and adherence in psychiatric intervention studies thus bringing study designs more in line with clinical practice. The goals are to estimate treatment efficacy and effectiveness so that both internal and external validity are optimized. In this discussion, we address design and analytic issues with respect to a number of enhancements of the randomized trial design, including partial patient-provider preference designs, randomized encouragement and consent designs, fixed adaptive design, and random between- and within-patient adaptive designs. Each has advantages and disadvantages depending on the effect under investigation. Some of these enhancements, such as the fixed adaptive design, have begun to be implemented in effectiveness trials in mental health services research, but all are worthy of more attention.

I blog at a number of sites, PLOS blog Mind the Brain, Quick Thoughts, and occasionally, Science-based Medicine. To receive alerts about all my blog posts, just sign up at CoyneoftheRealm.com. You’ll get advance notice of forthcoming e-books and web-based science writing courses as well.