A skeptical look at a study of acupuncture delivered in emergency rooms [updated]

Why an expensive, flawed, misrepresented trial of acupuncture in emergency rooms provided no evidence that acupuncture is effective in controlling pain.

acupuncture.needlesUpdate 6/24/2017

I  thank the pseudonymous commentator who called attention to one of the authors of this study previously providing a press release making claims for the study’s results before the data were analysed   The author stated:

“While data from the study is still being analysed and finalised for publication in a medical journal, one of the researchers, Dr Michael Ben-Meir, said it showed acupuncture offered the same level of pain relief as analgesic drugs when patients rated their pain one hour after treatment.”

We can now evaluate this misrepresentation.

A number people in social media urged me to take a quick look at a recent large, expensive trial of providing acupuncture in the emergency room. The study has begun getting some attention in the media. We will see the study is flawed from its inception, aimed at fulfilling the principal investigator’s agenda being of misrepresented as supportive of getting acupuncture into ERs. Coverage in the media is inaccurate, but consistent with statements made by the primary investigator.

The paper reporting the clinical trial is unfortunately in a pay-walled journal, but it is at least temporarily it is available here.  An abstract can be found here

Marc M Cohen, De Villiers Smit, Nick Andrianopoulos, Michael Ben-Meir, David McD Taylor, Shefton J Parker, Chalie C Xue and Peter A Cameron Acupuncture for analgesia in the emergency department: a multicentre, randomised, equivalence and non-inferiority trial. Medical Journal of Australia. 2017; 206 (11): 494-499. || doi: 10.5694/mja16.00771

The trial protocol is freely available and can be found here.

  • My summary assessment is that it is indeed an ambitious expensive clinical trial producing no evidence that acupuncture is effective.
  • Its rationale is based on misrepresentations of the literature.
  • The study protocol shows features are built in its design to be biased in favor of finding acupuncture is not inferior to routine care in the ER.
  • The article downplays important findings that undercut validity of trial and investigator’s intended conclusions.
  • The study was conducted by an investigator strongly committed to the promotion of acupuncture in routine medical care and securing reimbursement for it.
  • The conclusion is slanted toward cultivating the illusion that acupuncture is an evidence-based treatment. The intent is to obtain funding for an even more ambitious trial, ultimately getting acupuncture secure reimbursement and preferred treatment status.
  • The study capitalizes on the current opioid addiction epidemic and public health concerns about overreliance on pain medications.

Rationale for the randomized trial

The article and protocol seriously misrepresent the conclusions of the existing literature. Claims that are made are inconsistent with was is found in the specific citations that are provided.

The article states

Acupuncture can provide analgesia for chronic musculoskeletal pain and headache,6-9 and its use is covered by therapeutic guidelines for acute pain.10,11

And

Our study aimed to determine whether acupuncture is effective, safe, acceptable and feasible for patients presenting to the ED with low back pain, migraine or acute ankle injuries.

The protocol states

It has been shown that acupuncture analgesia in the treatment of chronic pain is comparable to morphine and that its better safety profile and lack of dependence makes it the preferred method of choice for these conditions [9].

Reference 9 in both sources is to

Ernst E, Lee MS, Choi TY. Acupuncture: does it alleviate pain and are there serious risks? A review of reviews. PAIN®. 2011 Apr 30;152(4):755-64.

This article is highly critical of the use of acupuncture and certainly does not say its effectiveness is comparable to morphine and should be the preferred method of treating chronic pain. Instead, it says:

In conclusion, numerous systematic reviews have generated little truly convincing evidence that acupuncture is effective in reducing pain. Serious adverse effects continue to be reported.

The protocol proclaims the pressing need for a study of the scope of the present one:

As of June 2011, virtually all of the 31 reviews and 32 review protocols listed in the Cochrane library suggest that the evidence for acupuncture is inconclusive. Consequently, there has been a call for the funding of well-planned, large-scale studies to assess the effectiveness and cost-effectiveness of acupuncture under real-life conditions [14]. It has been suggested that the best methodological approach to clinical acupuncture research is through pragmatic trials where acupuncture is compared to standard care rather than placebo-controlled trials. Such trials may be more able to provide data that is relevant to patients, practitioners and policymakers, and to inform decision-making about treatment options [15].

The issues about acupuncture are (a) the lack of a scientific rationale and (b) difficulty demonstrating it is superior to a sham acupuncture treatment. The difficulties in (b) is that it is difficult to blind a “real” versus sham treatment and the tendency of acupuncturists to communicate to patients whether they are receiving only a sham treatment.

There is a lack of evidence that acupuncture is anything but a highly ritualized placebo treatment. The investigator takes this as a cue for abandoning placebo-controlled studies and just comparing it with routine care- which is often lacking in the support and attention provided by a nonspecific placebo condition.

The investigator is intent on demonstrating that acupuncture is no worse- noninferior -to routine care.

Design features that  bias in favor of finding acupuncture is not inferior to routine care in the ER.

The emergency room (ER) is a bad place to get medical care. Don’t get your medical care unless required by circumstances.

The ER is also a bad place to evaluate acupuncture. There are reasons not to provide acupuncture in the ER even if one believes in the efficacy of acupuncture. ER care is urgent care intended to address medical issues needing immediate attention. For many medical conditions, emergency room care is only the first step in admitting a patient to inpatient services or to refer a patient for specialty outpatient services.

The conditions that are lumped together are too diverse, especially if the interest is the evaluation of the treatment of pain.

An ankle sprain is an acute injury. The article notes

All ankle injury patients received rest, ice, compression and elevation regardless of their treatment allocation.

Patients presenting to the ER with migraines are typically experiencing the exacerbation of a pre-existing chronic condition. Other symptoms besides pain may prompt a visit. The article notes that “migraine patients received intravenous fluids at the discretion of the treating physician.” The pharmacotherapy protocol included treatment for nausea, but no pain medication specific to migraines.

Patients presenting with acute low back pain may be injured and facing the onset of a chronic condition or facing an exacerbation of a chronic condition. Addressing the immediate circumstances of the ER visit is a goal, not analgesia. Long term goals are to preserve functioning and quality of life, not analgesia.

These three groups are different in their conditions and procedures, goals,  and expected outcomes for an ER visit. Even a trial as large as this one would be too small -underpowered- to explore crucial differences between groups. Or to see if overall results do not generalize to any patient group.

For all three patient groups, some will have self-limiting pain, which will contribute to a lack of differences among treatments.

There is no evidence for the efficacy of acupuncture versus a nonspecific treatment for any of these treatments. I recently discussed claims for acupuncture for migraines. I showed a lack of evidence for its efficacy. It is important to note, however, that in that study, acupuncture groups received treatment 5 days per week for 4 weeks for a total of 20 sessions. Even believers in acupuncture would not expect one session delivered in the ER would prove sufficient. But benign and even inert effects of one session might encourage patients to get more treatments later.

The primary outcomes of the study were assessed only at 24 and 48 hours. In those short periods, differences between effective and noneffective treatments may not have emerged. Differences in nonspecific treatments –attention, support, and communication of positive expectations may be maximized, but short-lived. And the subjective distress of some patients may have gotten dropped from the heights of distress which brought them to the ER.

Downplaying findings that undercut validity of trial and investigator’s intended conclusions

 Regardless of group assignment, all patients were entitled to “rescue medication” after 1 hour or if their attending physician deemed it appropriate. Patients assigned to acupuncture alone were three times more likely to receive rescue medication. This is a serious confound undermining the integrity of the trial and the meaningfulness of any results.

The acupuncture only group received significantly more rescue medication therapy than the groups that received pharmacotherapy (at T1: p = 0.016; after T1: P = 0.008).

Only a minority of patients in any condition achieved adequate pain relief.

Overall, 16% of patients had clinically relevant pain relief and 36.9% statistically relevant pain relief at T1, with no statistically significant differences between the three groups (Box 6).

Conducted by an investigator strongly committed to the promotion of acupuncture in routine medical care and securing reimbursement for it

 In Australia, the principal investigator is a sought after spokesperson advocating acupuncture and other unproven medical treatments.

Statements made in interviews with the media   show the motivation of the PI in conducting the trial and his spinning of his results:

“Emergency nurses and doctors need a variety of pain-relieving options when treating patients, given the concerns around opioids such as morphine, which carry the risk of addiction when used long-term.

And

“Our study has shown acupuncture is a viable alternative, and would be especially beneficial for patients who are unable to take standard pain-relieving drugs because of other medical conditions.

And

“But it’s clear we need more research overall to develop better medical approaches to pain management, as the study also showed patients initially remained in some pain, no matter what treatment they received.”

A sampling of the inaccurate coverage that this study received:

press coverage

Postscript

I enjoy probing suspicious studies like this one. I hope that readers benefit from what I find, not only in terms of my specific conclusions, but in acquiring the knowledge and skills I deploy. I draw on over 40 years experience as a researcher and a teacher, with over 400 peer-reviewed publications, as well as lots of editorial and peer review experience. In the case of this particular study of acupuncture, likely problems could be anticipated, as well as where to look for them.

Although I’ve becomes relatively efficient at this sort of activity, it remains time and labor intensive, particularly when I trace the study back into the claims being made about it in the media, as well as to any study protocol. And then look for discrepancies with what else is out there. And them format all of this for a blog post.

There is no direct benefit, professionally or financially to me for this activity. It’s an enjoyable, almost addictive distraction. I really should be working on the web-based science writing courses and e-books that will ultimately provide support for my activities, now that I am emeritus. I wonder, though, if there might be a way to monetize this. I would be uncomfortable simply doing it fee-for-service, because of the potential conflict of interest that it would create. I don’t come to a negative conclusion for everything I evaluate, but I wouldn’t want to be seen has endorsing weak or preliminary findings.

I’m considering creating a broad-based premium service. For a modest fee, subscribers would get weekly assessments of what I find in the scientific literature and how it is covered in social media. Subscribers could not only obtain this regular premium content, but also have the opportunity to nominate studies for review, with a subset selected and reported to all subscribers.

I know that my fans and followers have become accustomed to my providing a blog post and a social media presence without charge. I intensified this activity when I had a Carnegie Centenary Visiting Professorship at Stirling University, Scotland. One of my responsibilities was social engagement, and my involvement in blogging and social provided an effective and satisfying way of doing that. But now the professorship is over, and I have to think of alternative ways of monetizing my activities. I welcome readers’ thoughts and encourage them to sign up at CoyneoftheRealm.com to keep up on all of my activities.