Editorial from the
SEPTEMBER 28, 2017
“It was the criticism from within the scientific community that we had not anticipated.”
“This time the SMC also came under fire from our friends in science…Quack buster extraordinaire David Colquhoun tweeted, ‘More reasons to be concerned about @SMC_London?’
Other friends wrote to us expressing concern about the unintended consequences of SMC briefings – with one saying that policy makers were furious at having to deal with the fallout from our climate briefing and others worried that the briefing on the CFS/ME trial would allow the only private company offering the treatment to profit by over-egging preliminary findings.
Those of us who are accustomed to the Science Media Centre UK (SMC) highly slanted coverage of select topics can detect a familiar defensive, yet self-congratulatory tone to an editorial put out by the SMC in reaction to its broad coverage of Esther Crawley’s SMILE trial of the quack treatment, Phil Parker’s Lightning Process. Once again, critics, both patients and professionals, of ineffectual treatments being offered for chronic fatigue syndrome/myalgic encephalomyelitis are lumped with climate change deniers. Ho-hum, this comparison is getting so clichéd.
Perhaps even better, the SMC editorial’s concessions of poor coverage of the SMILE trial drew sharp amplifications from commentators that SMC had botched the job.
Here are some comments below, with emphases added. But let’s not be lulled by SMC into assuming that these intelligent, highly articulate comments, not necessarily from the professional community. I wouldn’t be surprised if hiding behind the pseudonyms are some of the excellent citizen scientists that the patient community has had to grow in the face of vilification and stigmatization led by SMC.
I actually think I recognize a spokesperson from the patient community writing under the pseudonym ‘Scary vocal critic.’
Scary vocal critic says:
September 29, 2017 at 5:59 am
The way that this blog glosses over important details in order to promote a simplistic narrative is just another illustration of why so many are concerned by Fiona Fox’s work, and the impact [of] the Science Media Centre.
Let’ s look in a bit more detail at the SMILE trial, from Esther Crawley at Bristol University. This trial was intended to assess the efficacy of Phil Parker’s Lightning Process©. Phil Parker has a history of outlandish medical claims about his ability to heal others, selling training in “the use of divination medicine cards and tarot as a way of making predictions” and providing a biography which claimed: “Phil Parker is already known to many as an inspirational teacher, therapist, healer and author. His personal healing journey began when, whilst working with his patients as an osteopath. He discovered that their bodies would suddenly tell him important bits of information about them and their past, which to his surprise turned out to be factually correct! He further developed this ability to step into other people’s bodies over the years to assist them in their healing with amazing results. After working as a healer for 20 years, Phil Parker has developed a powerful and magical program to help you unlock your natural healing abilities. If you feel drawn to these courses then you are probably ready to join.” https://web.archive.org/web/20070615014926/http://www.healinghawk.com/prospectushealing.htm
While much of the teaching materials for the Lightning Process are not available for public scrutiny (LP being copyrighted and controlled by Phil Parker), it sells itself as being founded on neurolinguistic programming and osteopathy, which are themselves forms of quackery. Those who have been on the course have described a combination of strange rituals, intensive positive affirmations, and pseudoscientific neuro-babble; all adding up to promote the view that an individual’s ill-health can be controlled if only they are sufficiently committed to the Lightning Programme. Bristol University appears to have embraced the neurobabble, and in their press release about the SMILE results they describe LP thus: “It is a three-day training programme run by registered practitioners and designed to teach individuals a new set of techniques for improving life and health, through consciously switching on health promoting neurological pathways.”
Unsurprisingly, many patients have complained about paying for LP and receiving manipulative quackery. This can have unpredictable consequences. This article reports a child attempting to kill themselves after going on the Lightning Process: Before conducting a trial, the researchers involved had a responsibility to examine the course and training materials and remove all pseudo-science, yet this was not done. Instead, those patient groups raising concerns about the trial were smeared, and presented as being opposed to science.
The SMILE trial was always an unethical use of research funding, but if it had followed its original protocol, it would have been less likely to generate misleading results and headlines. The Skeptics Dictionary’s page on the Lightning Process features a contribution which explains that: “the Lightning Process RCT being carried out by Esther Crawley changed its primary outcome measure from school attendance to scores on a self-report questionnaire. Given that LP involves making claims to patients about their own ability to control symptoms in exactly the sort of way likely to lead to response bias, it seems very likely that this trial will now find LP to be ‘effective’. One of the problems with EBM is that it is often difficult to reliably measure the outcomes that are important to patients and account for the biases that occur in non-blinded trials, allowing for exaggerated claims of efficacy to be made to patients.”
The SMILE trial was a nonblinded, A vs A+B design, testing a ‘treatment’ which included positive affirmations, and then used subjective self-report questionnaires as a primary outcome. This is not a sensible way of conducting a trial, as anyone who has looked at how junk-science can be used to promote quackery will be aware.
You can see the original protocol for the SMILE trial here (although this protocol refers to merely a feasibility study, this is the same research, with the same ethical review code, the feasibility study having seemingly been converted to a full trial a year into the research):
The protocol that: “The primary outcome measure for the interventions will be school attendance/home tuition at 6 months.” It is worth noting that the new SMILE paper reported that there was no significant difference between groups for what was the trial’s primary outcome. There was a significant difference at 12 months, but by this point data on school attendance was missing for one third of the participants of the LP arm. The SMC failed to inform journalists of this outcome switching, instead presenting Prof Crawley as a critic converted by a rigorous examination of the evidence, despite her having told the ethics review board in 2010 that “she has worked before with the Bath [LP] practitioner who is good”. https://meagenda.wordpress.com/2011/01/06/letter-issued-by-nres-following-scrutiny-of-complaints-in-relation-to-smile-lighting-process-pilot-study/
Also, while the original protocol, and a later analysis plan, refer to verifying self-reported school attendance with school records, I could see no mention of this in the final paper, so it may be that even this more objective outcome measure has been rendered less useful and more prone to problems with response bias.
Back to Fiona Fox’s blog: “If you had only read the headlines for the CFS/ME story you may conclude that the treatment tested at Bristol might be worth a try if you are blighted by the illness, when in truth the author said repeatedly that the findings would first have to be replicated in a bigger trial.”
How terrible of sloppy headline writers to misrepresent research findings. This is from the abstract of Esther Crawley’s paper: “Conclusion The LP is effective and is probably cost-effective when provided in addition to SMC for mild/moderately affected adolescents with CFS/ME.” http://adc.bmj.com/content/early/2017/09/20/archdischild-2017-313375
Fox complains of “vocal critics of research” in the CFS and climate change fields. There has been a prolong campaign from the SMC to smear those patients and academics who have been pointing out the problems with poor quality UK research into CFS, attempting to lump them with climate change deniers, anti-vaccinationists and animal rights extremists. The SMC used this campaign as an example of when they had “engineered the coverage” by “seizing the agenda”:
Despite dramatic claims of a fearsome group of dangerous extremists (“It’s safer to insult the Prophet Mohammed than to contradict the armed wing of the ME brigade”), a Freedom of Information request helped us gain some valuable information about exactly what behaviour most concerned victimised researchers such as Esther Crawley:
“Minutes from a 2013 meeting held at the Science Media Centre, an organisation that played an important role in promoting misleading claims about the PACE trial to the UK media, show these CFS researchers deciding that “harassment is most damaging in the form of vexatious FOIs [Freedom of Information requests]”.[13,16, 27-31] The other two examples of harassment provided were “complaints” and “House of Lords debates”. It is questionable whether such acts should be considered forms of harassment.
[A full copy of the minutes is included at the above address.]
Since then, a seriously ill patient managed to win a legal battle against researchers attempting to release key trial data, picking apart the prejudices that were promoted and left the Judge to state that “assessment of activist behaviour was, in our view, grossly exaggerated and the only actual evidence was that an individual at a seminar had heckled Professor Chalder.” http://www.informationtribunal.gov.uk/DBFiles/Decision/i1854/Queen%20Mary%20University%20of%20London%20EA-2015-0269%20(12-8-16).PDF
So why would there be an attempt to present request for information, complaints, and mere debate, as forms of harassment? Rather embarrassingly for Fiona and the SMC, it has since become clear. Following the release of (still only some of) the data from the £5 million PACE trial it is now increasingly recognised within the academic community that patients were right to be concerned about the quality of these researchers’ work, and the way in which people had been misled about the trial’s rsults. The New York Times reported on calls for the retraction of a key PACE paper (Robin Murray, the journal’s editor and a close friend of Simon Wessely’s, does not seem keen to discuss and debate the problems with this work): https://www.nytimes.com/2017/03/18/opinion/sunday/getting-it-wrong-on-chronic-fatigue-syndrome.html The Journal of Health Psychology has published as special issue devoted to the PACE trial debacle: http://journals.sagepub.com/doi/full/10.1177/1359105317722370 The CDC has dropped promotion of CBT and GET: https://www.statnews.com/2017/09/25/chronic-fatigue-syndrome-cdc/ And NICE has decided to a full review of its guidelines for CFS is necessary, citing concerns about research such as PACE as one of the key reasons for this: https://www.nice.org.uk/guidance/cg53/resources/surveillance-report-2017-chronic-fatigue-syndromemyalgic-encephalomyelitis-or-encephalopathy-diagnosis-and-management-2007-nice-guideline-cg53-4602203537/chapter/how-we-made-the-decision https://www.thetimes.co.uk/edition/news/mutiny-by-me-sufferers-forces-a-climbdown-on-exercise-treatment-npj0spq0w
The SMC’s response to this has not been impressive.
Fox writes: “Both briefings fitted the usual mould: top quality scientists explaining their work to smart science journalists and making technical and complex studies accessible to readers.”
I’d be interested to know how it was Fox decided that Crawley was a top quality scientist. Also, it is worrying that the culture of UK science journalism seems to assume that making technical and complex studies (like SMILE?!) accessible for readers is their highest goal. It is not a surprise that it is foreign journalists who have produced more careful and accurate coverage of the PACE trial scandal.
Unlike the SMC and some CFS researchers, I do not consider complaints or debate to be a form of harassment, and would be quite happy to respond to anyone who disagrees with the concerns I have laid out here. I have had to simplify things, but believe that I have not done so in a way which favours my case. It seems that there are few people willing to try to publicly defend the PACE trial anymore, and I have never seen anyone from the SMC attempt to respond to anything other than a straw-man representation of their critics. Lets see what response these inconvenient truths receive.
Michael Emmans-Dean says:
October 2, 2017 at 8:22 am
The only point I would add to this excellent post is to ask why on earth the SMC decided to feature such a small, poorly-designed trial as SMILE. The most likely explanation is that it was intended as a smokescreen for an inconvenient truth. NICE’s retrieval of their CFS guideline from the long grass (the “static list”) is a far bigger story and it was announced in the same week that SMILE was published.
Fiona Roberts says:
September 29, 2017 at 9:03 am