An Open Letter to David Tovey
April 13, 2017
Dear Dr. Tovey:
Almost one year ago to the date, we had our memorable cordial exchange in blog posts and direct communication. I am now again writing to you to alert you to a formal request for data to the collaboration that I recently made, but to which I have not received a reply.
In making my request, I could cite Cochrane’s widely circulated press release and data sharing
The Cochrane Collaboration urges free access to all data from all clinical trials: end to selective reporting can reduce the risk of harm to patients
Similarly, I could cite newly elected Cochrane board member Peter C Gøtzsche’s Why we need easy access to all data from all clinical trials and how to accomplish it
The arguments in favor of data sharing are so strong that it would seem difficult to argue convincingly that we should not share our data.
Dr. Gøtzsche has a sharp eye for the intrusions of industry into the conduct of clinical trials and the integration of trial data into systematic reviews and meta-analyses, when it is the pharmaceutical industry. But he and the rest of the Nordic Coordination Centre have a tin ear when it comes to detecting authors tied to the insurance industry tromping into a Cochrane review and imposing a coordinated switching of outcomes.
I am refering a recent Cochrane review for which there has been a long and fruitless exchange of comments:
Larun L, Brurberg KG, Odgaard-Jensen J, Price JR. Exercise therapy for chronic fatigue syndrome. Cochrane Database Syst. Rev. 2016 Jan 1;2.
This review was an update of a 2004 review. The largest proportion of trials (2/3) in key analyses comes from trials for which outcomes for protocol-specified primary outcomes were switched. Scoring of subjective self-report outcomes were switched after information became available that the original scoring would not yield significant effects. Objective outcomes that would have been particularly informative in light of the switched scoring of subjective primary outcomes were simply suppressed.
The net result was a shift in the conclusions of the meta-analyses so they were favorable to the interests of the insurance industry with which one of primary investigators of a trial had a financial conflict of interest. The trial itself was funded in part by the UK Department of Works and Pension.
My extraordinary claims are backed up by comments that are now incorporated in the Cochrane review.
For the pitfalls of such intrusion of author and sponsor bias, I refer you to an article by Hilda Bastian, a founding member of the Cochrane Collaboration and its Consumers and Communication Review Group.
Bastian H. ‘They would say that, wouldn’t they?’A reader’s guide to author and sponsor biases in clinical research. Journal of the Royal Society of Medicine. 2006 Dec 1;99(12):611-4.
Despite conflicts of interest, the investigators from these original trials are now authors on a forthcoming Cochrane review in which switched outcomes will prove decisive. The protocol is
Larun L, Odgaard‐Jensen J, Brurberg KG, Chalder T, Dybwad M, Moss‐Morris RE, Sharpe M, Wallman K, Wearden A, White PD, Glasziou PP. Exercise therapy for chronic fatigue syndrome (individual patient data). The Cochrane Library. 2014
The protocol itself is vague, loosely written, and underspecified, giving the authors considerable degrees of freedom in making decisions about which data to include; how the data will be coded and lumped and split; and in and how the analyses will be interpreted.
I refer you to an excellent recent letter in Nature: Genetics, Celebrating parasites -provides descriptions the accomplishments of past and current winners of the annual research parasite awards.
Under some proposals for data reuse, data would be shared with researchers working in concert with the investigators who initially analyzed the data1. We expect that this would counteract the recent focus of the US National Institutes of Health (NIH) on rigor, transparency and reproducibility. Any procedure that includes data generators as gatekeepers has the potential to compromise rigor and robustness. As gatekeepers, researchers could withhold data from those with contrary views or a reputation of challenging the status quo. We must expect data sharing to lead to some conclusions being challenged and, ultimately, refuted. If this fails to occur, it indicates a problem with the process and not the correctness of conclusions.
And my blog post based in part on it:
As you are well aware, investigators in this area of research have tenaciously clung to their data, resisting all requests except when they have been promised authorship or when compelled by loss of repeated appeals. Their behavior is inconsistent with the spirit and the letter of statements issued by the Cochrane collaboration and its members, as well as the opinions of rest of the international research community.
Policymakers, clinicians, and patients have every reason to be skeptical of the results reported in this forthcoming systematic review and meta-analysis.
On behalf of these stakeholders, I am formally requesting release of the individual participant level data from the systematic review and meta-analysis for independent reanalysis. My reanalyses will be forensic, and will include exploratory sensitivity analyses using the scoring of measures originally specified in original trial protocols. I of course will make the results of these analyses available for public scrutiny in the from of a preprint, solicit comments, and then submit the revised paper for peer review.
Consistent with the record of public statements from Cochrane, I should be able to expect a timely release of the data for re-analyses. However, given the record of these particular investigators, I could also anticipate a lengthy struggle ending up, if necessary, in the UK Informational Tribunal. I should hope that will not be the case.
I look forward to hearing from you,
My formal request to Cochrane
I am writing with regard to the protocol Exercise therapy for chronic fatigue syndrome (individual patient data) DOI: 10.1002/14651858.CD011040
I understand that the actual review will soon be published. I am writing to request the data so that I can reproduce basic analyses and perform sensitivity analyses with respect to the outcomes that were originally specified for the trials included in the protocols, but later switched.
It may seem premature for this request, but note that release of only a portion of the data for one of the trials, The Lancet PACE trial took five years after a request.
Similarly, I requested data from that trial that were promised to be available as a condition for publishing in PLOS One. Over 18 months later, I am still waiting.
I sincerely hope that you will simply make the data available. However, my request for the PLOS One PACE data was turned into a FOIA request by the investigators and rejected. In anticipation that might be the case for the Cochrane Review, I thought we could start early in the request-rejection-appeal process.
I write in hope that my reasonable request can be accommodated without instigating such a drawn-out process.
I look forward to hearing from you.
Thanks in advance.