An open letter to the Cochrane Collaboration: Bill Silverman lies a-moldering in his grave

To whom it may concern:

cochraneI’m writing to ask you to consider the implications of having authors conduct a systematic review with the Cochrane collaboration brand attached who have ties to an industry which would benefit from particular conclusions. These same conclusions would personally enrich some of the authors professionally and personally.

Larun L, Odgaard-Jensen J, Brurberg KG, Chalder T, Dybwad M, Moss-Morris RE, Sharpe M, Wallman K, Wearden A, White PD, Glasziou PP. Exercise therapy for chronic fatigue syndrome (individual patient data) (Protocol).  Cochrane Database of Systematic Reviews 2014, Issue 4. Art. No.: CD011040. http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD011040/abstract

The Cochrane Collaboration allowing this authorship for the systematic review is egregious enough. However your acceptance of this configuration has a role in a monumental battle over the sharing of data promised as a condition for publication in PLOS One. In a reply to my request, they state my request for the data is vexatious and motivated by the intent to damage their credibility as professionals and the conclusions of their PLOS One paper. They cite their involvement in the Cochrane review as evidence of their willingness to share patient-level data with more reasonable, collaborative people. They do not say so explicitly, but they imply their data will be shared with those willing to commit to protecting the credibility of their conclusions.

So, we find ourselves on opposite sides in the battle for data sharing. I think you are either not maintaining proper oversight of your operations or simply being embarrassingly hypocritical in your position.

I write to you as one of the authors that you honored with the Bill Silverman award for a paper in The BMJ,

Roseman Michelle, Turner Erick H, Lexchin Joel, Coyne James C, Bero Lisa A, Thombs Brett D et al. Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study BMJ 2012; 345 :e5155 [http://www.bmj.com/content/345/bmj.e5155.full ]

bill cochraneI might be tempted to give you back your US$1000 except my esteemed co-authors and I gave the money entirely to our graduate student first author. It would be unfair to punish her for your hypocritical behavior. Yet, from what I know of Bill Silverman, he would heartily approve of my giving you the money back, if it were mine to give.

Because this is an open letter that will be read by an international audience, I will draw upon your description of who Bill Silverman was and what he represented to the collaboration, i.e., a troublemaker.

 William (Bill) Silverman (1924-2004) was one of the founders of American neonatal medicine. He was honoured repeatedly as one of the pioneers in his specialty; however, he often evoked somewhat contradictory responses amongst his colleagues because he was in the habit of raising troubling questions about the scientific basis and ethics of his and their practices. Like many of the people who have helped to establish Cochrane, Bill Silverman could be regarded as a ‘troublemaker’. As he reiterated frequently, however, criticism is a form of troublemaking that can help to drive progress. Furthermore, criticism should not be limited to examining the work of others, but should also include self-criticism.

The prize

…Acknowledges explicitly Cochrane’s value of criticism, with a view to helping to improve its work, and thus achieve its aim of helping people make well-informed decisions about health care by providing the best possible evidence on the effects of healthcare interventions. The Cochrane Steering Group approved the establishment of the Prize in 2007, and it was awarded for the first time in 2008.

The study that earned us the award examined

 Systematic reviews of drug interventions published in 2010 in the Cochrane Database of Systematic Reviews.

We found

Only 16 of the 151 Cochrane reviews (11%, 7% to 17%) provided any information on trial author-industry financial ties or trial author-industry employment. Information on trial funding and trial author-industry ties was reported in one to seven locations within each review, with no consistent reporting location observed

We concluded

Most Cochrane reviews of drug trials published in 2010 did not provide information on trial funding sources or trial author-industry financial ties or employment. When this information was reported, location of reporting was inconsistent across reviews.

Among other things, we recommended

That the Cochrane Collaboration reconsider its position that trial funding and trial author-industry financial ties not be included in the risk of bias assessment. The 2008 version of the Cochrane handbook listed “inappropriate influence of funders” (section 8.14.1.6) (for example, data owned by industry sponsor) as a potential source of bias that review authors could optionally incorporate in the “other sources of bias” domain of the Cochrane risk of bias tool.37 The 2011 version of the handbook, however, argues that “vested interests” should not be included in the risk of bias assessment, which “should be used to assess specific aspects of methodology that might be been influenced by vested interests and which may lead directly to a risk of bias” (section 8.15.1.5).38 As previously noted,22 empirical criteria are generally used to select items (for example, sequence generation, blinding) that are included in assessments of risk of bias,38 48 including evidence of a mechanism, direction, and likely magnitude of bias. Empirical data show that trial funding by pharmaceutical companies and trial author-industry financial ties are associated with a bias towards positive results even when controlling for other study characteristics6 8 49 50 and, thus, meet these criteria. One concern might be that including conflicts of interest from included trials in the risk of bias assessment could result in “double counting” of potential sources of bias. However, ratings in the risk of bias table are not summed to a single score, and inclusion of risk of bias from conflicts of interest could reflect mechanisms through which industry involvement can influence study outcomes6 that are not fully captured by the current domains of the risk of bias tool (random sequence generation, allocation concealment, blinding of participants and staff, blinding of outcome assessment, incomplete outcome data, selective reporting, and other sources of bias). Furthermore, even if all relevant mechanisms were to be assessed, the degree of their influence may not be fully captured when reviewers only have access to the relatively brief descriptions of trial methods that are provided in most published reports. Inclusion of conflicts of interest from included trials in the risk of bias assessment would encourage a transparent assessment of whether industry funded trials and independently conducted trials reach similar conclusions. It would also make it explicit when an entire area of research has been funded by industry and would benefit from outside scrutiny.

Consistent with my past experience criticizing the collaboration, you graciously received our criticism. And in the case of The BMJ article, you did something different as a result, you modified your Risk of Bias Assessment to take trial authors’ conflict of interest into account.

In the name of Bill Silverman, I ask you to take this review away from the current authors. You had the opportunity to know Dr.Silverman better than I do. What would he do?

I await your response.

Sincerely,

scanned signature-page-0-1

Jim Coyne