The PACE trial of cognitive behaviour therapy and graded exercise therapy for chronic fatigue syndrome has been enormously destructive of the campaign for open, more trustworthy science in the UK. With clear conflicts of interest, the investigators of one of the largest psychotherapy trials ever switched scoring of some outcomes and suppressed other outcomes altogether and refused to share their data for independent re-analysis.
Perhaps the greatest damage was done in the ability of the investigators to gain support for the withholding of data with lame arguments.
The PACE investigators insisted that sharing data would violate their promising of confidentiality to patients consenting to enrollment in the trial. They insisted on their right to refuse to share data with those who had criticized them because of the risk of reputational damage posed by reanalysis. They even suggested that critics might tamper with the integrity of the data in order to ensure a contradiction of what the investigators had originally reported.
I am a critic who sought release of PACE trial data promised to be available as a condition for publishing in PLOS One, a journal they could have avoided if they did not wish to share data. Although the scene seems to be changing, I was taken aback by the lack of the support and even hostility from many whom I had assumed were quite sympathetic to routine data sharing. There was some kind of exceptionalism in play that I didn’t understand.
As it gets more likely that the data from the trial will be made available, it’s becoming more socially acceptable to reassert the necessity of routine data sharing and to dismiss implicitly the arguments made by the PACE investigators against sharing.
The big sea change will occur when the Cochrane Collaboration releases data from an individual participant level meta-analysis that currently has coordinated switching of outcomes consistent with what the PACE investigators did in their trial. The collaboration cannot maintain the appearance of supporting data sharing if they do not share data themselves.
But there are lots of other positive signs, including some clear statements by Vaughan Bell in a recent Lancet Psychiatry editorial. Here are three excerpts:
Vaughan Bell, , Open science in mental health research The Lancet Psychiatry
You can make your data available—open data. The intrinsic value of a dataset increases as it becomes more widely available. Nevertheless, making data public seems to make researchers nervous. It shouldn’t, because we typically declare when we publish that we will make anonymised data available to other researchers if they request it…
One common concern is that publishing data from studies will violate ethical approval or the terms of participant consent. It is good practice to mention the possibility of anonymous publication on study information but if data are sufficiently anonymised to protect against de-identification, specific consent is typically not required to publish them—including clinical data. I confirmed the above with the Health Research Authority in the UK, but check with your relevant ethics body. There are really very few barriers to publishing adequately anonymised data online for others to use, even in retrospect…
Related to this, is the concern about so-called hostile reanalysis. Science can be adversarial, and psychiatry is a controversial area. Some researchers fear not an honest re-evaluation of their work, but bad faith attempts to prove them wrong. Open science isn’t going to stop this but it helps protect against it. It makes everything open from the point of publication, making accusations of obscuring data and analysis methods fall flat, and motivated reanalyses mostly have the status of exploratory research if you have pre-registered your hypotheses.