Why I don’t know how PLOS will respond to authors’ refusal to release data

The simple answer is that I am not involved in the decision-making process and it is a bit opaque to me.

The more elaborate answer:

  • plos oneThe strategies I used to obtain the PACE data from the PLOS paper are only what are available to anyone else.
  • Based on what I know, I do not expect to appeal King’s College, London’s decision through the Information Commissioner’s Office  because my original request was not made under the Freedom of Information Act. So, I particularly depend on PLOS’ response.
  • I am one of thousands of PLOS One Academic Editors, as well as a blogger at the PLOS blog Mind the Brain. Yet, this does not give me any special standing, any special influence, or even any special understanding of the process by which PLOS decides how to respond to the refusal to release the data.
  • My reporting of all that is transpiring serves to highlight the importance of data sharing and policies to make sure that authors do indeed share data when requested.
  • I am also modeling how to proceed, at least with the PLOS journals when authors do not share data. The PLOS family of journals is years ahead of most other publishers in terms of their explicitness and transparency. I invite others to try what I’m doing with other journals and report the results.
  • We may have to settle for these accomplishments, if the data are not forthcoming, but I am more optimistic than that.

On November 13, 2015, I made a formal request to Paul McCrone, the first author of a PLOS article  to provide the data needed for re-analyses and sensitivity analyses  to see if results held up with different assumptions. I got a quick reply from that my request was being considered under the Freedom of Information Act.

I cc’ed PLOS One in my request. I promptly received an acknowledgment from a staff member. My request was assigned a case number.

On December 11 2015, I received a letter indicating that King’s College declined my request for the data.

I had previously called for release of data from the PACE paper published in The Lancet.  I also have called for statements concerning the availability of clinical trial data to be incorporated into the consent forms soliciting patient involvement in research , as an extension of a BMJ patient-oriented initiative.

But the request to PLOS was different than other request for the PACE data because it involved a journal that required a commitment to data sharing as a condition for publication.

King's college London

King’s College, London

The reasons given for King’s College’s refusal to release the data makes fascinating reading.

The King’s College letter deserves a read. Personally, I think they brought disaster down upon themselves by rejecting my request with the excuse that:

 

 

There is a lack of value or serious purpose to your request. The university also considers that there is improper motive behind the request.

The letter goes on to identify the motive and purpose behind this request as polemical and then described the request as “vexatious”.

Since then there has been lots of outraged expressed from a broadening group of academic and citizen scientists, with vigorous debate of the specifics of the letter. It’s been quite gratifying that calls for the release of data are coming from outside the UK and outside the patient community. There is no doubt that the international scientific community feels there are some key issues for them at stake.

The powerful Retraction Watch highlighted my blog in its Weekend Reads:

King’s College London doesn’t want to release data to James Coyne from a study of chronic fatigue syndrome. See if the absurd reasons make your blood boil as much as ours: “The university considers that there is a lack of value or serious purpose to your request. The university also considers that there is improper motive behind the request. The university considers that this request has caused and could further cause harassment and distress to staff.” And here’s more from someone else who submitted a related request data.

And that was followed by tweets by a number of champions of open science and data sharing.

 

brian nosek

JP de

if hiding data bothers you

lakensAlready, the chain of events has drawn tremendous attention to the value of data sharing policies, but the need to have means of enforcement. Previous to the PACE investigators’ stubborn refusal to share their data, apparently a lot of scientists felt that the data sharing problem had been solved by requirements of governmental and foundation funding bodies.

When I’m asked how PLOS will respond

I am frequently asked how PLOS journals will respond. I have the benefit of having read their exquisitely documented policies requiring the sharing of data for all articles published in PLOS One, PLOS Medicine, and other fine open access journals. I can reply to queries with what I have read.

But honestly, I don’t know how PLOS will respond because I am not involved in the decision-making process. Like anyone else, I can go on the Internet to discover the rules of the game for asking for data and for how to complain when when the data are not received. I thus have an advantage over those who didn’t know there were such options or how to exercise them. But I’m in uncharted waters.

But right now, like everyone else, I’m waiting to see how PLOS chooses to respond. I don’t know of another precedent in which authors refuse to provide data for a PLOS article, so I wouldn’t be surprised if a policy remains to be formulated. I have no doubt, however, that the issue will be carefully considered by PLOS and the principles behind a decision-making will be well articulated in a future policy statement. We need to be patient.

Disclosure

I happen to be a PLOS One Academic Editor, but there are thousands of us, certainly over 4000. You have to appreciate the hugeness of PLOS: last year PLOS one published 30,000 papers or 85 papers a day, making it the world’s largest journal, in terms of output.

I visit universities around the world giving talks. I inevitably try to recruit new PLOS One academic editors, particularly for subject areas where there has not previously been established expertise and yet where I want papers to be published after informative peer review. I encourage faculty, when they are asked to do a review for PLOS for which they do not have time, to supervise an advanced PhD student or postdoc doing a review as an educational experience.

A year ago I was in Hong Kong giving talks and providing evaluations of PhD student presentations, with the prices being all expense paid trips to international conferences. I was surprised to discover that one of the other judges was a senior staff member of PLOS based in London. That is the only time that I have met a senior editor. I know the names of only a few of them and certainly could not pick them out of a police lineup.

PLOS has regular events at conferences where anyone can meet a senior editor. But have never had the pleasure of being at one of those conferences at the same time as a PLOS representative.

Pascali's_Island_film_posterMuch of the time my editing of PLOS feels like the Ben Kingsley character in the 1988 movie Pascali’s Island. Kingsley files detailed reports with unfailing regularity about what he observes at Nisi, a small Greek island under Ottoman rule, but does not get a response. At least I am confident that someone at PLOS reads my editorial recommendations (I only recommend, I do not decide), but there is a remoteness to the whole process when I have never seen the people on the other end.

I am optimistic about the crisis of the King’s College refusing to release data ultimately being resolved. The release of the data will allow not only me, but others to provide independent evaluations of how reasonable the authors’ interpretations made in their article are.

What has already been accomplished

I judge my efforts as already being tremendously successful in highlighting the issues of data sharing policies and finding ways to ensure there is not such a hassle when the next request is made.

Many people are now reading the PLOS statements about data sharing, although I suspect that many people voicing strong opinions in the discussion have not read these sources. They should read them and demand that the policies be adopted elsewhere.

Prior to requesting the data from the PLOS article, I’ve been involved in a number of reanalyses of published data. One set of re-analyses  revealed that a loving-kindness meditation intervention did not have a significant effect on physical health outcomes, despite already being heavily marketed in workshops and corporate presentations. The original article had not even acknowledge that it was a randomized trial until the supplementary materials. Another set of reanalyses showed that a complicated set of 53 multiple regression equations claim to demonstrate that distinct dimensions of psychological well-being are differentially correlated with levels of expression of a selection of genes, which in turn, are  associated with distinct forms of immune response produced equivalent results when random numbers were substituted for the data from the study.

Personally, I believe that such reanalyses are just as important, perhaps more so than single-study declarations of breakthroughs that often prove premature or simply wrong. I am committed to promoting such re-analyses as a suitable contribution for junior academics building their careers.

An Expression of Concern and a retraction as possible penalties if the data are not forthcoming

I was particularly delighted when Groningen biologist Klaas van Dijk– who is not actually at my University and whom I have never met – provided an excellent letter calling for an Expression of Concern from PLOS that I have not received my data . In the letter he calls for the retraction of the article if the situation is not rectified within a month. It’s a model response, which I’m sure will have opportunities to use at other journals in the future.

I was also delighted at the very intelligent comments left at the PLOS article and even that the opportunity to make comments as been highlighted. I appreciate in particular a well articulated comment left by Alem Matthees who is the publicity-shunning source of an earlier request for PACE data from a Lancet article. The request was rejected by Queen Mary University of London. Matthees appealed to the UK information Commissioner’s Office. The Commissioner then ordered release of the data, but that decision was also appealed by Queen Mary University of London.

The widely acknowledged untrustworthiness of the scientific literature is harder to rectify if such a ridiculous bureaucratic process in order to access data to reanalyzed.

And then there are the comments at PubMed Commons. Anyone can leave comments there if they can claim to have been an author on any one of the 25 million articles listed in Pub Med, and register with either their email in that article or their Google account. And if you have never published a paper listed in Pubmed, you can still comment through a recognized PubMed Commons journal club.

I posted the letter by which I requested the data in a blog and I hope it will be used as a model by others. However, the basic structure of the letter did not originate with me, I borrowed a model that my graduate student at Groningen, Nick Brown had used in a request that did not involve me at a journal other than PLOS.

Maybe eventually I will assemble a toolbox for obtaining data and appealing when the data are not forthcoming . Certainly the letters by Klaas and Nick will be part of that an appropriate credit will be given.

My blogging

Besides being an PLOS One Academic Editor, I am a regular blogger at the PLOS blog, Mind the Brain. You can read my self-description at the blog site.

But I should make perfectly clear that any opinions I express at PLOS Mind the Brain are entirely my own and should not be construed as the official policy of PLOS or even anything with which the management of PLOS would agree.

For instance, I recently ended a blog post with the statement

The views expressed in this post represent solely those of its author, and not necessarily those of PLOS or PLOS Medicine.

And I previously provided at my secondary blog, Quick Thoughts much the same message that I am conveying in this blog post :

Note: Although I am one of thousands of academic editors for PLOS One, I have sole responsibility for my blog posts, which cannot be construed as reflecting the opinions of any institutions with which I am affiliated, including PLOS. In requesting the data for an article that appeared in PLOS, I am exercising the same rights to anyone in the world has with respect to this open access journal committed to transparency and data sharing.

I will keep repeating this message to avoid confusion. I characteristically express strong, irreverent opinions, but they are strictly my own.

A troublemaker certified by the Cochrane collaboration

My colleagues and I questioned how the Cochrane collaboration incorporated investigator conflict of interest as a formal risk of bias in assessment of papers for inclusion in systematic reviews and meta-analyses. We recommended better recognition be given to what could be a source of exaggerated claims . The Cochrane collaboration responded by giving us the Bill Silverman award which is named after a recognized troublemaker in the early days of their collaboration.

I suppose taking up authors on their commitment to sharing their data can be seen as just another form of troublemaking. I’m sure my request is received with at least some ambivalence by the administrative staff of PLOS, with the likely necessity of formulating and disseminating yet another policy. If so, it’s not the first time I have challenged the family of journals with the result a change in policy. I see myself as an ethical hacker  of sorts: exposing faults in a system so that new solutions can be developed. But I really want to reanalyze the PACE PLOS data.