A cranky elite fighting the inevitable: Tired old arguments in NEJM against data sharing

 Inspired by a tweet from Vinay Prasad

vinay prasad on data sharing

Are controversies about data sharing a matter of an established elite with expertise not possessed elsewhere versus an unwashed “democratic” group who couldn’t understand how to use the data if they obtained them?

Are those calling for routine data sharing motivated to misrepresent what the elite experts have already produced?

I briefly provide an annotation to a recent essay in the New England Journal of Medicine. The essay is an unsubtle attempt to caricature those who call for data sharing and to undermine the legitimacy of their position. At points, those who call for data sharing are demonized.

research parasitesOf course, we are all just an infestation of research parasites.

The essay starts with a false division of the range of views about data sharing and caricaturing of those who call for data sharing.

The essay presents unsubstantiated fantasies about the requirement that data be shared limiting the number of trials that are conducted. The incentives for conducting clinical trials remain in place and data sharing can make for more efficient use of data from past trials to inform future trials.

There is an annoying tendency to present those who call for data sharing as an ill-informed bunch who will probably put the data to misuse.

On the other hand, those who withhold data are seen as an elite, with the expertise that can’t be expected outside of their ranks

There is no sense of the crisis in trustworthiness of biomedical data driving calls for data sharing, no recognition of the conflicts of interest with which original investigators selectively report and interpret their data.

Bridging the Data-Sharing Divide — Seeing the Devil in the Details, Not the Other Camp Lisa Rosenbaum, M.D.

The movement toward sharing data from clinical trials has divided the scientific community, and the battle lines were evident at a recent summit sponsored by the Journal. On one side stand many clinical trialists, whose lifeblood — randomized, controlled trials (RCTs) — may be threatened by data sharing. On the other side stand data scientists — many of them hailing from the genetics community, whose sharing of data markedly accelerated progress in that field.

We should beware of beginning discussions with the false divisions of the range of views and those who hold them. A lot of the people supporting data sharing conduct clinical trials themselves. Some of the people seeking data from existing trials are keenly aware of the misinterpretation that the original investigators have offered.

I strongly support data sharing but I don’t recognize the so-called “data scientists” in which I am being lumped. I could not pick these “data scientists” out of a police lineup.

In a vast range of scientific research, sharing data would accelerate progress, including in the conduct and interpretation of clinical trials.

At a time when RCT funding is shrinking, trialists know that sharing data adds substantial costs to clinical trial execution; a requirement to share data might mean that fewer trials, and smaller ones, will be conducted.

Application for funding for RCTs can include provisions for the cost of sharing data. In the United States, NIH requires provisions for sharing data as part of the application.

citation neededThere is no evidence that a requirement to share data would mean fewer trials. This kind of assertion needs documentation.

Availability of data from past trials could be analyzed so that future researchers could avoid issues of feasibility and acceptability the trialists could’ve been anticipated if data from past trials were carefully examined.

Many trialists also worry that complex data will be misinterpreted by people who weren’t involved in generating them, and who may therefore produce misleading results.

Aha, the implicit position is that only the people who re-analyze data who are at risk of misinterpreting trial data, not the original investigators. There is strong evidence of an untrustworthiness to the existing published clinical trial data, in part driven by the questionable research and reporting practices of clinical trialists.

Any independent re-analyses could be subject to the same peer review processes that original researchers receive.

Many clinical trials have direct implications for human welfare, clinical practice, and public policy. These implications are too important to be left to be defined by the original trialists.

Furthermore, journal publications are the currency of academic advancement. Researchers often invest 5 to 10 years gathering trial data, expecting to write several papers after their primary publication. An expectation that data will be shared quickly may therefore create a disincentive for conducting RCTs.

This is a straw man argument. No one is saying the clinical trialists should not be given an opportunity to publish their findings. But once their findings are published, they should be subject to independent reanalysis.

Perhaps the most incisive question, posed by Rory Collins, a University of Oxford epidemiologist and trialist, was the most obvious one: What problem are we trying to solve? The advancement of science depends on the open exchange of ideas and the opportunity to replicate or refute others’ findings. But will data sharing address our current system’s shortcomings in a way that advances science? For example, though it’s troubling when trials provide incomplete information about adverse events, requiring the sharing of individual patient data from every trial might not be the best way to fix that problem.

A more effective solution, Collins suggests, may be publishing, alongside the primary trial results, an easily accessible appendix containing adverse-event data in tabular form. Proponents of data sharing also believe it will allow other investigators to generate new insights and hypotheses. But will such insights advance health in a way that justifies the cost?

The cost to whom? The untrustworthiness of the existing literature requires independent re-evaluation of data that is not controlled by the investigators.

A greater risk may be to the clinical trialist community. It’s assumed that data sharing will advance the public health, but will the public benefit if there are steep declines in the number and size of clinical trials? Though more “open” science may yield as-yet-unimagined innovations, unplanned and retrospective secondary analyses can only generate, not test, hypotheses. The type of hypothesis testing that can advance treatment of disease will always depend on active and motivated clinical trialists asking questions prospectively.

While the recent summit was civil and collaborative, the tenor of the broader data-sharing conversation has framed the matter as one of trialist self-interest versus public good. But such a frame vastly oversimplifies the situation — and tends to entrench people in polarized positions, articulated with righteous indignation.

This essay starts by creating artificially polarized positions and attempts to direct indignation towards those who call for routine data sharing.

The indignation of data-sharing advocates arises in part from the claim that the absence of data sharing slows the development of cures. In addition, at a political moment when promises of data democratization overshadow faith in traditional expertise, reservations about data sharing are easily dismissed as elitist — as are the experts who point out misunderstandings of a topic they’ve spent years studying.

So the author is pitting the promises of democratization versus faith in traditional expertise?

Ah ah, here we see the clinical trialists been characterized as possessing traditional expertise lacking among those who call for data sharing. Essentially the argument is being made that no one but the original investigators can understand what the original investigators were studying.

The value placed on transparency also contributes: any resistance to greater openness is branded as secrecy and deceit. Finally, the deepest (and perhaps most valid) source of moral outrage may be the sentiment that clinical trial data aren’t ours to begin with, that they should belong to the patients who put themselves at risk to participate. And in principle, patients want their data shared.

Not all resistance to data sharing is rooted in secrecy and deceit, but there are some well-documented examples of trialists who have switched outcomes and engaged in other questionable research practices spending huge amounts of money to avoid independent reanalysis. Sometimes trialists have something to hide and cite ‘threat of reputational damage’ in refusing to share data.

But patients also want better treatments for their diseases. And though data sharing may sometimes lead to better treatments, it may also divert limited resources to types of research that are less fruitful than RCTs, impeding the evidence generation required for improving care.

This is a repetition of earlier nonsense arguments without any further substantiation.

The irony in the framing of this debate is that to share data in a way that advances knowledge, we must be open to one another’s experience and expertise, setting aside ideology in pursuit of more objective truths. Fulfilling this obligation, as we refine the scientific process, will require not only sharing what we find but also resisting the temptation to demonize those who see different paths to our shared goal.

Yup, the  irony…

I blog at a number of sites, PLOS Mind the Brain, Quick Thoughts, and occasionally, Science-based Medicine. To receive alerts about all my blog posts, just sign up at CoyneoftheRealm.com. You’ll get advance notice of forthcoming e-books and science writing courses as well.