Most positive findings in psychology false or exaggerated? An activist’s perspective

Abstract of a  talk to be given at the Australian National University (ANU), room G08, Building  39, 3pm September  11, 2014.

UPDATE (October 24, 2014): Here is a link to the Powerpoint presentation and to a YouTube interview with ANU Research School of Psychology’s Professor Mike Smithson. A list of relevant links to James Coyne’s blogs has also been added at the end of this post.

confirmation biaIn 2005, John Ioannidis made the controversial assertion in a now famous PLoS Medicine paper that “Most Published Research Findings are False”. The paper demonstrated that many positive findings in biomedicine subsequently proved to be false, and that most discoveries are either not replicated or can be shown to be exaggerated. The relevance of these demonstrations was not appreciated until later in psychology.

Recent documented examples of outright fraud in the psychological literature have spurred skepticism. However, while outright fraud may be rare, confirmatory bias and flexible rules of design and analysis are rampant and even implicitly encouraged by journals seeking newsworthy articles. Efforts at reform have met with considerable resistance, as seen in the blowback against the replicability movement.

This talk will describe the work of one loosely affiliated group to advance reform by focusing attention not only on the quality of the existing literature, but on the social and political processes at the level of editing and reviewing. It will give specific examples of recent and ongoing efforts to dilute the absolute authority of editors and prepublication reviewers, and instead enforce transparency and greater reliance on post-publication peer review of claims and data.

Optional suggested readings (I suggest only one or two as background)

Is evidence-based medicine as bad as bad Pharma?

I am holding my revised manuscript hostage until the editor forwards my complaint to a rogue reviewer.

Reanalysis: No health benefits found for pursuing meaning in life versus pleasure.

A formal request for retraction of a cancer article

UPDATED Blog posts coordinated with the slide presentation relevant to Are most positive findings in psychology false or exaggerated? An activist’s perspective.

Exposing Meta Analyses with an agenda: American anti-abortionist gets flawed meta Analysis into British Journal of Psychiatry

Negative Mental Health Effects of Abortion: Does getting an abortion damage women’s mental health?  October 24, 2011

More on Review Claiming Abortion Hurts Women’s Mental Health: Author had conflict of interest, used scare tactics. November 15, 2011

Editor Should Have Caught Bias and Flaws in Review of Mental Health Effects of Abortion: Was publication of a flawed anti-abortion review deliberate? December 5, 2011

Is Having an Abortion Likely to Damage a Woman’s Mental Health?: Evaluating the Evidence from a Controversial Review. November 6, 2011

Going after bad science that misleads cancer patients

A formal request for retraction of a Cancer article May 12, 2014

Doubts a Classic Lancet Study Showed Psychotherapy Improves Survival of Cancer Patients. eptember 2, 2013

Frightening Breast Cancer Patients with Bad Science September 28, 2012

Pseudoscience of positive psychology and health

Reanalysis: No health benefits found for pursuing meaning in life versus pleasure. August 25, 2014

Improving your health by pursuing meaning in your life versus happiness. August 5, 2014

Challenging unfair editorial practices

Whomp! Using invited editorial commentary to neutralize negative findings. November 13, 2013.

I am holding my revised manuscript hostage until the editor forwards my complaint to a rogue reviewer.  October 30, 2012

Exposures of conflicts of interests (COIs) associated with promoters of treatments

Soothing psychotherapists’ brains with NeuroBalm. June 24, 2014.

Are meta-analyses done by promoters of psychological treatments as tainted as those done by Pharma? May 20, 2014

Moving From Criticism and Activism to Proposals for Constructive Change

A. Democratizing Post-Publication Peer Review

Join PubMed’s Revolution in Post Publication Peer Review.October 22, 2013

B. Salvaging psychotherapy research

Salvaging psychotherapy research: a manifesto. June 10, 2014

What We Need to Do to Redeem Psychotherapy Research. June 10, 2014.