Apparently John Grohl was taken aback by my criticism of neurononsense in a PLOS One article. I am pleased at gaining recognition at his highly accessed blog, but I think he was at least a bit confused about what was going on. The following comment is left at his blog post for approval.
John, thank you for encouraging people to read my blog post. If I ever get around to monetizing my activity by collecting my blog posts in a book, I will remind you that you said I know bad research when I see it and that even that I am brilliant. I will ask for a dust jacket endorsement or maybe a favorable review at Amazon.
Like you, I find it amazing that I was allowed free reign as a blogger to mount such an attack on an article that PLOS had published. I had filed a complaint with the journal over the undisclosed conflict of interest of one of the authors. I informed the managing editors that I would be merciless in my blog post in exposing what could found with meticulous reads and rereads of an article, once I was alerted by a conflict of interest. The journal is processing my formal complaint. Management indicated no interest in previewing my blog post, but only asked that I indicate it was my personal opinion, not that of the journal. I am confident that if this process were unfolding at a for-profit journal, or worse, one associated with a professional organization such as American Psychological Association or Association for Psychological Science, there would have been an effort to muzzle me, but little prospects for a full formal review of my complaint that respected the authors’ rights as well as my own.
PLOS One requests disclosure of potential conflicts of interest in exquisite detail and accompanies every article with a declaration. It has an explicit procedure for reviewing complaints of breaches of its policies. I expressed my opinion in both in my blog post and in my complaint that the authors violated the trust of the journal and the readers by failure to make the disclosure of extensive financial benefits associated with an uncritical acceptance of the claims made in the article. But PLOS One does not go on the opinion of one person. Even when the complainant is one of its 4000 Academic Editors, it reviews the evidence and solicits additional information from authors. I am confident in the fairness of the outcome of that review. If it does not favor of my assessment, I will apologize to the authors, but still assert that I had strong basis for my complaints.
Like many people associated with PLOS, I have a great skepticism about the validity of prepublication review in certifying the reliability of what is said in published papers. My blog posts are an incessant effort to cultivate a skepticism in others and provide them with the tools to decide for themselves about whether to accept what they read. PLOS is different than most journals in providing readers with immediate opportunities to comment on articles in a way that will be available to anyone accessing them. I encourage readers to make more use of those opportunities, as well as PubMed Commons for post publication peer review.
I am pleased and flattered that you think I laid out the problems in the article so bare that they are now obvious and should have precluded publication of the article. But it took a lot of work and lots of rereads and expertise that I alone do not possess. I got extensive feedback from a number of persons, including neuroscientists and I am quite indebted in particular to Neurocritic. I highly recommend his earlier blog post about this article. He had proceeded to the point of sniffing out that something was amiss. But when he called out Magnetto, the BS detector to investigate, he was thwarted by the lack of some technical details, as well has his inability to get into the down and dirty of the claims were being made about clinical psychology science. As we went back and forth building upon the revelations of the other, we were both shocked and treated to aha experiences – why didn’t I notice that?
Initially, we trusted the authors citing a previous paper in Psychological Science for the validity of their methods and their choice of Regions of Interest (ROIs) of the brain for study. It took a number of reads of that paper to discover that they were not accurately reporting what was in that paper or the lack of correspondence to what was done in the PLOS paper. I consider the Psychological Science paper just as flawed and at risk for nonsense interpretations, but I have no confidence in APS or that journal’s tolerance for being called out on shortcomings in their peer review. I speak from experience.
Taken together, I consider my blog post and the flaws in the PLOS article that I targeted as indications of the need for readers to be skeptical about depending on prepublication peer review in evaluating the reliability of articles. And let us see the outcome of the formal review as to whether there is the self correction, if it is necessary, that I think we can depend on PLOS to provide.
Finally, let’s all insist on disclosure of conflicts of interest in every paper, not just those coming from the pharmaceutical industry. My documentation of the problems with promoters of Triple P Parenting have led to bar fights with some journals and even formal complaints to the Committee on Publication Ethics (COPE). Keep watching my blog posts to see the outcome of those fights. Disclosures of conflicts of interest depend on the candor of authors. I would have been a hypocrite if I did not call out the authors of a PLOS One article in the same way that I call out authors of publications in other journals.