BMC Medicine gets caught up in Triple P Parenting promoters’ war on critics and null findings

Undeclared conflicts of interest constitute scientific misconduct.

Why we should be as concerned about conflicts of interest in evaluations of nonpharmacological treatments, like psychotherapy.

whackWhack! Triple P promoters (3P) Cassandra L Tellegen and Kate Sofronoff struck again against critics and null findings, this time in BMC Medicine. As usual, there was an undisclosed financial conflict of interest.

Until recently, promoters of the multimillion-dollar enterpriseNothing_to_Declare controlled perception of their brand of treatment. They authored most reports of implementations and also systematic reviews and meta-analyses. They did not report financial conflicts of interest and denied any conflict when explicitly queried.

The promoters were able to insist on the official website:

No other parenting program in the world has an evidence base as extensive as that of Triple P. It is number one on the United Nations’ ranking of parenting programs, based on the extent of its evidence base.

At least two of the developers of 3P and others making money from it published a systematic review and meta-analysis they billed as comprehensive:

Sanders, M. R., Kirby, J. N., Tellegen, C. L., & Day, J. J. (2014). The Triple P-Positive Parenting Program: A systematic review and meta-analysis of a multi-level system of parenting support. Clinical Psychology Review, 34(4), 337-357.

Promoters of 3P are still making extravagant claims, but there has been noticeable change in the view from elsewhere. An independently conducted meta-analyses in BMC Medicine  demonstrated that previous evaluations depended heavily on flawed, mostly small studies that very often had undeclared conflicts of interest. I echoed and amplified the critique of the 3P Parenting literature, first in blog posts [1 , 2]  and then in an invited commentary in BMC Medicine.

The sordid history of the promoters’ “comprehensive” meta-analysis was revealed  and its overwhelming flaws were scrutinized.

Over 30 errata, addenda, and  corrigenda have been attached to previously published 3P articles and more keep accumulating. Just try Google scholar with “triple P parenting” and “erratum” or “addendum” or “corrigendum.” We will be seeing more errata as more editors are contacted.

Please click to enlarge
Please click to enlarge
Please click to enlarge
Please click to enlarge

There were reports in social media of how studies with null findings have been previously sandbagged in anonymous peer review or how authors were pressured by peer reviewers to spin results. Evidence surfaced of 3P founder Matt Sanders attempting to influence the reporting of a supposedly independently conducted evaluation. It is unclear how frequently this occurs, but represents a weakening of the important distinction between independent evaluations and those with conflicts of interest.

The Belgian government announced defunding of 3P programs. Doubts whether 3P was the treatment of choice were raised in 3P’s home country. 3p is a big ticket item in Australia, with New South Wales alone spending $6.6 million on it.

A detailed critique called into question the positive results claimed for one of the largest and influential population-based 3P interventions, and the non-disclosed conflicts of interest of the authors and the editorial board of the journal in which it appeared – Prevention Sciencewere exposed.

Are we witnessing the decline effect  in the evaluation of 3P? Applied to intervention studies, the term refers to the recurring pattern when weaker results accumulate from larger, more sophisticated studies not conducted by promoters of the intervention who initially had produced glowing reports of efficacy and effectiveness.

But the 3P promoters viciously and unethically fought back. Paid spokespersons took to the media to denounce independently conducted negative evaluations. Critics were threatened in their workplace, letters of complaint were written to their universities. Programs threatened with withdrawal of 3P resources if the critics weren’t silenced. Publications with undisclosed conflicts of interest authored by paid promoters of 3P continue to appear, despite the erratum and addendum apologizing for what had occurred in the past.

In this issue of Mind the Brain, I review the commentary in BMC Medicine. I raise the larger issue of whether the promoters of 3P’s recurring undeclared conflicts of interests represents actionable scientific misconduct. And I deliver a call to action.

My goal is to get BMC Medicine to change its policies concerning disclosure of conflict of interest and its sanctions for nondisclosure. I am not accusing the editorial board of BMC Medicine of wrongdoing.

The journal was the first to publish serious doubts about the effectiveness of 3P. Scottish GP  Phil Wilson and colleagues went there after his meta analysis was trashed in anonymous peer review at Elsevier’s Clinical Psychology Review (CPR). He faced retaliation from the workplace after he was contacted directly by the founder of 3P immediately after his submission to CPR. Matt Sanders sent him papers published after the end date Wilson had set for the papers included in his meta analysis. Bravo for BMC Medicine for nevertheless getting Wilson’s review into print. But the BMC Medicine editors have been repeatedly duped by 3P promoters and they now have the opportunity to serve as a model for academic publishing in mounting an effective response.

Stepping Stones Triple P: the importance of putting the findings into context

The BMC Medicine commentary by Tellegen and Sofronoff  is available here. The commentary first appeared without a response from the authors who were being criticized, but that has now been rectified.

Tellegen and Sofronoff chastised  the authors of a recent randomized trial [d], also published in BMC Medicine that evaluated the interventions with parents of children with Borderline to Mild Intellectual Ability (BMD).

Firstly, the authors present a rationale for conducting the study that does not accurately represent the current state of evidence for SSTP. Secondly, the authors present an impoverished interpretation of the findings within the paper.

The “current state of evidence for SSTP” about which Tellegen and Sofronoff complain refers to a systematic review and meta-analysis authored by Tellegen and Matt Saunders. I previously told how

  • An earlier version of this review was circulated on the Internet labeled as under review at Monographs of the Society of Research in Child Development. It’s inappropriate to distribute manuscripts indicating that they are “under review” at particular journals. APA guidelines explicitly forbid it. This may have led to the manuscript’s rejection.
  • The article nonetheless soon appeared in Clinical Psychology Review in a version that differed little from the manuscript previously available on the Internet, suggesting weak peer-review.
  • The article displays numerous instances of meta analysis malpractice. It is so bad and violates so many standards, that I recommend its use in seminars as an example of bad practices.
  • This article had no declared conflicts of interests.

Tellegen and Sofronoff’s charge of ”impoverished interpretation of the findings within the paper” refers to the investigators failing to cite 4 quite low quality studies that were not randomized trials but were treated as equivalent to RCTs in Tellegen and Sanders own meta-analyses.

In their response to the commentary from 3P, three of the authors – Sijmen A Reijneveld, Marijke Kleefman, and Daniëlle EMC Jansen of the original trial calmly and effectively dismissed these criticisms. They responded a lot more politely than I would have.

is youThe declarations of conflict of interest of 3P promoters in BMC Medicine: Is you is or ain’t you is making money?

An earlier commentary in BMC Medicine whose authors included 3P developer Matt Sanders and Kate Sofronoff – an author of the commentary under discussion – stated in the text:

Triple P is not owned by its authors, but by The University of Queensland. Royalty payments from dissemination activities, principally the sale of books, are paid by the publisher (Triple P International) to the University of Queensland’s technology transfer company (UniQuest), and distributed to the university’s Faculty of Social and Behavioural Sciences, School of Psychology, Parenting and Family Support Centre and contributory authors in accordance with the university’s intellectual property policy. None of the program authors own shares in Triple P International, the company licensed by the University of Queensland to disseminate the program worldwide.

What is one to make of this? It seems to answer “no” to the usual question of whether authors own stock or share ownership in a company. It doesn’t say directly about what happens to the royalties from the sale of books. Keep in mind, that the multimillion dollar enterprise of 3P involves selling lots of books, training materials, workshops, and government contracts. But a reader would have to go to the University of Queensland’s intellectual property policy to make sense of this disclaimer.

The formal COI statement in the article does not clarify much, but should arouse curiosity and skepticism –

…Royalties stemming from this dissemination work are paid to UniQuest, which distributes payments to the University of Queensland Faculty of Social and Behavioural Sciences, School of Psychology, Parenting and Family Support Centre, and contributory authors in accordance with the University’s intellectual property policy.

No author has any share or ownership in Triple P International. MS is the founder and lead author of the Triple P-Positive Parenting Program, and is a consultant to Triple P International. JP has no competing interests. JK is a co-author of Grandparent Triple P. KT is a co-author of many of the Triple P interventions and resources for families of children up to 12 years of age. AM is a co-author of several Triple P interventions for young children including Fuss-Free Mealtime Triple P. TM is a co-author of Stepping Stones Triple P for families of children with disabilities. AR is a co-author of Teen Triple P for parents of adolescents, and is Head of Training at Triple P International. KS has no competing interests.

omgThe authors seem to be acknowledging receiving money as “contributory authors” but there is still a lot of beating around the bush. Again, one needs to know what more about the university’s intellectual properties policy. Okay, take the trouble to go to the website for the University of Queensland to determine just how lucrative the arrangements are. You will surely say “Wow!” If you keep in mind the multimillion dollar nature of the 3P enterprise.

Please click to expand
Please click to expand

The present commentary in BMC Medicine seems to improve transparency –

The Triple P – Positive Parenting Program is owned by The University of Queensland (UQ). The University through its main technology transfer company, UniQuest Pty Ltd, has licensed Triple P International Pty Ltd to publish and disseminate the program worldwide. Royalties stemming from published Triple P resources are distributed to the Faculty of Health and Behavioural Sciences at UQ, Parenting and Family Support Centre, School of Psychology at UQ, and contributory authors. No author has any share or ownership in Triple P International Pty Ltd. Cassandra Tellegen and Kate Sofronoff are employees of the UQ and members of the Triple P Research Network

But the disclosure remains evasive and misleading. One has to look elsewhere to find out that there is only a single share of Triple P International Pty Ltd, owned by Mr Des McWilliam. He was awarded a 2009 honorary doctorate by the University of Queensland in 2009. The citation … acknowledged that

Mr McWilliam’s relationship with Triple P had provided grant leveraging, both nationally and internationally, for ongoing research by the PFSC and had supported ongoing international trials of the program.

another wedding photoInteresting, but there is still an undeclared COI that is required for adherence to the International Committee of Medical Journal Editors (ICMJE) to which BMC Medicine subscribes. Just as Matt Sanders is married to Patricia Sanders, Cassandra L Tellegen is married to James Kirby, a psychologist who has written at least 12 articles with Sanders on 3 P and a 3P workbook for grandparents. Aha, both Sanders and Tellegen are married to persons financially benefiting from 3P programs. All in the family. And spousal relationships are reportable conflicts of interest.

I don’t know about you, but I’m getting damn sick and tired of all the shuck ‘n jiving from triple P parenting when they’re required to disclose conflicts of interest.

shark-life-guardWhy get upset about conflict of interests in evaluations of nonpharmacological trials and reviews?

My colleagues and I played a role in improving the tracking of conflicts of interest going from industry-supported clinical trials to inclusion in meta-analyses. Our criticism prompted Cochrane Collaboration to close a loophole in investigator conflict of interest not having been identified as a formal risk of bias. Prior to the change, results of an industry sponsored pharmacological trial could be entered into a meta-analysis where the origins were no longer apparent. The collaboration awarded us the Bill Silverman Award for pointing out the problem.

It’s no longer controversial that in the evaluation of pharmacological interventions involving financial conflicts of interest are associated with inflated claims for efficacy. But the issue is ignored in evaluating nonpharmacological interventions, like psychotherapies or social programs like 3P.

Undeclared conflicts of interest in nonpharmacological trials threaten the trustworthiness of the psychological literature.

Readers are almost never informed about conflicts of interest in the trials evaluating psychotherapy evaluations and their integration in meta-analyses. Yet, “investigator allegiance” a.k.a. undeclared conflict of interest is one of the most robust predictors of effect size. Indeed, knowing the allegiance of an investigator more reliably predicts the direction of results than the particular psychotherapy being evaluated.

As reviewed in my numerous blog posts  [1,2,3], there are no doubts that evaluations of 3P are inflated with a strong confirmation bias associated with undeclared complex of interest.

But the problem is bigger than that when it comes to 3P. Millions of dollars are being invested in on claims that improvement in parenting skills resulting from parents’ participation in 3P are a solution for pressing larger social problems. The money that could be being wasted on 3P is diverted from other solutions. And participation of parents in 3P programs is often not voluntary. They participate to avoid other adverse outcomes like removal of the children from their home by enrollment in 3P. That’s not a fair choice, when 3P may not provide them any other benefit and certainly not what it is advertised as providing.

HMarriage-image2We should learn from the results of President George W. Bush committing hundreds of millions of dollars to promote stable and healthy marriages. The evidence for the programs selected for implementation were almost entirely from small-scale, methodologically flawed studies conducted by their developers who typically did not publish with declared conflicts of interest. Later evaluations showed the programs to be grossly ineffective. An independent evaluation  showed positive findings of the particular programs did not occurred more than would be expected by chance. What a waste, but I doubt President Bush cared. As part of a larger package, he was able to slash welfare payments to the poor and shorten the allowable time for unemployment payments.

Politicians will accept ineffective social programs if they are in the service of being able to claim that they are not just doing nothing, they are offering solutions. And the ineffective social programs are particularly attractive when they cost less than a serious effort to address the social problems.

must declare
Please click to enlarge

goods to declare2pgWhat I’m asking of BMC Medicine: A model response

  • Consistent with Committee on Publication Ethics (COPE) recommendations, persons with conflict of interest should not be invited to write commentaries. I’m not sure that wanting to respond to null findings for their prized product is a justifiable override of this restriction. But if a commentary is deemed justified, there needs to be no ambiguity about the declaration of conflict of interest by the authors.
  • If journals have a policy of commentaries not undergoing peer review, it should be indicated at each and every commentary that is the case. That would be consistent with COPE recommendations concerning non-peer-reviewed papers in journals identifying themselves as peer-reviewed.
  • Consistent with the opinion of many universities, failure to declare conflicts of interest constitutes scientific misconduct.
  • Scientific misconduct is grounds for retraction. Saying “Sorry, we forgot” in an erratum is an inadequate response. We need some sort of expanded pottery barn rule by which journals don’t just allow author to publish an apology when the journal discovers an undeclared conflict of interest.
  • Articles for which authors declare conflicts of interest should be subject to particular editorial scrutiny, given the common association of conflicts of interest and spinning of results and other confirmatory bias.
  • Obviously, 3P promoters have had problems figuring out what conflicts of interest they have to declare. How about requiring all articles to require a statement that I first saw in a BMJ article, something like

I have read all ICMJE standards and on that basis declare the following:

If authors are going to lie, let’s make it obvious and more actionable.

Please listen Up, PLOS One

I am grateful to PLOS One for carefully investigating my charges that the authors of an article had substantial undeclared conflicts of interest.

The situation was outrageous. Aside from the conflicts of interest, the article was – as I documented in my blog post – neurobalm. The appearance of positive results was obtained by selective reporting of the data from analyses redone after previous analyses did not produce positive results. A misleading video was released on the internet accompanied by soft music and claims to demonstrate scientific evidence in PLOS One that a particular psychotherapy “soothed the threatened brain.” Yup, that was also in the title of the PLOS One article. The highly spun article was part of a marketing of workshops to psychotherapists who likely had little or no research training.

I volunteer as an Academic Editor for PLOS One and I resent the journal being caught up in misleading clinicians – and the patients they treat.

Upon investigation, the journal added an elaborate conflict of interest statement to the article. I’m impressed with the diligence with which the investigation was conducted.

Yet, the absence of a previous statement meant that the authors had denied any conflicts of interest in response to a standard query from the journal during the submission process.I think their failure to make an appropriate disclosure is scientific misconduct. Retraction should be considered.

Given the strong association between conflicts of interests or investigator allegiance in outcomes of psychosocial research, revelation of the undisclosed conflict of interest should have at least precipitated a careful re-review with heightened suspicion of spin and bias. And not by an editor who had not been informed of the conflict of interest and had missed the flaws the first time the article was reviewed. Editors are humans, they get defensive when embarrassed.

Disclaimer: The opinions I express here are my own, and not necessarily those of the PLOS One or other members of the editorial board. Thankfully, at Mind the Brain, bloggers are free to speak out for themselves without censorship or even approval from the sponsoring journal. Remember what happened at Psychology Today and how I came to blog here.

 

 



Leave a comment

Please note, comments must be approved before they are published