When an investigator encounters a gang of unruly data…

1st Cherished Beliefs Clash with Evidence-page-001

Editors abhor negative findings, reviewers will tell you that they reveal nothing about the phenomena under study, and journalists ignore the rare occurrence of unadorned negative findings being published except when they can craft a man-bites-dog story out of them. So, what are investigators to do when unruly data do not seem to confirm hypotheses? It happens all the time, but a positive spin, a confirmatory bias is still possible has result of HARKING (hypothesizing after results are known), flexible rules of design, data analysis, and reporting, and significance chasing. Unruly data are no match for investigators bent on telling a story confirming what they were sure they already knew.

2nd Cherished Beliefs Clash with Evidence-page-0There are numerous examples in clinical and health psychology of null findings cleverly being spun into highly cited, acclaimed papers (1,2). Investigators just have to stand their ground and slay the unruly data.

The slides accompanying this brief blog post come from a presentation, When Cherished Beliefs Clash With Evidence , that can be accessed here.

1. Dimidjian, S., Hollon, S. D., Dobson, K. S., Schmaling, K. B., Kohlenberg, R. J., Addis, M. E., … & Jacobson, N. S. (2006). Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. Journal of consulting and clinical psychology, 74(4), 658.

2. Antoni, M. H., Lehman, J. M., Klibourn, K. M., Boyers, A. E., Culver, J. L., Alferi, S. M., … & Carver, C. S. (2001). Cognitive-behavioral stress management intervention decreases the prevalence of depression and enhances benefit finding among women under treatment for early-stage breast cancer. Health Psychology, 20(1), 20.

3rd Cherished Beliefs Clash with Evidence 2nd-page-001