A forthcoming post at PLOS blog Mind the Brain focuses on the ongoing conversation concerning what has come to be known as Amy Cuddy’s power posing paper:
Carney DR, Cuddy AJ, Yap AJ. Power posing brief nonverbal displays affect neuroendocrine levels and risk tolerance. Psychological Science. 2010 Oct 1;21(10):1363-8.
I will argue in that post that a multimillion dollar financial enterprise has grown around a flawed paper that received inadequate peer review. Scrutiny of the resulting published paper threatens huge financial stakes. Powerful allies protect a promoter of pop psychology advice and her work from criticism. The conversation needs to be shifted back to the exploitation of consumers, particularly women, by junk mind-body psychological pseudoscience.
The present blog post has a more modest goal of demonstrating that the original paper was incoherent with respect to male-female differences but that bad reporting distracts from problems that are hiding in plain sight.
A substantial portion of readers form opinions about papers based on nothing more than the abstract and the attention it receives. Yet abstracts are notoriously uninformative and often hide crucial details of a study needed for readers to evaluate what is said in the abstract.
The abstract to Amy Cuddy’s paper is a prime example that can be used for teaching purposes:
Humans and other animals express power through open, expansive postures, and they express powerlessness through closed, contractive postures. But can these postures actually cause power? The results of this study confirmed our prediction that posing in high-power nonverbal displays (as opposed to low-power nonverbal displays) would cause neuroendocrine and behavioral changes for both male and female participants: High-power posers experienced elevations in testosterone, decreases in cortisol, and increased feelings of power and tolerance for risk; low-power posers exhibited the opposite pattern. In short, posing in displays of power caused advantaged and adaptive psychological, physiological, and behavioral changes, and these findings suggest that embodiment extends beyond mere thinking and feeling, to physiology and subsequent behavioral choices. That a person can, by assuming two simple 1-min poses, embody power and instantly become more powerful has real-world, actionable implications.
In the abstract we are told results hold for both males and females. We lack basic details of the study needed to evaluate this claim, and especially the number of males and females.
In the Methods section, we learn that there were only 42 participants, 26 females and 16 males. We are not told how the randomization distributed the men and women across the two conditions.
We are then told:
Men were higher than women on testosterone at both Time 1, F(1, 41) = 17.40, p < .001, r = .55, and Time 2, F(1, 41) = 22.55, p < .001, r = .60. To control for sex differences in testosterone, we used participant’s sex as a covariate in all analyses. All hormone analyses examined changes in hormones observed at Time 2, controlling for Time 1. Analyses with cortisol controlled for testosterone, and vice versa.
These become rather weird, uninterpretable results. Most basically, how could the 16 males be distributed across the two groups so that the authors could confidently say that differences held for both males and females? Especially when all analyses control for sex? Sex is highly correlated with testosterone and so an analysis that controlled for both the variables, sex and testosterone would probably not generalize to the measurements of testosterone without such controls.
We are never given the correlation between cortisol and testosterone, but differences in time2 cortisol controlled for time1 cortisol, time1 testosterone and gender. Analyses would not be generalizable in a sample with 42 participants distributed across 2 groups, certainly not separately for the 26 females and 16 males taken separately.
Can anyone tell me what is going on here, except maybe the authors do did not know what they were doing or saying? Really, who would conduct a study giving a key role to testosterone in mediating self-reported effects of 2 1-minute behavioral manipulations and not discover until they peeked at results that somehow sex differences in baseline testosterone had to be take into account?
Of course, if statements are going to be made about males and females, it would be appropriate to construct a stratified sample with sufficient numbers of males and females, rather than rely on post hoc adjustments in the data.
Time for a pop psychology product recall? Stay tuned, this is only a small sampling of the problems of this paper.