When Did We Get so Delicate?

Replication issues are rampant these days. The recent round of widespread concern over whether supposedly established findings can be reproduced began in biology and the related life sciences, especially medicine. Psychologists entered the fray a bit later, largely in a constructive way. Individuals and professional societies published commentaries on methodology, journals acted to revise their policies to promote data transparency and encourage replication, and the Center for Open Science took concrete steps to make doing research “the right way” easier. As a result, psychology was viewed not as the poster child of replication problems, quite the opposite. It became viewed as the best place to look for solutions to these problems.

So what just happened? In the words of a headline in the Chronicle of Higher Education, the situation in psychology has suddenly turned “ugly and odd.”  Some psychologists whose findings were not replicated are complaining plaintively about feeling bullied. Others are chiming in about how terrible it is that people’s reputations are ruined when others can’t replicate their work. People doing replication studies have been labeled the “replication police,” “replication Nazis” and even, in one prominent psychologist’s already famous phrase, “shameless little bullies.” This last-mentioned writer also passed along an anonymous correspondent’s description of replication as a “McCarthyite nightmare.”  More sober commentators have expressed worries about “negative psychology” and “p-squashing.” Concern has shifted away from the difficulties faced by those who can’t make famous effects “work,” and the dilemma about whether they dare to go public when this happens. Instead, prestigious commentators are worrying about the possible damage to the reputations of the psychologists who discovered these famous effects, and promulgating new rules to follow before going public with disconfirmatory data.

First, a side comment: It’s my impression that reputations are not really damaged, much less ruined, by failures to replicate. Reputations are damaged, I fear, by defensive, outraged reactions to failures to replicate one’s work. And we’ve seen too many of those, and not enough reactions like this.

But now, the broader point: When did we get so delicate? Why are psychologists, who can and should lead the way in tackling this scientific issue head-on, and until recently were doing just that, instead becoming distracted by reputational issues and hurt feelings?

Is anybody in medicine complaining about being bullied by non-replicators, or is anyone writing blog posts about the perils of “negative biology”? Or is it just us? And if it’s just us, why is that? I would really like to know the answer to this question.

For now, if you happen to be a psychologist sitting on some data that might undermine somebody’s famous finding, the only advice I can give you is this:  Mum’s the word.  Don’t tell a soul.  Unless you are the kind of person who likes to poke sticks into hornets’ nests.

5 thoughts on “When Did We Get so Delicate?

  1. It seems that what Meehl called the “spun-glass theory of the mind” may be getting applied by some psychologists to themselves as a profession. While I’m not a fan of the “If you can’t stand the heat, get out of the kitchen” school of discourse, I think that PIs in psychology ought to be able to read critical evaluations of their work in peer-reviewed journals — even special issues of those journals with a distinctly “replication” flavour, although of course such things ought not to be necessary — without resorting to claims of bullying. Absent substantial evidence of malicious intent, the latter is in effect is an ad hominem attack on the replicators.

  2. “Is anybody in medicine complaining about being bullied by non-replicators, or is anyone writing blog posts about the perils of “negative biology”? Or is it just us? And if it’s just us, why is that? I would really like to know the answer to this question.”

    Maybe the kind of researchers that are currently in the field have some characteristics that allow for such a reaction. Maybe due to the “publish or perish” culture and accompanying processes, the “winners” in/of this system are researchers that a) don’t bother with replication themselves , b) downplay the importance or even the possibility of direct replications , c) heavily critisize it when it is performed in those rare instances.

    If it is “just us” I would think that perhaps findings in psychology have in it a decent amount of unreliability or uncertainty compared to other disciplines, which may result in the extra possibility for the original authors to come up with tons of reasons why a certain finding did not replicate (“moderator variables”) without even starting to doubt their own original findings. It’s almost impossible this way to disprove any finding out there, which is funny in my opinion.

    So my guess is that if it is “just us”, psychology has arrived at a point in which there are some?/ many? researchers who do not have basic scientific characteristics (due to the progression of the field in the last decades) combined with the suggestion that psychology’s findings could be relatively unreliable or uncertain. This results in cases in which replications are not viewed as important and if they are performed, the results can always be heaviliy contested with the many possible reasons why a certain finding did not replicate due to “moderation variables” and the like.

    • This is just anecdotal information, and gives no information about possible causes, but my experience is that when I point out a problematical statistical practice to a biologist, their reaction is typically one of concern — they thank me for pointing it out, or ask questions to help them understand my criticism, or ask me for references to help them learn more, or ask for suggestions on what they can do that is better, or at least say, “I’ll have to think about that.”

      In contrast, when I (try to) point out a problematical statistical practice (which I try to word in a considerate manner) about psychological research, I typically get no response, or at best something like “Thank you for taking an interest in my research.”

      If you’d like to see some examples of some problems that I believe need to be addressed in research in psychology (and also in other fields), please see some of my recent comments (and also some from January 2013) at http://www.ma.utexas.edu/blogs/mks/

  3. Pingback: sometimes i'm wrong: self-correction hurts

  4. Psychologists are so delicate, because they have been able to fool the public about their scientific status and now their cover has been blown. To top it all, many psychologists are some of the most nasty, whiny, politically correct and hypocrite people in academia (and many other social institutions) today. Hopefully, the reign of bad faith in psychology ends with this movement. Don’t count on it though

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s