In a previous post, I wrote about the contentious atmosphere that so often surrounds replication studies, and fantasized a world in which one might occasionally see replication researchers and the original authors come together in “a joint effort to share methods, look at data together, and come to a collaborative understanding of an important scientific issue.” Happily one example that comes close to this ideal has been recently accepted for publication in Psychological Science — the same journal that published the original paper. The authors of both the original and replication studies appear to have worked together to share information about procedures and analyses, which while perhaps not a full collaboration, is at least cooperation of a sort that’s seen too rarely. The result was that the original, intriguing finding did not replicate; two large new studies obtained non-significant findings in the wrong direction. The hypothesis that anxiously attached people might prefer warm foods when their attachment concerns are activated was provocative, to say the least. But it seems to have been wrong.
With this example now out there, I hope others follow the same path towards helping the scientific literature perform the self-correcting process that, in principle, is its principal distinctive advantage. I also hope that, one of these days, an attempt to independently replicate a provocative finding will actually succeed! Now that would be an important step forward.
UPDATE, April 15: Via Eric Eich, the editor of Psychological Science who accepted the paper, discussed above, by LeBel and Campbell that failed replicate the study by Matthew Vess. Eich offered Vess the opportunity to publish a rejoinder and this is what Vess said:
Thank you for the opportunity to submit a rejoinder to LeBel and Campbell’s commentary. I have, however, decided not to submit one. While I am certainly dismayed to see the failed attempts to reproduce a published study of mine, I am in agreement with the journal’s decision to publish the replication studies in a commentary and believe that such decisions will facilitate the advancement of psychological science and the collaborative pursuit of accurate knowledge. LeBel and Campbell provide a fair and reasonable interpretation of what their findings mean for using this paradigm to study attachment and temperature associations, and I appreciated their willingness to consult me in the development of their replication efforts. Once again, thank you for the opportunity.
Hats off to Matthew Vess. Imagine if everyone whose findings were challenged responded in such a civil and science-promoting manner. What a wonderful world it would be.