Don't hate the replicator, hate the game

Key takeaways

The replication crisis poses serious questions about the validity of many published scientific studies.
Abel Brodeur's replication games are an innovative approach to improve research integrity.
Teams analyze existing studies to check both coding accuracy and the robustness of conclusions.
Transparency and sharing of data codes is crucial for effective replication in academic research.
The replication games encourage a cultural shift in academic research to prioritize honesty and reliability.

Summary

The Replication Crisis

The episode dives into the replication crisis in science, discussing how many published studies in social sciences fail to yield the same results when replicated. This crisis has implications across psychology, medicine, and economics, questioning the validity of many established findings and leading to doubts about what we actually know.

Abel Brodeur and the Replication Games

Economist Abel Brodeur is introduced as a key figure attempting to address the replication crisis. He organizes the replication games, events where teams of researchers attempt to replicate the results of previously published studies to determine their reliability. Brodeur believes that monitoring research more closely could alter behavior within the academic community.

The Process of Replication

During the replication games, researchers break into teams to methodically check previously published papers. They investigate whether the original codes and data produce the same findings and conduct robustness checks to assess the validity of the conclusions drawn from the data. Common issues include coding errors and missing variables.

Importance of Transparency in Research

The need for transparency in research is emphasized throughout the episode. Brodeur argues that scholars must publish both data and the code used in their studies to enable effective replication and scrutiny, thereby enhancing trust in published findings.

Impact of the Replication Games

The episode concludes by highlighting the potential impact the replication games could have on the broader academic community, as researchers become more aware that their work could be scrutinized at any time. This awareness is expected to foster better practices and produce more reliable research outcomes.

Show notes

The world of science has been stuck in an existential crisis over whether we actually know the things we thought we knew. Re-running an old study today doesn't always yield the same result. Same with re-enacting old experiments. Collectively, this is known as the “replication crisis.”

Economist Abel Brodeur has come up with one way to help fix this crisis: he’s invented an internationally crowdsourced surveillance system, designed to keep social scientists honest. He calls it the “Replication Games.”

Further Listening:

This episode was hosted by Mary Childs and Alexi Horowitz-Ghazi. It was produced by James Sneed and Emma Peaslee, with help from Willa Rubin. It was edited by Jess Jiang, fact-checked by Sam Yellowhorse Kesler, and engineered by Ko Takasugi-Czernowin. Alex Goldmark is Planet Money’s executive producer.

Find more Planet Money: Facebook / Instagram / TikTok / Our weekly Newsletter.

Listen free at these links: Apple Podcasts, Spotify, the NPR app or anywhere you get podcasts.

Help support Planet Money and hear our bonus episodes by subscribing to Planet Money+ in Apple Podcasts or at plus.npr.org/planetmoney.

See pcm.adswizz.com for information about our collection and use of personal data for sponsorship and to manage your podcast sponsorship preferences.

NPR Privacy Policy