Open science and multiple hypothesis testing

Content for Wednesday, March 12, 2025

Readings

• Randall Munroe. “Significant,” xkcd, no date.

• Alexander Coppock. “10 Things to Know About Multiple Comparisons,” egap, February 16, 2021.

• Preregistration and Registered Reports (Reich 2021)

Optional material

If you are interested, listen to the following podcast where “[h]ost Jeff Greene sits down for a discussion with Hunter Gehlbach and Carly Robinson, who recently served as guest editors for a special issue of Educational Psychologist entitled ‘Educational Psychology in the Open Science Era.’”

• The Division 15 Podcast Series, “Hunter Gehlbach and Carly Robinson,” APA Division 15, 2021.

You may also read the following series of short blog posts.

• Daniel Stein. “More Outcomes, More Problems: A Practical Guide to Multiple Hypothesis Testing in Impact Evaluations (Parts 1, 2, and 3),” IDinsight, November 12, 2019.

Slides

On the day of class, you will find my slides on Canvas.

References

Reich, Justin. 2021. “Preregistration and Registered Reports.” Educational Psychologist 56 (2): 101–9. https://doi.org/10.1080/00461520.2021.1900851.