Normal Curves: Sexy Science, Serious Statistics

What happens when you discover that your published paper is wrong? Does retracting it ruin your reputation—or could owning the mistake actually make things better? In this episode, we meet the inaugural winners of the Ctrl-Z Awards, which recognize scientists for correcting or retracting their own work. Behavioral ecologist Kate Laskowski takes us inside a scientific detective story involving suspicious spider data, three retracted papers, and the fear that going public might tank her career. Then computational social scientist Alyssa Smith tells us how one small error in a figure led her to retract the paper at the center of her PhD and job talk. Along the way, we talk about data forensics, the stigma around retractions, and why self-correcting scientists are the heart of science.

Statistical topics
Data management and documentation
Open science
Reproducible workflows
Research integrity
Retractions
Scientific misconduct
Statistical forensics

Methodologic Morals
“Science is self-correcting, which means scientists need to be self-correcting too.”
“Good documentation is boring—right up until it saves you.”
“If you want scientists to correct their mistakes, reward them for correcting their mistakes.”
“I'd rather be considered an honest idiot than for people to wonder if I'm untrustworthy.”

References
Retraction Watch: https://retractionwatch.com/
Ctrl-Z Awards: https://retractionwatch.com/ctrl-z-award/
Announcement of inaugural Ctrl-Z winners: https://centerforscientificintegrity.org/2026/08/04/2026-winners-ctrl-z-award-courage-retracting-own-work/
 Kate Laskowski’s blog post: https://laskowskilab.faculty.ucdavis.edu/2020/01/29/retractions/
Kate Laskowski’s Data Detective Course syllabus
Alyssa Smith’s site: https://asmithh.github.io/ 
Laskowski KL, Montiglio PO, Pruitt JN. Individual and Group Performance Suffers from Social Niche Disruption [retracted in: Am Nat. 2020 Feb;195(2):393. doi: 10.1086/708066.]. Am Nat. 2016;187(6):776-785. doi:10.1086/686220
Laskowski KL, Pruitt JN. Evidence of social niche construction: persistent and repeated social interactions generate stronger personalities in a social spider [retracted in: Proc Biol Sci. 2020 Jan 29;287(1919):20200077. doi: 10.1098/rspb.2020.0077.]. Proc Biol Sci. 2014;281(1783):20133166. Published 2014 Mar 26. doi:10.1098/rspb.2013.3166
Modlmeier AP, Laskowski KL, DeMarco AE, et al. Persistent social interactions beget more pronounced personalities in a desert-dwelling social spider [retracted in: Biol Lett. 2020 Feb;16(2):20200062. doi: 10.1098/rsbl.2020.0062.]. Biol Lett. 2014;10(8):20140419. doi:10.1098/rsbl.2014.0419
Smith AH, Green J, F Welles B, Lazer D. Emergent structures of attention on social media are driven by amplification and triad transitivity [retracted in: PNAS Nexus. 2026 May 07;5(5):pgag137. doi: 10.1093/pnasnexus/pgag137.]. PNAS Nexus. 2025;4(4):pgaf106. Published 2025 Apr 1. doi:10.1093/pnasnexus/pgaf106

Kristin and Regina’s online courses: 
Demystifying Data: A Modern Approach to Statistical Understanding  
Clinical Trials: Design, Strategy, and Analysis 
Medical Statistics Certificate Program  
Writing in the Sciences 
Epidemiology and Clinical Research Graduate Certificate Program 
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:
Kristin -  LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com

Statistical power is one of the most important ideas in statistics—and one of the most misunderstood. In this episode, we explain what statistical power really means, why researchers need to think about it before they collect their data, and what goes wrong when they calculate “post hoc power” after the study is over. Along the way, we hunt for UFOs, test Kristin’s psychic abilities, explore the ethics of underpowered studies, and discover why post hoc power is really just a p-value wearing a fake mustache. Then statistician and powerlifter Andrew Althouse joins us for the inside story of a years-long scientific feud over post hoc power, where we discuss what happens when being wrong about statistics turns into doubling down, angry letters to the editor, and debates about how science corrects itself.

Statistical topics
Effect size
False negatives
P-values
Post hoc power
Power calculations
Research ethics
Sample size
Scientific controversy
Sensitivity analysis
Statistical power
Statistical significance
Study design
Type II error

Methodologic Morals
“Big questions need studies powerful enough to answer them.”
“Post hoc power is circular reasoning masquerading as new information.”
“Know when to cut your losses. A mistake gets more expensive every time you double down.”
References
Althouse AD, Chow ZR. Comment on “Post-hoc power: If you must, at least try to understand.” Ann Surg. 2019;270:e78-e79.
Althouse AD. Post hoc power: Not empowering, just misleading. J Surg Res. 2021;259:A3-A6.
Altman DG. Statistics and ethics in medical research: III. How large a sample? BMJ. 1980;281:1336-1338.
Bababekov YJ, Hung YC, Hsu YT, et al. Is the power threshold of 0.8 applicable to surgical science?—empowering the underpowered study. J Surg Res. 2019;241:235-239.
Bababekov YJ, Stapleton SM, Mueller JL, et al. A proposal to mitigate the consequences of type 2 error in surgical science. Ann Surg. 2018;267:621-622.
Chang DC, Stapleton SM. Response: The proliferation and misinterpretation of “as safe as” statements in surgical science: A call for professional discourse to search for a solution. J Surg Res. 2021;259:A12-A15.
Freiman JA, Chalmers TC, Smith H Jr, et al. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial: Survey of 71 negative trials. N Engl J Med. 1978;299:690-694. 
Hoenig JM, Heisey DM. The abuse of power: The pervasive fallacy of power calculations for data analysis. Am Stat. 2001;55:19-24.
Nuzzo RL. Statistical Power. PM&R. 2016;8(9):907-912. doi:10.1016/j.pmrj.2016.08.004
Nuzzo RL. Post hoc Power. PM&R. 2021;13(4):422-424. doi:10.1002/pmrj.12476

Kristin and Regina’s online courses: 
Demystifying Data: A Modern Approach to Statistical Understanding  
Clinical Trials: Design, Strategy, and Analysis 
Medical Statistics Certificate Program  
Writing in the Sciences 
Epidemiology and Clinical Research Graduate Certificate Program 
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:
Kristin -  LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com

(00:00) - Intro

(00:43) - What is Statistical Power?

(03:22) - The UFO Telescope Analogy

(06:19) - The Psychic Ability Experiment

(10:36) - Planning Power Before You Study

(12:46) - History and the Landmark 1978 Paper

(17:42) - Enter Post-Hoc Power: The Problem

(19:26) - Three Problems with Post-Hoc Power

(24:33) - The Surgery Journal Disaster

(29:59) - Guest Interview: Andrew Althouse

(32:30) - The 2018 Paper and the Controversy Begins

(39:54) - Doubling Down and the Cost of Being Right

(53:03) - Rating the Claim

Can weed protect your brain as you get older? We unpack a study linking lifetime cannabis use to larger brain volumes and better cognitive test scores in older adults—and investigate how those findings became headlines urging readers to “pass the pot to grandma.” Along the way, we explore collapsing categorical variables, confounding, clinically versus statistically significant results, multiple testing, and how a press release subtly changed a story. We also meet permanently stoned mice, learn about Carl Sagan’s surprising connection to cannabis research, and discover how a single missing word can change the entire meaning of a study.
Statistical topics
Categorical variables
Confounding
Effect size
False discovery rate
Healthy volunteer bias
Measurement error
Multiple testing
Observational studies
Recoding variables
Science communication
Statistical vs practical significance

Methodologic Morals
“Always follow the chain from data to result to conclusions and pay attention to the words along the way.”
“Huge samples can give tiny differences delusions of grandeur.”

References
Guha A, Fu Z, Calhoun V, Hutchison KE. Lifetime Cannabis Use Is Associated With Brain Volume and Cognitive Function in Middle-Aged and Older Adults. J Stud Alcohol Drugs. 2026;87(4):739-751. doi:10.15288/jsad.25-00346
Press Release: https://news.cuanschutz.edu/news-stories/study-finds-cannabis-usage-in-middle-aged-and-older-adults-associated-with-larger-brain-volume-better-cognitive-function
Bilkei-Gorzo A, Albayram O, Draffehn A, et al. A chronic low dose of Δ9-tetrahydrocannabinol (THC) restores cognitive function in old mice. Nat Med. 2017;23(6):782-787. doi:10.1038/nm.4311
NY Post: https://nypost.com/2026/02/07/health/cannabis-may-benefit-aging-brains-study-finds/
David Grinspoon: https://www.nytimes.com/2020/07/02/science/lester-grinspoon-dead.html?unlocked_article_code=1.7lA.CyS1.neeqGGNgx8g1&smid=url-share

Kristin and Regina’s online courses: 
Demystifying Data: A Modern Approach to Statistical Understanding  
Clinical Trials: Design, Strategy, and Analysis 
Medical Statistics Certificate Program  
Writing in the Sciences 
Epidemiology and Clinical Research Graduate Certificate Program 
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:
Kristin -  LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com

(00:00) - Opening: The Brain on Drugs

(00:37) - The Cannabis Headline

(03:38) - The Claim: Brain Benefits for Older Adults

(06:53) - The Mouse Study: Pothead Mice vs. Control Mice

(11:25) - From Mice to Humans: UK Biobank

(15:01) - Measurement Problems and Category Collapse

(21:39) - Results: Statistically Significant, Clinically Meaningless

(27:13) - Individual Brain Regions and Cherry-Picking

(32:32) - Cognitive Tests: What Actually Matters

(40:09) - How It Gets Lost in Translation

(45:32) - Rating the Claim

Can the meningitis vaccine do double duty against gonorrhea? We examine a fascinating randomized trial inspired by years of encouraging observational studies—and what happened when those earlier results were finally put to the test. Along the way, we discuss ecological studies, case-control studies, why statisticians distrust phrases like “trend toward significance,” how intention-to-treat differs from per-protocol analysis, and why no amount of statistical adjustment can fully substitute for randomization. Along the way we celebrate outrageously named clinical trials, ask whether “flirting with statistical significance” belongs in a romance novel or a medical journal, and discover that “practice safe statistics” may be the best advice of the episode.

Statistical topics
Case-control studies
Confounding
Ecological studies
Generalizability
Intention-to-treat
Meta-analysis
Null results
Observational studies
P-values
Per-protocol analysis
Randomized clinical trials
Survival analysis

Methodologic Morals
“Don't mistake a p-value of 0.06 for a p-value of 0.04 with bad luck.”
 “Practice safe statistics; randomize whenever possible.”
References
Seib KL, Donovan B, Jin F, et al. Meningococcal B Vaccine to Prevent Neisseria gonorrhoeae Infection. N Engl J Med. 2026; 395: 349-61. 
Petousis-Harris H, Paynter J, Morgan J, et al. Effectiveness of a group B outer membrane vesicle meningococcal vaccine against gonorrhoea in New Zealand: a retrospective case-control study. The Lancet. 2017; 390: 1603–10.
Wang, B., Mohammed, H., Andraweera, P., McMillan, M., Marshall, H., 2024. Vaccine effectiveness and impact of meningococcal vaccines against gonococcal infections: A systematic review and meta-analysis. Journal of Infection. 2024; 89: 106225. 
Molina JM, Bercot B, Assoumou, L, et al. Doxycycline prophylaxis and meningococcal group B vaccine to prevent bacterial sexually transmitted infections in France (ANRS 174 DOXYVAC): a multicentre, open-label, randomised trial with a 2 × 2 factorial design. The Lancet Infectious Diseases. 2024; 24: 1093–1104.
Thng C, Eskandari S, Jin F, et al. Efficacy of the meningococcal vaccine against Neisseria gonorrhoeae: a randomised clinical trial (MenGO). npj Vaccines. 2026.

Still Not Significant | Probable Error: List of funny disguises scientists use to describe p>.05

Kristin and Regina’s online courses: 
Demystifying Data: A Modern Approach to Statistical Understanding  
Clinical Trials: Design, Strategy, and Analysis 
Medical Statistics Certificate Program  
Writing in the Sciences 
Epidemiology and Clinical Research Graduate Certificate Program 
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:
Kristin -  LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com

(00:15) - - Introduction

(00:51) - - The Claim: Can a Meningitis Vaccine Prevent Gonorrhea?

(04:10) - - The Biology Behind the Idea

(10:37) - - How Observational Studies Made the Link

(14:44) - - Earlier Clinical Trials: MENGO and DOXYVAC

(17:30) - - Flirting with Significance: A Statistical Rant

(22:40) - - GoGoVax: Study Design and Analysis

(33:31) - - The Shocking Null Result

(35:38) - - Why These Results Don't Generalize

(37:42) - - Rating the Claim

Can you improve your endurance by sitting in a hot tub instead of doing another hard workout? We dig into a study that put elite runners through five weeks of what we affectionately call “endurance hot tubbing” to see whether heat alone could trigger the same cardiovascular adaptations as altitude training. Along the way, we explore crossover study designs, multiple testing, why significant physiological changes don’t necessarily translate into better athletic performance, and how an exploratory regression analysis ended up answering the wrong scientific question. We also discover that serious hot tub research involves surprisingly little relaxation, debate whether bubbles should count as an experimental condition, and learn why distance runners pay good money to sleep in tents with some of the oxygen sucked out.

Statistical topics
best subsets regression
causal inference
crossover studies
exploratory analyses
mediation / mechanisms
model specification
multiple testing
peer review
sample size

(00:15) - Introduction

(00:44) - Can Hot Tubs Replace Exercise?

(01:27) - Crazy Runners and Crazy Training

(05:44) - The Amazon Delivery System Metaphor

(12:12) - From Heat Training to Hot Tubs

(13:33) - The Study: 10 Runners in Hot Water

(22:31) - Results and the VO2 Max Question

(29:26) - Statistical Sleuthing: Best Subsets Regression

(36:50) - The Peer Review Problem

(41:39) - Rating the Claim

Methodologic Morals
“A model can be very good at answering the wrong question.”
 “The label ‘peer reviewed’ does not guarantee ‘carefully reviewed’.”
References
Jenkins EJ, Killick JA, Zerilli O, et al. Long-term passive heat acclimation enhances maximal oxygen consumption via haematological and cardiac adaptation in endurance runners. J Physiol. Published online November 20, 2025. doi:10.1113/JP289874
Stembridge M, Jenkins E. Marathon training: Why hot baths might help you run faster. The Conversation. Published March 16, 2026. doi:10.64628/AB.6vknm9esh

Kristin and Regina’s online courses: 
Demystifying Data: A Modern Approach to Statistical Understanding  
Clinical Trials: Design, Strategy, and Analysis 
Medical Statistics Certificate Program  
Writing in the Sciences 
Epidemiology and Clinical Research Graduate Certificate Program 
Programs that we teach in:
Epidemiology and Clinical Research Graduate Certificate Program 

Find us on:
Kristin -  LinkedIn & Twitter/X
Regina - LinkedIn & ReginaNuzzo.com

(00:15) - Introduction

(00:44) - Can Hot Tubs Replace Exercise?

(01:27) - Crazy Runners and Crazy Training

(05:44) - The Amazon Delivery System Metaphor

(12:12) - From Heat Training to Hot Tubs

(13:33) - The Study: 10 Runners in Hot Water

(22:31) - Results and the VO2 Max Question

(29:26) - Statistical Sleuthing: Best Subsets Regression

(36:50) - The Peer Review Problem

(41:39) - Rating the Claim