Confirmatory versus Exploratory Research
Written and curated by Emily Potts, MS, Jimmy Duong, MPH, and Shing Lee, PhD.
Overview
In a study, investigators develop a primary research question (RQ) that can be exploratory or confirmatory. The former is used when the goal is to generate hypotheses to be tested in future studies, whereas the latter allows you to test an existing hypothesis and draw inferences. Results from secondary or exploratory analyses should be interpreted with caution, due to the increased risk of false-positive findings and the potential of not having enough sample size to detect an effect in the secondary outcome measure. While many researchers are interested in multiple questions surrounding their topic, it is important to delineate between exploratory or confirmatory RQs to support the generation of clinically meaningful and reproducible scientific results.
Videos
- Exploratory and Confirmatory Analysis (5 minutes)
- Exploratory Versus Confirmatory Research (53 minutes)
- Interpretability and Credibility of Research Findings - Durham University, England (50 minutes)
- Framed in the context of pre-registration of trials, but discusses HARKing, undisclosed multiple testing, p-hacking, and selective reporting.
- "The Perils of HARKing: Hypothesizing After Results are Known" - UC Davis (33 minutes)
- Understanding the Scientific Method - Children's Hospital of Philadelphia (12 minutes)
- The steps of hypothesis-driven research and importance.
Websites
- HARKing: What is it and why is it bad? - UC Davis | Archive
- Readable lecture slides on the importance of pre-specifying RQs and hypotheses.
- THE LOGIC OF EXPLORATORY AND CONFIRMATORY DATA ANALYSIS - University of Minnesota | Archive
- Hypothesizing after the results are known (HARKing) | Archive
Readings
- Andrade, C. (2023). Types of analysis: Planned (prespecified) vs post hoc, primary vs secondary, hypothesis-driven vs exploratory, subgroup and sensitivity, and others. Indian Journal of Psychological Medicine, 45(6), 640–641. https://doi.org/10.1177/02537176231216842
- Schwab, S., & Held, L. (2020). Different worlds: Confirmatory vs exploratory research. Significance. The Royal Statistical Society. https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/1740-9713.01369
- Concise readable article.
- Kimmelman, J., Mogil, J. S., & Dirnagl, U. (2014). Distinguishing between exploratory and confirmatory preclinical research will improve translation. PLOS Biology, 12(5), e1001863. https://doi.org/10.1371/journal.pbio.1001863
- Geared toward pre-clinical studies, but gives an intuitive explanation.
- Fife, D. A., & Rodgers, J. L. (2022). Understanding the exploratory/confirmatory data analysis continuum: Moving beyond the “replication crisis”. American Psychologist, 77(3), 453–466. https://doi.org/10.1037/amp0000886
- Ring, A., Schall, R., Loke, Y. K., & Day, S. (2017). Statistical reporting of clinical pharmacology research. British Journal of Clinical Pharmacology, 83(6), 1159–1162. https://doi.org/10.1111/bcp.13254
- Focuses on how exploratory research results should be explained and presented.
- Bridges, A. J. (2022). Hypothesizing after results are known: HARKing. In W. O'Donohue, A. Masuda, & S. Lilienfeld (Eds.), Avoiding Questionable Research Practices in Applied Psychology. Springer, Cham. https://doi-org.ezproxy.cul.columbia.edu/10.1007/978-3-031-04968-2_8