Frank Zenker, 12 March 2019, 16h

ArgLab Research Colloquium

Av. de Berna 26, I&D Building, ground floor, room 0.06

Why replication is your problem, too

Frank Zenker, Lund University, Sweden

Scholars, as opposed to scientists, do normally not conduct empirical studies. But their own work regularly cites—and thus fully relies on—published results that others provide. In the social and behavioral sciences, the main strategy to generate such empirical results has remained null-hypothesis significance testing (NHST). Recent attempts, however, broadly failed to replicate allegedly well-established result, giving rise to an ongoing “replication crisis.” The crisis entails that NHST retains too many errors; otherwise many more such effects should have replicated successfully. This makes trusting and citing even results that top-tier journals publish a true difficulty. On this background, we advocate the research program strategy (RPS)—employing both Frequentist NHST-elements and elements of Bayesian hypothesis-testing (BHT)—as superior to NHST. Data-simulation shows that RPS’s six steps (leading from a discovery against a random model to a statistically verified hypothesis) retain far fewer errors, and thus deserve far greater trust, than NHST- or BHT-results. We also show how to estimate the expectable error proportion among published results. A background in inferential statistics is helpful, but not necessary to profit from this talk. It is based, among others, on these publications:

Witte, E.H., and Zenker, F. (2017b). From discovery to justification. Outline of an ideal research program in empirical psychology. Frontiers in Psychology, 8, 1847 (ISSN 1664-1078). https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01847/full

Krefeld-Schwalb, A; Witte, E.H., and Zenker, F. (2018). Hypothesis-testing demands trustworthy data—a simulation approach to statistical inference advocating the research program strategy. Frontiers in Psychology, 9, 460 (ISSN 1664-1078). https://www.frontiersin.org/articles/10.3389/fpsyg.2018.00460/full

Witte, E.H., and Zenker, F. (2018). Data replication matters, replicated hypothesis-corroboration counts. (Commentary on “Making Replication Mainstream” by Rolf A. Zwaan, Alexander Etz, Richard E., Lucas, and M. Brent Donnellan). Behavioral and Brain Sciences, 41, e120 (ISSN: 0140-525X). https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/data-replication-matters-to-an-underpowered-study-but-replicated-hypothesis-corroboration-counts/90E7D8EFF04C42E0857CAA943DCBBBAC