Bitte verwenden Sie diesen Link, um diese Publikation zu zitieren, oder auf sie als Internetquelle zu verweisen: https://hdl.handle.net/10419/283318 
Erscheinungsjahr: 
2024
Schriftenreihe/Nr.: 
I4R Discussion Paper Series No. 102
Verlag: 
Institute for Replication (I4R), s.l.
Zusammenfassung: 
A typical empirical study involves choosing a sample, a research design, and an analysis path. Variation in such choices across studies leads to heterogeneity in results that introduce an additional layer of uncertainty not accounted for in reported standard errors and confi dence intervals. We provide a framework for studying heterogeneity in the social sciences and divide heterogeneity into population heterogeneity, design heterogeneity, and analytical heterogeneity. We estimate each type's heterogeneity from multi-lab replication studies, prospective meta-analyses of studies varying experimental designs, and multi-analyst studies. Our results suggest that population heterogeneity tends to be relatively small, whereas design and analytical heterogeneity are large. A conservative interpretation of the estimates suggests that incorporating the uncertainty due to heterogeneity would approximately double sample standard errors and confi dence intervals. We illustrate that heterogeneity of this magnitude-unless properly accounted for-has severe implications for statistical inference with strongly increased rates of false scientifi c claims.
Dokumentart: 
Working Paper

Datei(en):
Datei
Größe
3.19 MB





Publikationen in EconStor sind urheberrechtlich geschützt.