Can the Replication Rate Tell Us About Publication Bias?
A leading explanation for widespread replication failures is publication bias. I show in a simple model of selective publication that, contrary to common perceptions, the replication rate is unaffected by the suppression of insignificant results in the publication process. I show further that the expected replication rate falls below intended power owing to issues with common power calculations. I empirically calibrate a model of selective publication and find that power issues alone can explain the entirety of the gap between the replication rate and intended power in experimental economics. In psychology, these issues explain two-thirds of the gap.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Identification of and correction for publication bias
Some empirical results are more likely to be published than others. Such selective publication leads to biased estimates and distorted inference. This paper proposes two approaches for identifying the conditional probabi…
Replication Markets: Results, Lessons, Challenges and Opportunities in AI Replication
The last decade saw the emergence of systematic large-scale replication projects in the social and behavioral sciences, (Camerer et al., 2016, 2018; Ebersole et al., 2016; Klein et al., 2014, 2018; Collaboration, 2015). …
Publication Bias in Asset Pricing Research
Researchers are more likely to share notable findings. As a result, published findings tend to overstate the magnitude of real-world phenomena. This bias is a natural concern for asset pricing research, which has found h…
Testing for Underpowered Literatures
How many experimental studies would have come to different conclusions had they been run on larger samples? I show how to estimate the expected number of statistically significant results that a set of experiments would …
Towards Assessing Data Replication in Music Generation with Music Similarity Metrics on Raw Audio
Recent advancements in music generation are raising multiple concerns about the implications of AI in creative music processes, current business models and impacts related to intellectual property management. A relevant …
ManagementMusic Generation