More replications do get published on top 5 general interest economics journal articles 1 Jan H. Höffler 2 , Francisco Rosales Marticorena 3 University of Göttingen Centre of Statistics Replication Working Paper 1/2020 October 6, 2020 In their 2019 study “Replication studies in economics — How many and which papers are chosen for replication, and why?”, the authors conclude that “The replication probability is lower for articles published in top 5 economics journals. ” This result is based on missampling. The authors’ own data shows that replication studies published in the top 50 economics journals refer clearly more often to studies published in the top 5 general interest journals in economics than to the other 45 economics journals. JEL classification: A14, B4, C12, C13, C83 Keywords : Replication, Sample selection biases, Economics of science, Science policy, Economic methodology Contribution The authors make a valuable contribution by identifying a number of replication studies published in 50 leading economics journals in the years 1974-2014 and sharing their data on them. Even though the topic of the article is replication, they do not make available the full list of studies they use for their regression analysis or the code they use for their analysis. Even without this information, one can see an important flaw in the analysis. Importance of the correction The relevance of correcting the study is twofold: From a methodological point of view, it should be avoided to establish an erroneous approach in the literature. Other researchers could follow the wrong example. From a policy perspective, the wrong result in this particular case could lead to wrong and harmful conclusions: One might be misled to follow the authors ’ reasoning that the top general interest journals apparently have better reviewers and thus publish studies with less flaws. This could make researchers think that they should preferably cite these journals because they seem to have been shown to be more reliable. However, in fact the original study did not show any evidence supporting such conclusions. Flaw The authors argue that top journals have better reviewers, thus publish less flawed papers and thus replications of their studies are less frequently published in leading journals. The evidence they present for this in their tables 3-5 is driven by the way they reduce their full sample of 126.505 studies published in 50 top economics journals in the years 1974 to 2014. For their analysis, they looked at articles for which they found a replication published in the 50 chosen journals in the observation period. They then look at all articles published in the journal issues in which these replicated studies 1 Acknowledgement: We thank Rafael Becerril for his very helpful comments on this manuscript. 2 ReplicationWiki, EQ-Lab 3 Universidad ESAN