1 Identification of Relevant Studies, Data Extraction and Detailed Analyses Related to the Paper Entitled ‘Synthesizing Research Findings in Software Engineering: Problems with Effect sizes and Meta-Analyses’ Barbara Kitchenham, Member, IEEE, Lech Madeyski, Member, IEEE, Abstract This is an online appendix to our journal paper on synthesizing research findings in software engineering in general and problems with effect sizes and meta-analyses in particular. Index Terms meta-analysis, research synthesize, moderator analysis, research design, experimental design, software engineering ✦ 1 Identification of other relevant studies Before re-doing Ciolkowski’s meta-analysis [1] to investigate the impact of experimental design, we believe it is necessary to ensure that all relevant studies have been identified and included in the analysis. Ciolkowski did not report how he identified the set of studies he included in his meta-analysis, furthermore the most recent study in the meta-analysis was published in 2006. For this reason we performed a systematic review to check the completeness of the set of primary studies. The goal of this systematic review was to search for additional relevant papers comparing team defect effectiveness of PBR with CBR or an Ad-hoc reading method. We describe our search methods and results below. 1.1 Search Strategy Our search strategy can be described in terms of a search method, inclusion and exclusion criteria and validation of the proposed search process. 1.1.1 Planned Search Process Given the existing set of known studies, we decided to undertake a citation-based (i.e. snowballing-based) search including a backwards search of all the references in the set of known papers recognized by the Scopus digital indexing system and a forward search identifying all studies that cited the identified known papers. We believe this to be an appropriate search strategy because, the set of known papers includes two papers co-authored by Basili (i.e. [2] and [3]), who pioneered the empirical investigations of PBR and it would be highly unlikely that other researchers reporting such studies would fail to reference at least one of Basili’s papers on the topic. We chose the Scopus system for our search because it had appropriate citation management facilities. • Barbara Kitchenham is with the School of Computing and Mathematics, Keele University, Stoke-on-Trent, UK. • Lech Madeyski is with the Institute of Informatics, Wroclaw University of Technology, Poland. E-mail: lech.madeyski@pwr.wroc.pl. WWW: http://madeyski.e-informatyka.pl/ Report SPR3/2014 (http://madeyski.e-informatyka.pl/download/app/KitchenhamMadeyskiA14.pdf) related to the paper by Kitchenham and Madeyski