Journal of Industrial Engineering Research, 1(10) Special 2015, Pages: 1-7 IWNEST PUBLISHER Journal of Industrial Engineering Research (ISSN: 2077-4559) Journal home page: http://www.iwnest.com/AACE/ Corresponding Author: Nurul Afiqah Misran, Department of Computational and Theoretical Sciences, Faculty of Science, International Islamic University Malaysia,Kuantan Campus,25200 Kuantan, Pahang, Malaysia Tel: +6013 3634550; E-mail: fiqamis12@gmail.com An Empirical Assessment of Meta-Analysis Estimates from Multi- level Studies Nurul Afiqah Misran and Nik Ruzni Nik Idris Department of Computational and Theoretical Sciences, Faculty of Science, International Islamic University Malaysia,Kuantan Campus,25200 Kuantan, Pahang, Malaysia ARTICLE INFO ABSTRACT Article history: Received 3 August 2015 Accepted 28 October 2015 Available online 31 October 2015 Keywords: Meta-analysis;Combine data; Individual patient data; Aggregate data; Simulation; Bias Background: A conventional meta-analysis may be performed using studies which are available at individual patient level (IPD) or aggregate level (AD). Presently however, meta-analysis that combine the two levels of studies is increasingly common. The implications of utilising different levels of data on the overall estimates have not been fully explored. Objective: This study examined the efficacy of the estimates of overall treatment effect from AD, IPD and the mixed AD: IPD studies, and investigated how they differ from the true treatment effect. Additionally, this study investigated the influence of the ratio of AD: IPD on the precision of the overall treatment effects estimates. The bias, root mean-square-error (RMSE) and coverage probability were used to assess the efficiency of the overall estimates. Results: The results showed that the IPD meta-analysis produced better estimates in terms of RMSE compared to AD meta-analysis and the mixed AD:IPD meta-analysis. For the cases where both the AD and IPD studies were available, our findings showed that the combined AD : IPD data produced better estimates, in terms of precision, than utilising the AD alone. Conclusion: It is therefore recommended that available IPD should always be included in a conventional meta-analysis using summary level data as significant statistical benefit is gained by pooling the two levels of data. © 2015 IWNEST Publisher All rights reserved. To Cite This Article: Nurul Afiqah Misran and Nik Ruzni Nik Idris., An Empirical Assessment of Meta-Analysis Estimates from Multi- level Studies. J. Ind. Eng. Res., 1(10): 1-7, 2015 INTRODUCTION Meta-analysis is a statistical technique for integrating quantitative results from several sources. The main aim is to provide conclusions based on the whole body of research. A traditional meta-analysis involves integration of aggregate data (AD) which is extracted from the individual study publications [1]. Typical AD includes a mean difference for continuous outcomes or the number of events and participants for binary outcomes. The overall treatment effect is computed by taking the weighted average of the effects across the trials using methods such as the inverse variance method [2] or the Mantel-Haenszel method [3] for the binary data. Alternatively, meta-analysis may be performed based on individual patient data (IPD), where raw data from individual study is obtained and synthesized directly. Although it has numerous advantages compared to the traditional meta-analysis, particularly in terms of type of analyses that can be done, IPD meta-analysis is usually relatively costly and time consuming [4][5]. Another potential problem for IPD analysis is that IPD are seldom or may not be available from all the individual studies. Combining available IPD with the AD has been advocated [6][7] in order to maximized available information and allows larger number of patients and greater part of the evidence-based to be included. Currently however, combined-study level meta-analysis is not very common. A review of 199 meta-analyses [8] which has both the IPD and AD available, found 33 combined the data in their analysis and 166 did not. The review noted that the articles that combined the IPD and the AD in their studies had, on average, IPD available in 64% of the studies, while articles which did not combine the studies had an average of 90% IPD available. The implications of these ratios on the overall meta-analysis estimates are yet to be explored. A study [9] examined the models used for combining the IPD and AD in meta-analysis of continuous outcomes. In part of their study, estimates extracted from IPD-only (i.e. only IPD were utilized when both type are available) were compared with those from the combinations of both IPD and AD. The study was based on real data from a Hypertension study [10]. The results showed that the benefit of combining the IPD with AD increases as the proportion of IPD studies decreases. The comparison was made against the estimates from all IPD studies.