ORIGINAL ARTICLE Staged decision making was an attractive alternative to a plenary approach in panel diagnosis as reference standard Loes C.M. Bertens a, * , Yvonne van Mourik a , Frans H. Rutten a , Maarten-Jan M. Cramer b , Jan-Willem J. Lammers c , Arno W. Hoes a , Johannes B. Reitsma a , Karel G.M. Moons a a Julius Center for Health Sciences and Primary Care, University Medical Center, Heidelberglaan 100, 3584 CX Utrecht, The Netherlands b Division of Heart and Lungs, Department of Cardiology, University Medical Center Utrecht, The Netherlands c Division of Heart and Lungs, Department of Pulmonology, University Medical Center Utrecht, The Netherlands Accepted 29 September 2014; Published online xxxx Abstract Objectives: To assess differences between three different decision-making approaches in the method of panel diagnosis as reference standard in diagnostic research. Study Design and Setting: Within a diagnostic study, the prevalence of heart failure (HF) and chronic obstructive pulmonary disease (COPD) was compared using three approaches of decision making in panel diagnosis. These were (1) a plenary discussion among experts followed by a consensus decision (plenary); (2) a predefined decision rule based on final diagnoses made by each member individually (individual); and (3) a staged procedure in which first the final diagnosis per individual member is generated followed by a plenary dis- cussion of those cases with disagreement (staged). Results: Prevalence of HF and COPD according to plenary approach was 46% and 28%, respectively. Individual approach diagnosed 28% of patients with HF and 31% with COPD and revealed 28 and 8 discordant diagnoses, respectively, compared with plenary approach. Staged approach revealed a prevalence of 43% and 28% for HF and COPD, respectively, with eight discordant diagnoses for HF and none for COPD. Conclusion: The staged approach is an attractive choice as it produces very similar results to the full plenary approach, while having the advantage of being less time consuming. Additionally, it provides insights into the decision-making process of the panel, and the ‘‘diffi- cult-to-diagnose’’ patients can easily be identified. Ó 2014 Elsevier Inc. All rights reserved. Keywords: Diagnosis; Reference standards; Prevalence; Sensitivity and specificity; Heart failure; COPD 1. Introduction A key challenge in any diagnostic study is to obtain a correct final diagnosis in all included patients. Ideally, a single, error-free reference test can be used to classify the condition of interest. For most conditions, however, such a single, error-free reference standard is not available. This is problematic, as errors in the final disease classification can seriously bias the results [1e3]. A panel diagnosis, in which multiple experts combine the results from different tests, sometimes also including patient follow-up, to decide on the presence or absence of the disease or condition under study, is an attractive option to overcome the absence of an error-free reference standard [2e5]. When dealing with the absence of a perfect reference stan- dard by using a panel diagnosis, several approaches of deci- sion making exist within this method. A recent systematic review on the methods used in panel diagnoses revealed that, in general, two approaches are used to decide on the final diagnosis in each patient: (1) the results of the different diag- nostic tests and all other relevant information are discussed among all experts during consensus meetings and followed by a consensus diagnosis and (2) an a priori determined rule (typically a majority rule) based on the final diagnoses made by each panel member individually is used to reach a final Funding: The study was conducted as part of the Dutch National Care for the Elderly Program (ZonMw-NPO) and funded by a research grant from the Netherlands Organization for Health Research and Development (ZonMw grant 311040302). K.G.M.M. received funding from the Netherlands Organisation for Scientific Research (project 9120.8004 and 918.10.615). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Conflict of interest: None. * Corresponding author. Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, PO Box 85060, Stratenum 6.131, 3508 AB Utrecht, The Netherlands. Tel.: þ31-88-756-9415; fax: þ31-88-756-9028. E-mail address: L.C.M.Bertens-2@umcutrecht.nl (L.C.M. Bertens). http://dx.doi.org/10.1016/j.jclinepi.2014.09.020 0895-4356/Ó 2014 Elsevier Inc. All rights reserved. Journal of Clinical Epidemiology - (2014) -