Assessing the value of diagnostic tests: a framework for designing and evaluating trials The value of a diagnostic test is not simply measured by its accuracy, but depends on how it affects patient health. This article presents a framework for the design and interpretation of studies that evaluate the health consequences of new diagnostic tests Lavinia Ferrante di Ruffano research fellow 1 , Christopher J Hyde professor of public health and clinical epidemiology 2 , Kirsten J McCaffery associate professor and principal research fellow 3 , Patrick M M Bossuyt professor of clinical epidemiology 4 , Jonathan J Deeks professor of biostatistics 1 1 Department of Public Health, Epidemiology, and Biostatistics, School of Health and Population Sciences, University of Birmingham, Birmingham B15 2TT, UK; 2 PenTAG, Institute for Health Services Research, Peninsula College of Medicine and Dentistry, University of Exeter, Exeter, UK; 3 Screening and Test Evaluation Program, School of Public Health, University of Sydney, Sydney, Australia; 4 Department of Clinical Epidemiology and Biostatistics, Academic Medical Centre, University of Amsterdam, Amsterdam, Netherlands Most studies of diagnostic tests evaluate only their accuracy. Although such studies describe how well tests identify patients with disease (sensitivity) or without disease (specificity), further evidence is needed to determine a test’s true clinical value. Firstly, since tests are rarely used in isolation, studies are needed that evaluate the performance of testing strategies, accounting for when and how a new test is used within a diagnostic pathway, and how its findings are combined with results of other tests. 1 Secondly, decision making involves selecting among multiple testing strategies; thus studies that compare test strategies and estimate differences in sensitivity and specificity are more informative than those that evaluate the accuracy of one test or diagnostic strategy. 2 Thirdly, improvements in test accuracy will not benefit patients unless they lead to changes in diagnoses and patient management, requiring evaluations of the effect of improved accuracy on decision making. 3 Finally, improved decision making is only one route by which tests affect patient health, and empirical evaluations are needed to compare the effect of test strategies on patient health. 4 Ideally, new tests should only be introduced into clinical practice if evidence indicates that they have a better chance of improving patient health than existing tests. 56 Tests can be compared by evaluating the downstream consequences of testing on patient outcomes, either directly in a randomised controlled trial or by decision analysis models that integrate multiple sources of evidence. Test-treatment trials randomly allocate patients to tests, follow up subsequent management, and measure outcomes only after treatment has been received (fig 1⇓). 7 Decision models use existing clinical data to extrapolate, through a number of assumptions, the link between intermediate outcomes (such as accuracy) and long term outcomes. 8 A key issue for trials and decision models is the selection of outcomes that need to be measured or modelled to evaluate how tests are affecting patients. This selection requires a priori knowledge of the mechanisms by which tests affect patient health. In this article, we provide a comprehensive review of the mechanisms that can drive changes to patient health from testing, and include a summary checklist to assist readers, researchers, and funders who wish to design or appraise studies evaluating diagnostic tests. We have based our framework on a review of a large cohort of published test-treatment trials 9 and key methodological literature. Effect of tests on patient health To establish whether a new diagnostic test will change health outcomes, it must be examined as part of a broader management strategy. Testing represents the first step of a test-treatment process: (1) a test is administered to identify a target condition, (2) the test result is considered (3) alongside other evidence to decide a diagnosis, and (4) a course of treatment is identified (5) and implemented (fig 2⇓). 10 Changes to any aspect of this pathway after the introduction of a new test could trigger changes in health outcomes. Table 1⇓ lists the mechanisms that commonly affect health outcomes. Direct test effects Test procedure Some diagnostic procedures carry a risk of harm, hence alternatives that offer reduced procedural morbidity will be of immediate benefit to patients. For example, use of sentinel lymph node biopsy rather than dissection of the axillary node Correspondence to: Jonathan J Deeks j.deeks@bham.ac.uk For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ 2012;344:e686 doi: 10.1136/bmj.e686 (Published 21 February 2012) Page 1 of 9 Research Methods & Reporting RESEARCH METHODS & REPORTING on 15 June 2020 by guest. Protected by copyright. http://www.bmj.com/ BMJ: first published as 10.1136/bmj.e686 on 21 February 2012. Downloaded from