Vol.:(0123456789) 1 3 Aging Clinical and Experimental Research https://doi.org/10.1007/s40520-020-01542-y MINI REVIEW Statistical methods to assess the prognostic value of risk prediction rules in clinical research Graziella D’Arrigo 1  · Mercedes Gori 2  · Annalisa Pitino 2  · Claudia Torino 1  · Stefanos Roumeliotis 1,3  · Giovanni Tripepi 1 Received: 7 February 2020 / Accepted: 24 March 2020 © Springer Nature Switzerland AG 2020 Abstract Prognosis aims at estimating the future course of a given disease in probabilistic terms. As in diagnosis, where clinicians are interested in knowing the accuracy of a new test to identify patients afected by a given disease, in prognosis they wish to accurately identify patients at risk of a future event conditional to one or more prognostic factors. Thus, accurate risk predictions play a primary role in all felds of clinical medicine and in geriatrics as well because they can help clinicians to tailor the intensity of a treatment and to schedule clinical surveillance according to the risk of the concerned patient. Statistical methods able to evaluate the prognostic accuracy of a risk score demand the assessment of discrimination (the Harrell’s C-index), calibration (Hosmer–May test) and risk reclassifcation abilities (IDI, an index of risk reclassifcation) of the same risk prediction rule whereas, in spite of the popular belief that traditional statistical techniques providing rela- tive measures of efect (such as the hazard ratio derived by Cox regression analysis or the odds ratio obtained by logistic regression analysis) could be per se enough to assess the prognostic value of a biomarker or of a risk score. In this paper we provide a brief theoretical background of each statistical test and a practical approach to the issue. For didactic purposes, in the paper we also provide a dataset (n = 40) to allow the reader to train in the application of the proposed statistical methods. Keywords Prognostic research · Discrimination · Calibration · Risk reclassifcation analysis Introduction Prognosis, together with diagnosis and treatment, is one of the three decisional processes of clinical medicine, and a fundamental element of public health as well. Prognosis of a given patient over a pre-defned time period is generally done by prognostic biomarkers and/or risk prediction rules, the latter being mathematical combinations of multiple prognostic factors (i.e. biomarkers and/or other quantitative and qualitative variables such as age and gender), to be able to calculate the probability of a specifc outcome on indi- vidual basis. Furthermore, biomarkers and risk prediction rules should be intended as a support in clinical medicine and not be used alone. Before being adopted in daily clinical practice a candi- date risk prediction rule needs to be carefully developed in a representative patients’ cohort and externally validated in an independent series of patients afected by the same dis- ease. Finally, a randomized controlled clinical trial would be ideally needed to demonstrate that the allocation of patients to specifc treatments according to a given risk stratifca- tion tool leads to better outcomes as compared to those of patients allocated to a diferent risk stratifcation rule. The evaluation of the prognostic accuracy of a risk score demands the assessment of discrimination, calibration and risk reclassifcation abilities of the same prediction rule. In this paper, using a simulation study, we describe how to formally calculate the Harrell’s C-index (to assess discrimi- nation) [1], the Hosmer–May Test (an index of calibration) [2] and the integrated discrimination improvements (IDI, an Electronic supplementary material The online version of this article (https://doi.org/10.1007/s40520-020-01542-y) contains supplementary material, which is available to authorized users. * Giovanni Tripepi gtripepi@ifc.cnr.it 1 Institute of Clinical Physiology (IFC-CNR), Clinical Epidemiology and Physiopathology of Renal Diseases and Hypertension of Reggio Calabria, Ospedali Riuniti, Via Vallone Petrara snc, Reggio Calabria, Italy 2 Institute of Clinical Physiology (IFC-CNR), Rome, Italy 3 Division of Nephrology and Hypertension, 1st Department of Internal Medicine, AHEPA Hospital, School of Medicine, Aristotle University of Thessaloniki, Thessaloniki, Greece