© 2010 THE AUTHORS 844 BJU INTERNATIONAL © 2 0 1 0 B J U I N T E R N A T I O N A L | 1 0 8 , 8 4 4 – 8 5 0 | doi:10.1111/j.1464-410X.2010.09896.x 2010 THE AUTHORS; BJU INTERNATIONAL 2010 BJU INTERNATIONAL Urological Oncology RISK-ADJUSTED FUNNEL PLOTS FOR CYSTECTOMY OUTCOMES MAYER ET AL. What is the role of risk-adjusted funnel plots in the analysis of radical cystectomy volume–outcome relationships? Erik K. Mayer* † , Alex Bottle ‡ , Paul Aylin ‡ , Ara W. Darzi*, Justin A. Vale † and Thanos Athanasiou* *Division of Surgery, Department of Surgery and Cancer, Imperial College London, † Department of Urology, Imperial College Healthcare NHS Trust, St Mary’s Hospital, and ‡ Dr Foster Unit, School of Public Health, Imperial College London, London, UK Accepted for publication 1 September 2010 model three (case mix and ‘clustering’ of patients) and model four (additional adjustment for institutional structural and process-of-care variables). RESULTS • In the final complex model (model four), no Trusts had abnormally high mortality or re-intervention rates. • Comparison of the funnel plots showed the importance of adjusting for certain confounding factors, such as the surgeon, at the institutional level, before they could be labelled as having truly outlying performance. CONCLUSION • Risk-adjusted funnel plots have a useful role to play as a component of a methodological framework for investigating the volume–outcome relationship at the institutional level. They can act as a complementary method of validating data by displaying disaggregated outcomes at provider level and account for unmeasured confounders, so reducing the opportunity for spurious labelling of outliers. KEYWORDS funnel plots, outcome assessment (healthcare), radical cystectomy Study Type – Therapy (outcomes research) Level of Evidence 2b What’s known on the subject? and What does the study add? The use of funnel plots has helped to overcome the limitations and risks of ranking surgical performance. The case for a more widespread application of funnel plots in assessing and reporting performance in surgery has been made. No study has previously published a funnel plot analysis of outcomes for radical cystectomy in England. No Trust, using the final complex model for risk-adjustment, can be confidently said to have a performance worse than the national average for both mortality and re- intervention rates following radical cystectomy. Funnel plots act as a complementary method of validating volume-outcome data by displaying disaggregated outcomes at a provider level and reduce the opportunity for spurious labelling of outliers. OBJECTIVE • To explore whether risk-adjusted funnel plots are a useful adjunct to analyse volume–outcome data and to further facilitate our understanding of institutional performance data by combining funnel-plot methodology with an incremental statistical modelling approach. PATIENTS AND METHODS • Risk-adjusted funnel plots were generated for mortality and re-intervention rates after elective radical cystectomy using administrative data from NHS Hospital Trusts between 2000/01 and 2006/07. Trusts were divided into volume tertiles based on their average annual cystectomy rate. • A funnel plot was produced for each of the following four incremental statistical models: model one (no adjustment), model two (adjusted for patient case mix variables), INTRODUCTION Transparency of performance and outcome data continues to be an enabler for improving quality in healthcare [1]. The use of improved statistical techniques to overcome the limitations and risks of ranking performance using ‘league tables’ has helped to engage healthcare professionals in this process [2]. Displaying performance data using funnel plots has advantages over conventional bar graphs and caterpillar plots [3]. Funnel plots for event rates are constructed by plotting the rate against the volume of cases and therefore they have the added advantage of simultaneously interpreting surgical performance and informally assessing the presence of a volume– outcome relationship [3,4]: the theory that providers undertaking greater numbers of particular operations achieve improved surgical outcomes over ‘low-volume’ providers. The use of an incremental statistical model approach has shown the benefits of adjusting volume–outcome relationship data for structural and process-of-care factors and considering its hierarchical nature by co- BJUI BJU INTERNATIONAL