Journal of Evaluation in Clinical Practice, 12, 5, 491–500 Journal of Evaluation in Clinical Practice 12 (2006) 491–500 © 2006 The Authors Journal compilation © 2006 Society/Blackwell Publishing Ltd 491 Blackwell Publishing LtdOxford, UKJEPJournal of Evaluation in Clinical Practice1356-1294© 2006 The Authors; Journal compilation © 2006 Society/Blackwell Publishing Ltd 2006125491500Original ArticleMeasuring physiotherapists’ guideline adherenceG.M.J. Rutten et al . Correspondence Geert Rutten Centre for Quality of Care Research (WOK) University Medical Centre Nijmegen Radboud Geert Grooteplein Noord 21 Route 114 117 KWAZO PO Box 9101 6500 HB Nijmegen the Netherlands E-mail: G.Rutten@kwazo.umcn.nl; gh.rutten@home.nl Keywords: guideline-adherence, low back pain, physiotherapy, validation studies Accepted for publication: 21 October 2005 Measuring physiotherapists’ guideline adherence by means of clinical vignettes: a validation study Geert M.J. Rutten MPH, 1,2 Janneke Harting PhD, 3 Stephen T.J. Rutten MPH, 4 Geertruida E. Bekkering PhD 5 and Stef P.J. Kremers PhD 3 1 Junior researcher, Centre for Quality of Care Research (WOK), UMC St Radboud, Nijmegen, the Netherlands 2 Physiotherapist, Practice for physiotherapy and manual therapy, Uden, the Netherlands 3 Postdoctoral research, Department of Health Education and Promotion, Maastricht University, Maastricht, the Netherlands 4 Physiotherapist, Praxis für Krankengymnastik und Manuelle Therapy, Aachen, Germany 5 Research associate, Department of Social Medicine, University of Bristol, Bristol, UK Abstract Rationale, aims and objectives To assess the criterion validity of paper- and-pencil vignettes to assess guideline adherence by physiotherapists in the Netherlands. The evidence-based physiotherapy practice guideline for low back pain was used as an example. Methods Four vignettes were con- structed and pre-tested. Three vignettes were found to represent an ade- quate case-mix. They described one patient with specific low back pain, one with non-specific low back pain and a normal recovery process and one with non-specific low back pain and a delay in the recovery process. Invited to participate were 113 primary care physiotherapists who had joined an ran- domized controlled trial study 8 months before, in which guideline adher- ence had been measured by means of semi-structured treatment recording forms. The criterion validity was determined with Spearman’s r s , using Cohen’s classification for the behavioural sciences to categorize its effect size. Results Of the 72 physiotherapists who agreed to participate, 39 com- pleted the questions on the vignettes. In the end, both adherence measures were available for 34 participants, providing 102 vignettes and 268 record- ing forms. Mean guideline adherence scores were 57% (SD = 17) when measured by vignettes and 74% (SD = 15) when measured by recording forms. Spearman’s r s was 0.31 (P = 0.036), which, according to Cohen’s clas- sification, is a medium effect size. Conclusion Vignettes are of acceptable validity, and are an inexpensive and manageable instrument to measure guideline adherence among large groups of physiotherapists. Further vali- dation studies could benefit from the use of standardized patients as a gold standard, a more diverse case mix to better reflect real physiotherapy prac- tice, and the inclusion of longitudinal vignettes that cover the patients’ course of treatment. Introduction Practice guidelines play an important role in improv- ing the quality of physiotherapists’ practice (Bekker- ing et al. 2000). They create opportunities to systematically apply scientific evidence in practice (Hendriks et al. 1998; De Visser et al. 2001; Grol 2001), and to improve and monitor the efficiency and quality of physiotherapists’ performance (Bekkering et al. 2000). In addition, guidelines are assumed to