Toward an Evidence-Based Education: A Systematic Review of Pedagogy in Speech-Language Pathology Thomas W. Powell and Mary Pannbacker Department of Rehabilitation Sciences, Speech-Language Pathology Program LSU Health Sciences Center–Shreveport, Shreveport, LA, 71103-4601, USA Introduction Evidence-based practice is a guiding principle of speech-language pathology (SLP), as evidenced by numerous publications (e.g., Baker & McLeod, 2004; Lass & Pannbacker, In press; Reilly, Douglas, & Oates, 2003; Schlosser & Raghavendra, 2004; Vallino-Napoli & Reilly, 2004). The philosophy underlying evidence-based practice has implications for professional education, and emerging challenges to professional education (e.g., McAllister, 2005) have prompted a call for pedagogical studies of SLP (Kellum, 2007). This systematic review represents an initial step toward an evidence-based approach to professional education (Davies, 1999; Moran & Malott, 2004). The goals of this poster session are: (a) To document a methodology for systematic review, (b) To identify trends and challenges to pedagogic research, and (c) To encourage creative methodologies for studying professional education. Method This systematic review followed the model of Pai and colleagues (2004). First, a literature search was initiated using MEDLINE and CINAHL databases. The MEDLINE subject heading Speech-Language Pathology / ed [Education] returned 356 hits, whereas the CINAHL subject heading Education, Speech-Language Pathology returned 238 hits. Next, citation data were downloaded to a reference manager (EndNote®) to facilitate management. Duplicate entries (N= 53) were deleted, leaving 541 unique references. A supplemental search, using traditional methodologies, identified 182 additional citations for a total of 723. Results Evidence · Level IV. More than half of the 416 publications relied upon anecdotal evidence or opinions of authorities. These papers included philosophical discussions and descriptions of educational programs and curricula. · Level III. Nearly 40% of the papers provided non- experimental evidence, usually in the form of survey data. · Levels II and I. Quasi-experimental and true experimental designs accounted for less than 10% of this sample. The ratio of experimental to non-experimental publications was explored. For each decade, the number of Level I and II publications was divided by the sum of Level III and IV papers. This index peaked in the 1980s, with a value of .14; thereafter, the ratio decreased to .08 (1990s) and .10 (2000s). Journals Ten journals accounted for more than 75% of the sample. These journals included publications of the US, UK, and Australian associations, as well as the IALP. All other journals combined accounted for less than 25% of the sample. Asha, alone, published 28% of the studies; no other journal published more than 10% of the sample. Productivity There has been a steady increase in the number of pedagogical publications. On average, the number of publications per year increased from fewer than two (1960s), to 8.7 (1980s), and to 17.1 in the current decade. Topic Prior to 1990, most studies focused on clinical teaching. Since 1990 there have been increasing numbers of publications on student attributes, diversity, communication sciences, and specific disorders (especially voice, AAC, fluency, and language). In the 2000s, many international descriptions of professional education have been published. Discussion · Increases were documented in the number of publications and the diversity of topic, suggesting an emerging scholarship of teaching (Boyer, 1990). · Although the number of publications has increased, the ratio of experimental to non-experimental designs has decreased since the 1980s. · Through the 1990s, Asha was the leading outlet for pedagogical research in SLP. It was replaced by the ASHA Leader, which does not solicit manuscripts for peer review. Also, the tabloid format makes the ASHA Leader less likely to be archived in libraries, although selected articles are available through the ASHA web site. References American Speech-Language-Hearing Association. (2005). Evidence-based practice in communication disorders [Position statement]. http://www.asha.org/members/deskref-journals/deskref/default Baker, E., & McLeod, S. (2004). Evidence-based management of phonological impairment in children. Child Language Teaching and Therapy, 20, 265-285. Boyer, E.L. (1990). Scholarship reconsidered: Priorities of the professoriate. Carnegie Foundation for the Advancement of Teaching. San Francisco, CA: Jossey-Bass. Davies, P. (1999). What is evidence-based education? British Journal of Educational Studies, 47, 108-121. Kellum, G. D. (February, 2007). Models of Training from Other Professions: Implications for the Future. Paper presented at The Subject is Change: Creating a Vision for the Future Education of Speech- Language Pathologists, New Orleans, LA. Lass, N., & Pannbacker, M. (In press). The application of evidence-based practice to oral-motor treatment. Language, Speech, and Hearing Services in Schools. McAllister, L. (2005). Issues and innovations in clinical education. Advances in Speech-Language Pathology, 7, 138-148. Moran, D.J., & Malott, R.W. (2004). Evidence-based educational methods. San Diego, CA: Elsevier. Pai, M., McCulloch, M., Gorman, J.D., Pai, N., Enanoria, W., Kennedy, G., Tharyan, P., & Colford, J.M. (2004). Systematic reviews and meta-analyses: An illustrated, step-by-step guide. The National Medical Journal of India, 17, 86-95. Reilly, S., Douglas, J., & Oates, J. (Eds.). (2004). Evidence-based practice in speech pathology. London: Whurr. Schlosser, R. W., & Raghavendra, P. (2004). Evidence-based practice in augmentative and alternative communication. Augmentative and Alternative Communication, 20, 1-21. Scottish Intercollegiate Guidelines Network. (2002). Glossary of key terms. http://www.sigh.ac.uk Vallino-Napoli, L.D., & Reilly, S. (2004). Evidence-based health care: A survey of speech pathology practice. Advances in Speech-Language Pathology, 6, 107-112. 0 5 10 15 20 25 30 35 40 1960 1961 1965 1967 1969 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 IV III IIb IIa Ib 1% 3% 5% 39% 52% Ib IIa IIb III IV Inclusion and exclusion criteria were applied prior to analysis. In short, the articles under study were: (a) published in peer-reviewed journals, (b) focused on SLP education, and (c) published in English. Some references (N=36) could not be retrieved; exclusion criteria eliminated 271 additional references. The remaining 416 publications were reviewed and coded to document: (a) the nature of the publication (e.g., philosophical, empirical), (b) the research design (qualitative, survey, etc.), (c) the topic (e.g., clinical teaching, voice disorders), (d) findings, and (e) level of evidence. Evidence coding was based on AS(A’s ȋ2005Ȍ adaptation of the Scottish Intercollegiate Guidelines Network (2002) system. To facilitate coding, levels of evidence were operationalized as a flowchart (see figure 1). Reliability was established by having a second researcher code 10% of the sample. Figure 2 Distribution of Relevant Articles (N = 416) by Level of Evidence Figure 3 Publications broken down by level of evidence and year. Level Ia Level Ib Level IIa Level IIb Level III Level IV examine method Meta Analysis? Control Group? Data Based? Randomly Assign? Quasi Exper? Figure 1 Evidence-based practice Levels of evidence Classification flowchart yes no no no yes yes yes yes no no