ELSEVIER DISCOURSE ANALYSIS PROCEDURES: RELIABILITY ISSUES KAREN HUX, DIXIE SANGER, ROBERT REID. and AMY MASCHKA Department of Special Education and Communication Disorders, UniversiO' of Nebraska-Lincoln Performing discourse analyses to supplement assessment procedures and facilitate interven- tion planning is only valuable if the observations are reliable. The purpose of the present study was to evaluate and compare four methods of assessing reliability on one discourse analysis procedure--a modified version of Damico's Clinical Discourse Analysis (1985a, 1985b, 1992). The selected methods were: (a) Pearson product-moment correlations, (b) in- terobserver agreement, (c) Cohen's kappa, and (d) generalizability coefficients. Results showed high correlation coefficients and high percentages of interobserver agreement when error type was not taken into account. However, interobserver agreement percentages ob- tained solely for target behavior occurrences and Cohen's kappa revealed that much of the agreement between raters was due to chance and the high frequency of target behavior non- occurrence. Generalizability coefficients revealed that the procedure was fair to good for discriminating among persons with differing levels of language competency for some as- pects of communication performance but was less than desirable for others; the aggregate score was below recommended standards for differentiating among people for diagnostic purposes. © 1997 by Elsevier Science Inc. Educational Objectives: The readers will understand the strengths and weakness of lbur methods of assessing reliability and will become aware of the importance of reliability when using discourse analysis procedures for diagnostic purposes. INTRODUCTION The popularity of obtaining and analyzing samples of children's discourse as part of language assessment procedures has steadily increased over the past several years (Gallagher, 1983; Owens, 1995; Stickler, 1987). This popularity is attributable, in part, to the inherent disadvantages of many standardized lan- guage measures (Owens, 1995; Damico, 1985a; Naremore, Densmore, & Har- man, 1995). Put succinctly, discourse analyses are important because stan- dardized tests alone do not provide sufficient information about how children use language. However, despite their potential contribution, discourse analy- Address correspondenceto Karen Hux, Ph.D., 318E Barkley Memorial Center, University of Ne- braska-Lincoln, Lincoln, NE 68503-0738. J. COMMUN. DISORD. 30 (1997), 133-150 © 1997 by Elsevier Science Inc. 655 Avenue of the Americas, New York, NY 10010 0021-9924/97/$17.00 Pll S0021-9924(96)00060-3