rn 1 , Karrie Karahalios 1 , Laura DeThorne 2 , Jim Halle 3 University of Illinois at Urbana Champaign Joshua Hailp e 2 , Jim Halle 3 Joshua Hailpern 1 , Karrie Karahalios 1 , Laura DeTho University of Illinois at Urbana Champaign VocSyl: Visualizing Syllable Production for Children with ASD and Speech Delays Joshua Hailpe ern 1 , Karrie Karahalios 1 , Laura DeThor rne 2 , Jim Halle 3 University of Illinois at Urbana Champaign 1 Dept of Computer Science 2 Dept of Speech and Hearing Science 3 Dept of Special Education Urbana, IL 61801 Champaign, IL 61820 Champaign, IL 61820 [jhailpe2,kkarahal]@cs.uiuc.edu lauras@illinois.edu halle@illinois.edu 1. ABSTRACT Communication disorders occur across the lifespan and encom- pass a wide range of conditions that interfere with individuals’ abilities to hear (e.g., hearing loss), speak (e.g., voice disorders; motor speech disorders), and/or use language (e.g., specific lan- guage impairment; aphasia) to meet their communication needs. Such disorders often compromise the social, recreational, emo- tional, educational, and vocational aspects of an individual’s life. This research examines the development and implementation of new software that facilitates multi-syllabic speech production in children with autism and speech delays. The VocSyl software package utilizes a suite of audio visualizations that represent a myriad of audio features in abstract representations. The goal of these visualizations is to provide children with language impair- ments a new persistent modality in which to experience and prac- tice speech-language skills. ACM Classification Keywords H5.2 [Information Interfaces and Presentation]: Screen design, Voice I/O. K4.2 [Social Issues]: Assistive technologies for per- sons with disabilities General Terms Human Factors Author Keywords Autism, Visualization, Speech Delays, Syllables 1.INTRODUCTION & JUSTIFICATION Children’s speech-language impairments serve as one of the most prevalent early childhood developmental challenges with ap- proximately 24% of all children affected [2]. These impairments, combined with delays in social and interpersonal skills, can result in severe difficulty in communication and increased anxiety dur- ing human-to-human interaction [3]. The ability to combine sylla- bles, whether as word combinations (e.g., more juice) or as one multisyllabic word (e.g., doughnut), represents an important de- velopmental milestone that is delayed or impaired in a variety of clinical groups including children with apraxia of speech, autistic spectrum disorders (ASD), and children identified as late-talking Copyright is held by the author/owner(s). ASSETS’10, October 25–27, 2010, Orlando, Florida, USA. ACM 978-1-60558-881-0/10/10. toddlers [8,14]. In fact, challenges with multisyllabic words may represent a residual difficulty for some adolescents and adults with a history of phonological difficulties [5,13]. The importance of facilitating multisyllabic productions not only relates to speech intelligibility but also to optimizing language growth. This research focuses on the development and design of the VocSyl software package. Based on existing research, VocSyl utilizes abstract visualizations of voice to encourage and shape the vocalizations (specifically syllable production) of children with ASD and speech delays. We hypothesize that by changing vocali- zations and clinicians’ models from a modality that is amphoral and fleeting to one that is persistent and informative/reflective we can help increase the linguistic abilities of children struggling to combine syllables into words and phrases. 1.1.HCI and Voice Visualizations Since the 1990’s, the Human Computer Interaction (HCI) com- munity has examined how computers can aid in diagnosis of ASD [7]. HCI researchers have also studied audio perception [16], vo- calization of infants[4], and teaching of human-to-human interac- tion to high-functioning children with ASD [17]. Elements of play have also been studied demonstrating that technology/computers can reduce the apprehension caused by human-to-human interac- tion [10]. Our prior work [6,9] illustrates the potential of abstract visualizations of voice (pitch, volume, and duration) to increase the rate and duration of speech-like vocalizations in low function- ing children with ASD. However, [6,9] did not focus on shaping vocalizations. This research seeks to pick up where we had left off; using abstract visualizations of voice, to shape and teach word production in children with ASD and other speech delays. 2.VOCSYL The aim of this project is to develop, implement, and measure the effectiveness of software tools for facilitating multisyllabic speech production in children with speech-language disabilities, specifi- cally low and moderate functioning children with autism. In addi- tion to the novel software and empirical data we plan to generate, this cross-disciplinary project will provide an innovative and im- portant approach to developing treatment for children with speech and language disabilities. To this end, we have built VocSyl a software package that provides real-time visual feedback in re- sponse to vocal pitch, loudness, duration, and syllables (See Fig- ure 1). Such a re-interpretation of voice will allow for both a new understanding of one’s vocalization by audio/visual feedback and will also allow for a tangible comparison to models presented by clinicians. In other words, a child will be able to both see and hear 297