SPEECH TRAINING FOR DEAF AND HEARING- IMPAIRED PEOPLE ISAEUS consortium Ramón García Gómez 1 (project coordinator), Ricardo López Barquilla 1 , José Ignacio Puertas Tera 1 , José Parera Bermudez 1 , Marie-Christine Haton 2 , Jean Paul Haton 2 , Pierre Alinat 4 , Sofia Moreno 3 , Wolfgang Hess 5 , Mª Araceli Sanchez Raya 6 , Eduardo Alberto Martínez Gual 6 , Juan Luis Navas- Chaveli Daza 6 , Christophe Antoine 2 , Marie-Madeleine Durel 7 , Genevieve Maugin 8 , Silke Hohmann 5 1: UPM, 2: LORIA, 3: Teima Audiotex, 4:Thomson Marconi Sonar 5:U.Bonn, 6: Promi, 7: Cetre Hospital Regional de Nancy, 8: Urapeda. ramong@gtd.ssr.upm.es. Abstract In this paper we will describe the results of the ISAEUS project (TIDE DE 3004) achieved until now. The objective of this project is to develop a state of the art, cost effective, prototype for training deaf people in German, French and Spanish languages. We describe the user needs as understood for the users of ISAEUS: deaf persons and speech therapist. The paper also includes a brief description of the main blocks of the system: the acoustic-phonetic analyser in which will be based the exercises, the set of exercises that will be included, other features of the system and the real implementation. The paper will finalise with the description of the strategies for evaluating the effectiveness of this tool. Introduction In spite of the widely recognised necessity of having visualisation tools for representing speech that can help the deaf people to speak with good intelligibility and quality (by using visual feedback instead of the natural auditive feedback) the problem remains basically unsolved. The ISAEUS project addresses this problem. We are implementing a system for training deaf or partially deaf persons. The first point that we have to realise is that the hearing and seeing have neither the same effective value nor the significance for the haring impaired persons as for normal hearing persons. This fact has a profound impact: the sign language is the natural language of the deaf people. The sign language allows deaf people with major deafness to follow a normal evolution. We must recognise the strong interdependence between thinking and language. Consequently, the problem must be stated in a proper way: we have to address the problem of optimisation of the oral language for deaf people as a way of relating them with the oral speaking community. We need to combine sign, written and spoken languages in a tool based on the standard PC that could be useful for deaf and partially deaf persons. The three languages will be used for providing information to the deaf. The system must also be accessible to a wide range of ages: for children (the population most flexible within the deaf community) and for adults who have been oralised and who have the willingness of correcting their voices. The motivation of the user is an important requirement in order to maintain the attention enough time for obtaining advances in the intelligibility and quality of their voices. See the resume of the User’s Needs. Speech training of deaf persons is language dependent. In this sense one of the ISAEUS purposes is to implement exercises in three different languages: German, French and Spanish and compare the results. In this paper we are going to describe the exercises, classified in six different levels, as envisaged by ISAEUS in the three languages. All the exercises will be based on the analysis of the speech signal by mean of the Speech Analyser, which will be described in the following section. The Speech Analyser provides a wide set of parameters both categorical and non- categorical. For each lesson we select from this set of parameters the more relevant ones to compare speech with the target schemes. In order to provide the visual feedback we need to represent the multidimensional time varying information about speech. The visual representation given by ISAEUS must be intelligible for the deaf people: after the sensibilisation phase the users will be able to modify their voices in order to achieve the target objectives. Different alternatives of mapping the multidimensional time varying information into a graphical representation will be given and how to provide the feedback advises for the deaf. The paper will conclude with the hardware and software description of ISAEUS system and the strategies for carry out the corresponding evaluation of the effectivity of ISAEUS as a training tool for the deaf people. User’s Needs The user needs were defined based on three main sources of knowledge: the deaf people themselves, the speech therapists (represented in the ISAEUS project as three associations of deaf people), and commercial available systems (Speech viewer, VISHA and specially the SIRENE II).