Technology and Disability 18 (2006) 99–105 99 IOS Press Mobile phone video as an aid to speech understanding for persons with hearing impairment Bj¨ orn Lidestam a,∗ , Henrik Danielsson a,b and Ther´ ese L ¨ onnborg a a Department of Behavioural Sciences, Link ¨ oping University, Sweden b The Swedish Institute for Disability Research, Sweden Abstract. This study investigated if a mobile video telephone may facilitate distance communication for persons with impaired hearing, and how users’ speech recognition performance is associated with difficulty ratings. Ten persons with impaired hearing participated. A mobile video telephone was used to record sentences and visual-contextual cues. Sentences were presented visually, auditorily, and audiovisually; and both with and without visual-contextual cues. The results showed that audiovisual presentation modality and visual-contextual cues together significantly enhanced speech recognition, and that both audiovisual presentation modality and visual-contextual cues by themselves gave tendencies toward significance. Also, speech recognition and difficulty ratings were significantly correlated over all conditions. It was concluded that mobile video telephones may today already enhance speech recognition for individuals with impaired hearing, but for substantial effects, audiovisual synchronization needs to be improved. Keywords: Hearing impairment, hearing, speechreading, cues, communication aids 1. Introduction The primary purpose of the present study was to in- vestigate if a mobile video telephone may facilitate dis- tance communication for persons with impaired hear- ing, in comparison to how they perceive speech over a regular (non-video) telephone. A secondary purpose was to investigate the association between objective speech recognition performance and subjective diffi- culty ratings. Hearing impairment is common. For example, 13% of the Swedish population have some degree of hear- ing impairment [15]. The most common type of hear- ing impairment is sensorineural hearing impairment, which may be caused by degenerative processes such ∗ Address for correspondence: Bj¨ orn Lidestam, Department of Behavioural Sciences, Link¨ oping University, SE-581 83 Link¨ oping, Sweden. Tel.: +46 (0)13 282155; Fax: +46 (0)13 282145; E-mail: bjli@ibv.liu.se. as ageing and exposure to noise [8]. In most cases, the sensorineural hearing impairment affects the abili- ty to sense and perceive sounds in the higher frequen- cies [8]. A person with sensorineural hearing loss may, hence, have problems distinguishing consonants with features in the upper formants. Thus, many individu- als with impaired hearing have problems with speech perception, and especially in noisy environments. As a consequence, talking over the telephone is difficult or impossible for many individuals with impaired hearing. However, individuals with hearing loss in the upper frequencies often experience that it is much easier to understand what someone is saying if they can see the person who is talking, rather than just hear the person. This is because some of the auditory speech features in the higher frequencies (specifically, some consonant features) are relatively easy to distinguish visually. For example, the consonant /f/ consists of sounds of rela- tively high frequency, but is one of the easiest conso- nants to identify visually [4,10]. Thus, visually per- ISSN 1055-4181/06/$17.00  2006 – IOS Press and the authors. All rights reserved