Investigating the impact of lip visibility and talking style on speechreading performance Karen Lander ⇑ , Cheryl Capek School of Psychological Sciences, University of Manchester, Oxford Road, Manchester M13 9PL, UK Received 26 October 2012; received in revised form 18 December 2012; accepted 15 January 2013 Available online 30 January 2013 Abstract It has long been known that visual information from a talker’s mouth and face plays an important role in the perception and under- standing of spoken language. The reported experiments explore the impact of lip visibility (Experiments 1 & 2) and speaking style (Exper- iment 2) on talker speechreadability. Specifically we compare speechreading performance (words in Experiment 1; sentences in Experiment 2 with low level auditory input) from talkers with natural lips, with brightly coloured lips and with concealed lips. Results reveal that highlighting the lip area by the application of lipstick or concealer improves speechreading, relative to natural lips. Further- more, speaking in a clear (rather than conversational) manner improves speechreading performance, with no interaction between lip vis- ibility and speaking style. Results are discussed in relation to practical methods of improving speechreading and in relation to attention and movement parameters. Ó 2013 Elsevier B.V. All rights reserved. Keywords: Speechreading, Speechreadability; Lip visibility, Speaking style 1. Introduction Visual information from a talker’s mouth and face plays an important role in the perception and understanding of spoken language (Massaro, 1998; Summerfield, 1992). Under noisy conditions, viewing the talking face supple- ments the auditory signal, increasing perceptual accuracy (Sumby and Pollack, 1954). Furthermore, we make use of visual information from the face even during the under- standing of clear and unambiguous speech (Reisberg et al., 1987). Finally, there is evidence that access to visual speech is necessary for normal speech development (Mills, 1987). The use of the visual speech signal, or speechreading is an important aid to communication, both within the hear- ing population and for the estimated nine million deaf and hearing-impaired individuals living in the UK (Valentine and Skelton, 2008). For speechreading, the lip and mouth area is of huge importance. Indeed automatic speechread- ing by machines is typically based on a robust lip image analysis (Vogt, 1997), often including modelled informa- tion about the teeth and tongue. Marassa and Lansing (1995) and Ijsseldijk (1992) found that information from the lips and mouth region alone is sufficient for word rec- ognition, and that adding facial motion in other areas did not increase speech perception significantly. Con- versely, considerable research has found that facial move- ments outside the oral area (for example, of eyes, cheeks) also provides useful information for speechreading (see Massaro, 1998; Preminger et al., 1998 ). Movement of the extraoral areas of the face are known to be highly corre- lated with movement of the oral articulators (Munhall and Vatikiotis-Bateson, 1998). Indeed it is now established that the mouth region is the primary source of visual speech information, additional and correlated information is available elsewhere on the face (see Lansing and McCon- kie, 2003; Rosenblum et al., 1996). 0167-6393/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.specom.2013.01.003 ⇑ Corresponding author. Tel.: +44 (0) 161 275 2598; fax: +44 (0) 161 275 2588. E-mail address: karen.lander@manchester.ac.uk (K. Lander). www.elsevier.com/locate/specom Available online at www.sciencedirect.com Speech Communication 55 (2013) 600–605