Vol.:(0123456789) 1 3 Virtual Reality https://doi.org/10.1007/s10055-020-00473-3 ORIGINAL ARTICLE Emotional responses to watching and touching 3d emotional face in a virtual environment Muhammad Hassan Jamil 1  · Wanjoo Park 1  · Mohamad Eid 1 Received: 27 August 2019 / Accepted: 24 September 2020 © Springer-Verlag London Ltd., part of Springer Nature 2020 Abstract Facial expressions play a crucial role in modulating the emotional responses in the viewers. Touch is an important factor in shaping human emotions and social communication. The objective of this study is to investigate the efects of viewing and touching a virtual emotional face on the emotional responses of a viewer/toucher. In the case of touching the model, the efects of physical properties, namely stifness and texture, are examined. Emotional facial expressions for neutrality, anger, fear, disgust, happiness, surprise, and sadness are developed and experimentally validated for the visual stimuli whereas four combinations of stifness/texture properties are examined for the physical properties (low/high stifness and smooth/rough texture). 25 participants viewed and touched the virtual emotional face and reported their respective emotional responses. The results showed that watching angry, happy, surprised, and sad faces signifcantly increased their anger, happiness, sur- prise, and sadness levels, respectively (p < 0.05). Watching a scared or a sad face signifcantly modulated the participants’ surprise levels (p < 0.05). On the other hand, viewing and touching an angry face signifcantly reduced the surprise level in the toucher (p < 0.05). As for diferences based on physical properties, our results suggested that viewing and touching the disgusted face signifcantly modulated sadness. In particular, high stifness/rough texture condition resulted in a signifcant increase in sadness while viewing and touching the disgusted face, compared to the high stifness/smooth texture condition (p < 0.01). These conclusions suggest that viewing and touching an emotional face in a virtual environment modulates the emotional responses in the viewer/toucher. Findings of this study help the feld of virtual reality to expand to a greater understanding of building emotionally compelling interpersonal interactions in the virtual environments. Keywords Emotions · Facial expressions · Haptic interaction · Virtual environments 1 Introduction Visual stimuli are known to afect the emotional state among individuals (Min et al. 2005; Lane et al. 1999). Among other visual stimuli, Virtual Reality (VR) is an efective tool for creating immersive experiences that are able to originate or infuence users’ emotional states (Riva et al. 2007). For example, the study in Riva et al. (2007) demonstrated that VR simulation is capable of infuencing anxiety and relaxa- tion among viewers, and also highlighted the infuence on the emotional state by the sense of presence in virtual world. Similarly, VR has shown its efectiveness to help overcome social phobia and other mental disorders (Klinger et al. 2005). As for afective communication, the face is a primary channel through which emotions are expressed and facial expressions are of major importance in infuencing oth- ers’ emotions. Several studies have confrmed that facial expressions in virtual environments can induce emotional responses in an observer (Kret et al. 2013). Haptic modality is another afective input medium among other multimodal interactions that infuence our emotions. Haptic interac- tions are known to induce pleasant or unpleasant sensations in humans (Löken et al. 2009; Essick et al. 2010). There have been several research activities studying the role of Haptics (direct or mediated) to elicit or infuence emotional responses (Eid and Osman 2016). Physical properties of the haptic interaction can infuence specifc emotional responses. We experience tactile sticki- ness of fabric/clothing against our skin every day that afect * Mohamad Eid mohamad.eid@nyu.edu 1 New York University Abu Dhabi, 129188 Abu Dhabi, Saadiyat Island, United Arab Emirates