A. Nijholt, D. Reidsma, and H. Hondorp (Eds.): INTETAIN 2009, LNICST 9, pp. 9–18, 2009. © ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering 2009 Non-verbal Full Body Emotional and Social Interaction: A Case Study on Multimedia Systems for Active Music Listening Antonio Camurri InfoMus Lab – Casa Paganini Intl Research Centre DIST- University of Genova Piazza Santa Maria in Passione 34, 16123 Genova, Italy antonio.camurri@unige.it Abstract. Research on HCI and multimedia systems for art and entertainment based on non-verbal, full-body, emotional and social interaction is the main topic of this paper. A short review of previous research projects in this area at our centre are presented, to introduce the main issues discussed in the paper. In particular, a case study based on novel paradigms of social active music listening is presented. Active music listening experience enables users to dynamically mould expressive performance of music and of audiovisual content. This research is partially supported by the 7FP EU-ICT Project SAME (Sound and Music for Everyone, Everyday, Everywhere, Every Way, www.sameproject.eu). Keywords: non-verbal full-body multimodal interfaces, emotion, social signals, sound and music computing, active listening. 1 Introduction Entertainment applications characterized by full-body engagement of users has emerged in the recent years. The Nintendo Wii and the diffusion and familiarity by users with interactive art multimedia installations (e.g. in museums and science centers, in public spaces) are examples that bear witness of the diffusion of such “embodied” applications. Art and entertainment is contributing to move toward more effective, user-centric, embodied paradigms of interaction with multimedia content. In a broader scenario, we can envisage future internet networked media enabling users within the next 5-10 years to be fully immersed with 3D audiovisual content, using natural interfaces sensitive to the context and to full-body multimodal expressive intentions, and with the highest degrees of physical as well as social engagement. See for example the User Centric Media EU FP7 ICT projects for a state of the art on these visions. Already in the sixties, the choreographer Merce Cunningham and the composer John Cage developed an artistic project where a dancer could move in a sensitive space, where his body could modify electronically generated music: this was a sort of a large scale Theremin musical instrument, and we can consider it as a pioneer work The original version of this chapter was revised: The copyright line was incorrect. This has been corrected. The Erratum to this chapter is available at DOI: 10.1007/978-3-642-02315-6 30