1 Engaging the whole body in mobile interaction Jakob Tholander, Jarmo Laaksolahti,Elsa Kosmack-Vaara, Pedro Ferreira, Tove Jaensson,Ylva Fernaeus Mobile Life Swedish Institute of Computer Science/Stockholm University, Box 1263, SE-164 29 Kista, Sweden jakobth@dsv.su.se ,{jarmo},{elsa},{pedro}{ylva}@sics.se, tove.jaensson@teliasonera.se INTRODUCTION Research within mobile interaction and applications initially focused on building interfaces that adapt to the limited interaction and presentation resources of mobile devices such as the limited bandwidth and small screens [1]. More recent work has started to address richer interactive experiences by exploring ways of integrating what could be called the “whole body” in interaction in mobile settings. Such work includes for instance location- based systems, social networking, and collaborative applications of different kind. In line with this development our research investigates embodied and experience-oriented interaction with mobile and tangible devices. In our previous work we have explored a number of themes involving different forms of embodied interactions in support of physical, bodily, emotional, and affective experiences. Together these contribute towards an expanded range of engagement with and around mobile devices and applications. Some of our recent work addresses themes such as: What kind of actions and interactions are supported by mobile devices? How may we integrate subjective experiences and bodily actions, as well as interactions with bio-sensor data and higher-level representations of these. Our long term ambition is to explore these novel forms of interaction and how they can be used for developing frameworks for understanding embodied and experience oriented interactions with mobile and tangible devices. This position paper begins by outlining some of our conceptual starting points. Thereafter we present examples from our work in order to illustrate some of the ways that we have attempted to support bodily forms of interaction in mobile settings. Next, we outline some issues that we need to address in moving towards an interaction framework for mobile devices and applications that incorporates “whole- body” experiences. STARTING POINTS Interaction design and HCI are increasingly attempting to incorporate diverse physical forms as well as bodily engaging activities into interactive devices and applications. This has lead to discussions of how such a development shapes the way we look at interaction, and the perspectives we use to describe such forms of interaction. Two strands that have arisen out of this are embodied and experience- oriented perspectives on interaction, which both have been proposed as ways for moving away from the information- and data centric views that previously dominated HCI. These perspectives emphasize the integration between bodily action and human cognition in the shaping of subjective experiences and interpretations. From such a viewpoint, a broad range of physical, emotional, and cognitive practices comes together in forming human experiences. One of the present concerns in HCI is then to investigate how to provide opportunities for, and also to understand, an expanded range of actions and experiences in relation to interactive devices. Fernaeus, Tholander, & Jonsson [2] have proposed a model for understanding interaction with tangible artefacts based on what is called an action-centric perspective. Rather than focusing on how information is represented and transmitted this perspective emphasizes the concrete actions that users may take with tangible artefacts. At the end of this paper we will discuss this model in relation to some of the issues that have arisen from our work with applications presented here. Our ambition is to investigate how this perspective can be used for understanding and designing mobile applications and interaction. BODILY ENGAGING MOBILE INTERACTION Some of our applications and studies have attempted to (1), investigate how to design for the interplay between specific physical interactions and emotional expressions thus allowing users to engage in an “affective loop” in which bodily and affective expressions interact to form a rich emotional experience (eMoto), (2) design bodily memorabilia together with location and sensor information to support users in making connections between everyday physical experiences and their daily activities in order to personally reflect on their lifestyles (Affective Diary and Affective Health), (3) to support embodied affective interaction allowing groups of friends to create shared emotional expressions (FriendSense). eMoto eMoto is a mobile service for emotional messaging between friends [3]. The underlying design goal was to create a system that would be physically, intellectually and socially engaging, and also to provide an aesthetic experience. It extends upon mobile text messaging by adding a