Configuring Gestures as Expressive Interactions to Navigate Multimedia Recordings from Visits on Multiple Projections Giulio Jacucci ¹ ² ¹ Department of Information Processing Science, University of Oulu, P.O. Box 3000, 90014 Oulu, Finland ² Helsinki Institute for Information Technology (ARU), P.O. Box 9800, FIN-02015 HUT Juha Kela, Johan Plomp VTT Electronics, Kaitoväylä 1, P.O. Box 1100, 90571 Oulu, Finland juha.kela@vtt., johan.plomp@vtt.fi ABSTRACT The wide availability of digital recording devices leads to investigate how multimedia content can be navigated beyond a desktop computer set up. We present a system and a variety of applications, to navigate multimedia recordings from visits making use of: large multiple projections, location information to organise media and re-experience aspects of visits, physical interfaces as gesture-based interaction and other interfaces that render media more tangible and therefore more readily available. Analysing field trials we discuss the expressiveness and experiential aspects of gesture based interfaces as important features in navigating multimedia in “immersive” environments. Author Keywords Gesture-based interaction, visualising digital photographs, multiple projections. ACM Classification Keywords H5.m. Information interfaces and presentation (e.g., HCI): Miscellaneous. INTRODUCTION Moving interaction from the virtuality of the screen of a desktop computer to the physical environment can result in diverse challenges and opportunities, according to the activity that computers seek to support. A particularly interesting activity to investigate for ubiquitous computing is digital photography. The widespread use of digital cameras is resulting, in leisure and work settings, in using the computer (also television screens, and home theatre technologies) to view and share pictures often taken at remote sites or during visits. This paper seeks to explore opportunities and challenges of providing novel physical interfaces to visualise pictures and sounds from visits, moving the viewing and sharing of pictures from a desktop computer to a ubiquitous setting. In particular, we investigate three opportunities. Firstly, large multiple projections can provide more immersive environments to visualise digital media and sound can play a more important role. Secondly, location information can be used to organise media and re-experience aspects of visits. Finally, physical interfaces, as gesture-based interaction, can free users from the desktop and other interfaces can render media more tangible and therefore more readily available. A variety of questions and challenges accompany these opportunities. This paper explores novel interfaces to navigate collaboratively pictures from visits. In particular, we discuss the expressiveness and experiential aspects of gesture based interfaces as important features in navigating multimedia in “immersive” environments. For this purpose, we use a distributed environment developed in the Atelier project 1 that provides: computational support to record location information of recorded multimedia during visits, a hypermedia database to store visits as hyperdocuments, and a variety of physical interfaces to navigate the hyperdocuments. Physical interfaces include: barcode scanner and print-outs with thumbnails and barcodes, an infrared remote control interface, and a gesture based interface. We organised trials, where pair of participants, after having recorded a visit, navigated the multimedia recordings using two back projection screens. The contribution of this paper is twofold: to present a variety of applications to support gesture based-interaction in navigating multimedia recordings from visits on multiple screens, and to provide insights from user trials on the advantages and challenges of gestures as an interaction mode in such environments. 1 http://atelier.k3.mah.se Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. MUM 2004, October 27-29, 2004 College Park, Maryland, USA. Copyright 2004 ACM 1-58113- 981-0 /04/10... $5.00 157