Impromptu Collaborative Projection for Storytelling Simon Robinson 1 , Elina Vartiainen 1 , Matt Jones 1 , Gary Marsden 2 1 Future Interaction Technology Lab 2 Dept. of Computer Science Swansea University, SA2 8PP, UK University of Cape Town { s.n.w.robinson, e.vartiainen, Cape Town, South Africa matt.jones } @swan.ac.uk gaz@cs.uct.ac.za ABSTRACT Pico projectors are steadily becoming more common, but research to date has mainly investigated their usage as output devices. In this paper we consider how pico projectors might be used for input. We describe a concept for using projected content to create animated stories, allowing groups of people projecting photos, sketches, or even videos, to record and share the stories they tell. The concept is demonstrated by an early prototype using sensor data and a simple calibration method for device location and orientation tracking. INTRODUCTION In recent years the emergence of pico projectors has pro- vided an ideal platform for ad-hoc content sharing in any location. Naturally, researchers have looked at how these new output devices might be used and further developed for user-driven public displays, or collaborative manipulation of content. However, in this research we are interested in the use of these projectors as input devices. In this paper we propose the use of handheld projectors for group storytelling, recording the projected content as input for later retelling. We are particularly interested in how pico projectors might be used to help groups collaborate, both to re-live existing stories, and also to annotate and sketch with shared content to create fun, interactive narratives. Our design aims to support this group storytelling, record- ing and editing behaviour by allowing ad-hoc collabora- tive sketches and annotations to be developed into shared video stories. We have developed a prototype that supports collaboration between a small number of users, allowing participants to project simple sketches, animating their story by moving a handheld projector. Projected interactions are then composed into a video record of the action, allowing them to later revisit and retell. Although the prototype is in its early stages, we have addressed issues in both sensor- based tracking for spontaneous projection, and in the initial device setup and calibration that is required. BACKGROUND Projected images have been used in play and ad-hoc sto- rytelling throughout history. From digital shadow puppets [7] to torch-based exploration [4], researchers and artists have worked to develop digital versions of these interactions. Previous work has investigated the use of both lasers [5] and flashlights to interact with static content [6], and to play games by controlling a projected display [3]. Figure 1. Creating stories by collaboratively sketching and projecting: Sam and Ben are reading the Harry Potter books, and decide to act out their own alternative Death Eaters storyline... In our storytelling design we take a different approach, using the projector itself as the input device, allowing these types of interactions anywhere. Projector-based pointing interac- tion has previously been investigated (e.g. [1]), including multi-user variants, (e.g. [2]), but has usually focused on peephole-type displays. In addition, previous designs have made use of visual-based device tracking methods, mainly due to their higher accuracy. In our prototype we use a sensor-based approach, sacrificing pinpoint accuracy for the ability to project and interact spontaneously in any location. DESIGN Our proposed system will allow groups of people to record and annotate simple video stories to share their experiences. Fig. 1 illustrates how a group of people might tell a story using their individual projector phones. Participants begin the storytelling by projecting a photo or video, or sketching their own scene, moving and pointing their device to animate. Others can project their own over- lays, interacting with each other to create a shared event. While participants are animating the projected content, the interactions of each individual user are recorded, allow- ing the story to be replayed later on. Each participant’s projected content and interactions are tracked separately, allowing edited versions of the event to be created if desired. Current Prototype We have developed an early prototype to investigate possible interactions with this projected story tool. At present our system demonstrates only the base functionality, but we also address issues in setup, synchronisation and device tracking. Our current prototype setup consists of a Nokia 5800 mobile phone, firmly attached to an Optoma PK101 projector and a