Back to MARS: The unexplored possibilities in query result visualization Alfredo Ferreira INESC-ID/IST/TU Lisbon Lisbon, Portugal alfredo.ferreira@ist.utl.pt Pedro B. Pascoal INESC-ID/IST/TU Lisbon Lisbon, Portugal pmbp@ist.utl.pt Manuel J. Fonseca INESC-ID/IST/TU Lisbon Lisboa, Portugal mjf@inesc-id.pt ABSTRACT A decade ago, Nakazato proposed 3D MARS, an immer- sive virtual reality environment for content-based image re- trieval. Even so, the idea of taking advantage of post-WIMP interfaces for multimedia retrieval was no further explored for content-based retrieval. Considering the latest low-cost, off-the-shelf hardware for visualization and interaction, we believe that is time to explore immersive virtual environ- ments for multimedia retrieval. In this paper we highlight the advantages of such approach, identifying possibilities and challenges. Focusing on a specific field, we introduce a preliminary immersive virtual reality prototype for 3D ob- ject retrieval. However, the concepts behind this prototype can be easily extended to the other media. Categories and Subject Descriptors H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval; H.5.2 [Information Interfaces and Presentation]: User Interfaces—Interaction Styles, Input Devices and Strategies Keywords Multimedia Information Retrieval, 3D Object Retrieval, Im- mersive Virtual Environment 1. INTRODUCTION Despite advances on multimedia information retrieval (MIR), this field still on its infancy. Especially when compared to its textual counterpart. Actual textual search engines are maturely developed and its widespread use makes them fa- miliar to most users. The current scenario in MIR is quite different. Indeed, existing content-based MIR solutions are far from being largely used by the common user. A few exceptional systems were able to strive with relative success, such as Retrievr 1 , a search tool for Flickr 2 based 1 http://labs.systemone.at/retrievr/ 2 http://www.flickr.com/ Copyright c 2011 for the individual papers by the papers’ authors. Copy- ing permitted only for private and academic purposes. This volume is pub- lished and copyrighted by the editors of euroHCIR2011. on visual queries. However, most existing solutions still face major drawbacks and challenges to be tackled. Among oth- ers, extensively identified in Datta’s survey [5], we high- light two. First, queries rely mostly on meta-information, often keyword-based. This means that, in a closer analysis, searches can be reduced to text information retrieval of mul- timedia objects. Second, the result visualization follows the traditional paradigm, where the results are presented as a list of items on a screen. These items are usually thumbnails, but can be just filenames or metadata. Such methodology greatly hinders the interpretation of query results on collec- tions of videos or 3D objects. Notably, a decade ago, a new visualization system for content- based image retrieval(CBIR) was proposed by Nakazato and Huang from the University of Illinois. The 3DMARS [11] was an immersive virtual reality (VR) environment to per- form image retrieval. It worked on the NCSA CAVE [4] which provided fully immersive experience and later on desk- top VR systems. However, despite this ground-breaking work and recent developments in the interaction domain, little advantages have been taken by the multimedia infor- mation retrieval community from immersive virtual environ- ments. In this paper we bring up the work of Nakazato and Huang as a starting point to the exploration of new possibilities for result visualization in multimedia information retrieval. With the spreading of stereoscopic viewing and last gener- ation interaction devices outside lab environment and into our everyday lives, we believe that in a short time users will expect richer results from multimedia search engines than just a list of thumbnails. Following this rationale, and de- spite it could be applied to any type of media, we will focus our approach on 3D object retrieval (3DOR). 2. TRADITIONAL 3DOR APPROACHES The first and most noticeable 3D search engine, at least within researchers working on this area, is the Princeton 3D Model Search Engine[8]. This remarkable work provide content-based retrieval of 3D models from a collection of more than 36000 objects. Four query specification options are available: text based; by example; by 2D sketch; and by 3D sketch. The results of this queries are presented as an array of model thumbnails. Additionally to queries by example and sketch-based queries, the FOX-MIIRE search engine[1] introduced the query by