To appear in the International Symposium on Mixed and Augmented Reality (ISMAR) 2014. A Study of Depth Perception in Hand-Held Augmented Reality using Autostereoscopic Displays Matthias Berning * Daniel Kleinert Till Riedel Michael Beigl Karlsruhe Institute of Technology (KIT), TECO, Karlsruhe, Germany ABSTRACT Displaying three-dimensional content on a flat display is bound to reduce the impression of depth, particularly for mobile video see-trough augmented reality. Several applications in this domain can benefit from accurate depth perception, especially if there are contradictory depth cues, like occlusion in a x-ray visualization. The use of stereoscopy for this effect is already prevalent in head- mounted displays, but there is little research on the applicability for hand-held augmented reality. We have implemented such a proto- type using an off-the-shelf smartphone equipped with a stereo cam- era and an autostereoscopic display. We designed and conducted an extensive user study to explore the effects of stereoscopic hand-held augmented reality on depth perception. The results show that in this scenario depth judgment is mostly influenced by monoscopic depth cues, but our system can improve positioning accuracy in challeng- ing scenes. Keywords: Autostereoscopy, mobile devices, depth perception, augmented reality, user study Index Terms: H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems—Artificial, augmented, and vir- tual realities; H.5.2 [Information Interfaces and Presentation]: User Interfaces—Ergonomics 1 I NTRODUCTION Depth distortion is still one of the most common perceptual prob- lems found in Augmented Reality (AR) today [8]. It describes the issue that users cannot correctly identify the spatial relation be- tween objects based on their viewpoint, which is especially the case for the combination of real and virtual objects. Depending on the application this information can be crucial for a good performance, e.g. in maintenance task, manuals or when visualizing occluded objects [16]. There is already a large body of research in the area of depth perception for AR, which is focused on head-mounted displays [18, 10, 17, 5]. This is viable for professional applications, but due to commercial reasons, the current platform of choice for consumer oriented AR, are mobile hand-held devices. Since they are limited to video see-through AR and offer only a small field-of-view, these devices are even more prone to depth distortion. The availability of smartphones featuring an autostereoscopic display could enable the use of binocular depth cues in this area. The displays promise a high degree of immersion and increased spatial awareness through a single screen without the need for addi- tional glasses worn be the user (i.e. polarized or shutter). In combi- nation with the two cameras on the back of the device, it is possible to realize true stereoscopic AR. Our contribution in this work is an experimental evaluation on the effect of stereopsis for depth perception in mobile hand-held * e-mail: matthias.berning@kit.edu AR. Different virtual objects were embedded in a real scene and displayed to the user. The participants in our study had to align them accurately with a real object. The goals were a) to verify if the autostereoscopic display can help an user to understand the spatial relation between real and virtual objects and b) to compare the effect to other depth cues. 2 DEPTH PERCEPTION AND STEREOSCOPIC DISPLAYS According to Cutting [2], the relative importance of different depth cues is determined by the distance of the objects to the user. There are three different areas: In personal space, from 0-2m directly in front of the observer, binocular disparity provides the most accu- rate depth judgments. It is the most important depth cue provided by stereoscopic vision and particularly useful to resolve ambigui- ties created by other perceptual cues. Kyt¨ o et al. [9] list some of its special benefits for AR, e.g. the layering of augmentations to increase the information density. Current autostereoscopic displays are either of the lenticular sheet or the parallax barrier type [13]. Both have in common, that a normal display is used and the images for the left and right eye are arranged in interleaved columns. By applying a sheet to structure the emitted light, each image is only visible from a specific angle which should be the corresponding eye of the user. Due to this arrangement, stereopsis is only achieved in a specific distance and orientation to the display, which is one of the major drawbacks. An- other problem is the conflict between accommodation and conver- gence, which are directly linked in normal vision. When consum- ing binocular content on a flat screen, the relative orientation of the eyes will change (convergence) but the focus has to stay fixed on the screen (accommodation). Nevertheless, recent studies have shown that these devices can improve the user experience when consum- ing content on a hand-held device [14]. To this date, off-the-shelf smartphones are only available with parallax barrier type displays. They structure light by superimposing fine vertical lines onto the display, blocking every second column for each eye. Through the use of liquid-crystal arrays for this barrier, the stereoscopic function can be controlled dynamically. 3 RELATED WORK Understanding the perception of depth is a widely researched topic in VR and AR, although most of it is focused on HMDs. The work of Swan et al. [18] is exemplary for the conducted studies and is listed here explicitly because of the survey on the field included in the publication. They conducted two experiments to investigate the effects of stereoscopic vision on depth judgment in head-worn AR. Both showed a main effect of stereoscopy on depth judgment, resulting in greater accuracy. Dey et al. [3] looked at the effect of different hand-held screen sizes and resolutions on depth-perception. In several AR test sce- narios, they found a significant effect of resolution on distance esti- mation. Autostereoscopic displays were not part of their evaluation. The authors of [1] conducted a Wizard-of-Oz study, using a cutout phone to simulate the effects of binocular vision on position- ing accuracy of a hand behind the device. They show that perfor- mance was worse using a monocular setting. A functional prototype 1