90 March/April 2016 Published by the IEEE Computer Society 0272-1716/16/$33.00 © 2016 IEEE Spatial Interfaces Editors: Frank Steinicke and Wolfgang Stuerzlinger GyroWand: An Approach to IMU-Based Raycasting for Augmented Reality Juan David Hincapié-Ramos University of Manitoba Kasim Özacar Tohoku University Pourang P. Irani University of Manitoba Yoshifumi Kitamura Tohoku University O ptical see-through head-mounted displays (OST-HMDs), such as Epson’s Moverio (www.epson.com/moverio) and Micro- soft’s Hololens (www.microsoft.com/microsoft -hololens), enable augmented reality (AR) applica- tions that display virtual objects overlaid on the real world. At the core of this new generation of devices are low-cost tracking technologies using techniques such as marker-based tracking, 1 simul- taneous localization and mapping (SLAM), 2 and dead reckoning based on inertial measurement units (IMUs), 3,4 which allow us to interpret users’ motion in the real world in relation to the virtual content for the purposes of navigation and inter- action. These tracking technologies enable AR ap- plications in mobile settings at an affordable price. The advantages of pervasive tracking come at the cost of limiting interaction possibilities, however. Off-the-shelf HMDs still depend on peripherals such as handheld touch pads and other peripher- als. Mid-air gestures and other natural user inter- faces (NUIs) offer an alternative to peripherals but are limited to interacting with content relatively close to the user (direct manipulation) and are prone to tracking errors and arm fatigue. 5 Raycasting, an interaction technique widely explored in traditional virtual reality (VR), is an- other alternative for interaction in AR. 6 Raycasting generally requires absolute tracking of a handheld controller (known as a wand), but the limited tracking capabilities of novel AR devices make it difficult to track a controller’s location. In this article, we introduce a raycasting tech- nique for AR HMDs. Our approach is to devise a way to provide raycasting based only on the ori- entation of a handheld controller. In principle, the rotation of a handheld controller cannot be used directly to determine the direction of the ray as a result of intrinsic problems with IMUs, such as magnetic interference and sensor drift. Moreover, a user’s movement in space creates a situation in which the virtual content and ray direction are of- ten not aligned with the HMD’s field of view (FOV). To address these challenges we introduce Gy- roWand, a raycasting technique for AR HMDs using IMU rotational data from a handheld con- troller. GyroWand’s fundamental design differs from traditional raycasting in four ways: ■ interprets IMU rotational data using a state machine, which includes anchor, active, out-of- sight, and disambiguation states; ■ compensates for drift and interference by tak- ing the orientation of the handheld controller as the initial rotation (zero) when starting an interaction; ■ initiates raycasting from any spatial coordinate (such as a chin or shoulder); and ■ provides three disambiguation methods: Lock and Drag, Lock and Twist, and AutoTwist.