The Molecular Control Toolkit: Controlling 3D Molecular Graphics via Gesture and Voice Kenneth Sabir * Garvan Institute of Medical Research, Australia Christian Stolte † CSIRO Computational Informatics, Australia Bruce Tabor ‡ CSIRO Computational Informatics, Australia Se´ an I. O’Donoghue § CSIRO Computational Informatics, Australia and Garvan Institute of Medical Research, Australia Figure 1: Molecular Control Toolkit using Leap Motion and Aquaria. ABSTRACT Three-dimensional (3D) molecular graphic systems are widely used in the life sciences, both for research and communication. These systems need to enable a rich set of 3D operations, including three- axis rotation and translation, selection of parts of macromolecules, and the ability to redefine the center of rotation. As a result, graphi- cal interfaces for these systems typically require users to learn com- plex keyboard and mouse combinations. This can be a significant barrier for new or occasional users, and even for experts, precise control of 3D molecular structures can be challenging. To help ad- dress these challenges, we developed the Molecular Control Toolkit to support multiple consumer gesture and voice recognition de- vices, and provide an API that allows adaption to multiple molecu- lar graphics systems. The toolkit allows intuitive control, almost as if users are directly manipulating 3D objects in their hands. We ap- plied the toolkit to the Kinect and Leap Motion devices, and to the Aquaria molecular graphics system. We did a pilot user study with 18 life scientists to test the resulting system in different scenarios. Overall, users gave quite favorable ratings to using the Kinect and Leap Motion gesture devices to control molecular graphics, even though these devices initially proved less efficient for common 3D control tasks, compared to the more familiar mouse/keyboard. To * e-mail: k.sabir@garvan.org.au † e-mail: christian.stolte@csiro.au ‡ e-mail: bruce.tabor@csiro.au § e-mail: sean@odonoghuelab.org our knowledge, this is the first toolkit for macromolecular graphics that supports multiple devices with a set of controls sufficiently rich to be useful in the day-to-day work of a broad range of life scien- tists. The Molecular Control Toolkit and Aquaria can be accessed at http://aquaria.ws. Keywords: User Interface Toolkits, Gestural Input, Pointing, Voice Control, Molecular Graphics, Macromolecules. Index Terms: J.3 [Life and Medical Sciences]: Biology and genetics—; I.3.6 [Computer Graphics]: Methodology and Techniques—Interaction Techniques 1 I NTRODUCTION Understanding the function of biological systems often requires use of molecular graphics to examine atomic-resolution, three- dimensional (3D) structures of macromolecules, such as proteins, RNA, or DNA, and their interaction with small molecules. The state of the art method for manipulating these structures is via a combi- nation of mouse and keyboard commands. While this is generally adequate, for many infrequent users of molecular graphics it can be a steep learning curve to remember the required mouse/keyboard combinations needed to achieve full 3D control. Even for expert users, the state of the art is less than perfect; for example, molecular replacement [29] — one of the key tasks in X-ray crystallography — often requires intricate, manual manipu- lation of 3D molecular models into electron density maps. For this, the standard mouse/keyboard controls are often not sufficient, and experts use a variety of other control devices, such as dials, in addi- tion to making use of a relatively large number of mouse/keyboard combinations and shortcuts. 49 IEEE Symposium on Biological Data Visualization 2013 October 13 - 14, Atlanta, Georgia, USA 978-1-4799-1659-7/13/$31.00 ©2013 IEEE