Learning Non-Visual Graphical Information using a Touch-Based Vibro-Audio Interface Nicholas A. Giudice 1 , HariPrasath Palani 1 1 Spatial Informatics Program: School of Computing and Information Science, University of Maine, 5711 Boardman Hall, Orono, Maine, USA 04469 giudice@spatial.maine.edu, hariprasath.palani@maine.edu Eric Brenner 2 , Kevin M. Kramer 2 2 Advanced Medical Electronics, 6901 East Fish Lake Road, Suite 190, Maple Grove, MN 55368 brenneraeric@gmail.com, kkramer@ame-corp.com ABSTRACT This paper evaluates an inexpensive and intuitive approach for providing non-visual access to graphic material, called a vibro- audio interface. The system works by allowing users to freely explore graphical information on the touchscreen of a commercially available tablet and synchronously triggering vibration patterns and auditory information whenever an on- screen visual element is touched. Three studies were conducted that assessed legibility and comprehension of the relative relations and global structure of a bar graph (Exp 1), Pattern recognition via a letter identification task (Exp 2), and orientation discrimination of geometric shapes (Exp 3). Performance with the touch-based device was compared to the same tasks performed using standard hardcopy tactile graphics. Results showed similar error performance between modes for all measures, indicating that the vibro-audio interface is a viable multimodal solution for providing access to dynamic visual information and supporting accurate spatial learning and the development of mental representations of graphical material. Categories and Subject Descriptors H.5.2 [User Interfaces]: Auditory (non-speech) feedback, Evaluation/methodology; K.4.2 [Social Issues]: Assistive technologies for persons with disabilities General Terms Design, Experimentation, Human Factors Keywords Accessibility (blind and visually-impaired), assistive technology, information graphics, haptic cues, audio cues, android programming, graphs and diagrams. 1. INTRODUCTION Gaining access to graphical information such as graphs, figures, maps, and images represents a major challenge for blind and visually impaired people. Access to printed material has largely been solved via screen reading software using text-to-speech, for example, programs such as JAWS for Windows (www.freedomscientific.com) or VoiceOver for the Mac and iOS- based portable devices (www.apple.com/accessibility/voiceover/). However, these programs do not have the ability to convey meaningful information about graphic and non-text-based material. Given the vast amount of information which is conveyed through visually-based representations, whether it is in the classroom, the boardroom, or the living room, blind people will continue to miss out on a major component of our information- driven culture unless new non-visual solutions providing access to graphical information are developed. Although this problem has been widely studied (see Section 3, Current Research), approaches for improving the accessibility of graphical information have not made much progress in reaching blind and low-vision users. As this demographic is estimated to number around 12 million people in the U.S. and 285 million people worldwide [21], the need for developing devices that are both usable and likely to be adopted is of growing societal importance. The path forward requires addressing the following limitations which have plagued progress in this domain: research and development projects all too often languish in research labs; the design of new hardware/software is frequently driven by engineering principles without solid theoretical knowledge of relevant perceptual and cognitive characteristics of the human end-user; the systems developed generally have a steep learning curve and rely on unintuitive sensory translation rules; many solutions necessitate purchase of expensive single-purpose hardware; assistive technology often is built-around non-portable devices; and there is an emphasis in the literature on describing technical design features and algorithms, rather than conducting empirical experiments and behavioral evaluations. Our goal is to provide access to visually-based graphic material using an intuitive interface that provides dynamic information on a device which is inexpensive (i.e. is based on off-the-shelf commercial hardware vs. highly specialized adaptive equipment), is portable enough to be used in many contexts and environments, is multi-purposed (meaning that the underlying hardware can be used for other applications), and supports universal design principles (i.e., is highly customizable and includes many accessibility features in the native interface). To this end, this paper describes what we call a vibro-audio interface, used for conveying visual information via a commercial tablet, which satisfies these design criteria. We believe that the conjunction of considering these design factors from the onset, along with Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ASSETS’12, October 22–24, 2012, Boulder, Colorado, USA. Copyright 2012 ACM 978-1-4503-1321-6/12/10...$15.00. 103