Proceedings of ICAD 05-Eleventh Meeting of the International Conference on Auditory Display, Limerick, Ireland, July 6-9, 2005 ICAD05-121 EFFECT OF AESTHETICS ON AUDIO-ENHANCED GRAPHICAL BUTTONS Sami Ronkainen, Jonna Häkkilä Leena Pasanen Nokia Multimedia Department of Mathematical Sciences, University of Oulu. P.O.Box 300, 90401 Oulu, Finland P.O.Box 3000, 90014 Oulun Yliopisto, Finland {sami.ronkainen, jonna.hakkila}@nokia.com lpasanen@mail.student.oulu.fi ABSTRACT In this paper, two different auditory feedback schemes related to graphical buttons are compared to each other and to a visual- only condition. The results show that aesthetically pleasing auditory design is clearly preferred among the users, and can lead to performance benefits over not only a design with no auditory enhancements, but also a design with aesthetically less pleasing auditory enhancements. 1. INTRODUCTION A grapical push-button is an essential piece of today's graphical user interfaces. Whereas the use of graphical buttons (compared to physical ones) adds flexibility into user interface design and allows space savings, it also challenges the interaction with the device. Because of the lack of haptic feedback, it is beneficial aid the visual channel by offering also alternative feedback. There is a specific problem related to graphical buttons where the button action is only carried out when the pointer is still over the button when it is released. In that situation, especially with touchscreens it may happen that the pointer has in fact slipped off the button when it is lifted, causing the intended input event not to be carried out. This problem - a so-called slip-off error - is emphasized with handheld devices, where the small physical size of the screen often result small size and dense placing of buttons, along with usage situations where full visual attention may not be paid to the interaction. When used on the move, e.g. when walking or in a shaking environment such as a moving car, slip-off errors are likely to occur even more often than in static environments. Adding auditory feedback provides one way to enhance the usability of graphical input objects[1], [2]. Especially, Brewster et al have researched sonically enhanced graphical buttons and compared them in combination with different visual settings [3], [4]. In the studies, the occurrence of a pointer or stylus slipping off a graphical button before releasing the mouse button or lifting the stylus off a touchscreen, is presented utilizing auditory output. The results showed improved efficiency with no indication of increased annoyance in audio- enhanced conditions over the visual ones. When developing consumer products, the designer has to seriously consider the aesthetics of the design, as it is a major factor of attractivity affecting to the sales of the product. The possibility of annoying the user with sounds was a reason to add the annoyance factor to the Nasa Task Load Index (TLX) questionnaire [5] also in the previous studies. The issue of aesthetics and its effect on the prevalence of auditory displays in general has recently been raised up also by other researchers [6], [7]. Aesthetics plays a big role also in graphical user interfaces. For instance, its effect on the perceived usability of a system has been shown by Kurosu et al [8]. In fact, "aesthetic and minimalist design" has been selected as one of the ten usability heuristics by Nielsen [9]. Minimalism can also be an aesthetic value itself, as used e.g. in industrial design, architecture, music and other art forms. In this paper, we have continued from the framework set by previous research, and using the previous studies as basis, extended the study to compare three feedback conditions in using graphical buttons. Two feedback conditions utilized audio feedback. One of them was designed using a minimalistic approach. The other was similar to the one used in a previous study [4] where pressing a button caused continuously playing auditory feedback. In the third condition, no auditory feedback was used. All feedback conditions also utilized visual feedback, the third condition being one where the visual was the only method of feedback. The motivation for our research was to verify the amount of annoyance the two different auditory feedback conditions would cause, over a visual-only feedback condition. Furthermore, as the minimalistic design in one of the auditory feedback conditions could have lead into less recognizable sounds, degrading the performance of the users, we wanted to compare the users' performance with both auditory feedback conditions. Finally, we wanted to verify the performance benefits found in the previous studies in the audiovisual feedback conditions over the visual-only one. 2. SOUND DESIGN Two auditory feedback schemes were created. One was taken almost "as is" from a previous study where its usefulness has been proven [4]. In this condition, continuous sound is played when a cursor is over a button or when the button is pressed. The pitch of the sound changes from C4(261Hz) to C5(523Hz) when the button is pressed. A slip-off error is indicated by that the continuous sound is stopped when the slip-off occurs. A successful selection is indicated by a combination of two short beeps (40ms with 40ms silence in between) of C7(2093Hz). The timbre used in all sounds of the test was the same as in the previous study, i.e. an electronic organ sound was used. In this paper, this sound design type is referred to as the "continuous feedback" type. A demo of the feedback can be found on the WWW page of the Glasgow Multimodal Interaction Group http://www.dcs.gla.ac.uk/~murray/audiowidgets/button1.1.shtml . The on-button sound included in the demo was not used in the original study [4], but we decided to use in our test too. In practice, though, users did not hear the on-button sound much as the test was conducted on a touchscreen where the users