Citation: Papetti, Stefano, Hanna
Järveläinen, and Federico Fontana.
2023. Design and Assessment of
Digital Musical Devices Yielding
Vibrotactile Feedback. Arts 12: 143.
https://doi.org/10.3390/
arts12040143
Academic Editors: Justin Paterson
and Marcelo M. Wanderley
Received: 14 January 2023
Revised: 2 May 2023
Accepted: 1 June 2023
Published: 7 July 2023
Copyright: © 2023 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
arts
Article
Design and Assessment of Digital Musical Devices Yielding
Vibrotactile Feedback
Stefano Papetti
1,
* , Hanna Järveläinen
1
and Federico Fontana
2
1
Institute for Computer Music and Sound Technology, Zurich University of the Arts, 8005 Zurich, Switzerland
2
Department of Computer Science, Mathematics and Physics, University of Udine, 33100 Udine, Italy
* Correspondence: stefano.papetti@zhdk.ch
Abstract: Touch has a pivotal importance in determining the expressivity of musical performance
for a number of musical instruments. However, most digital musical devices provide no interactive
force and/or vibratory feedback to the performer, thus depriving the somatosensory channel of a
number of cues. Is the lack of haptic feedback only an aesthetic issue, or does it remove cues essential
for digital instrument playing? If so, at which level is the interaction objectively impoverished? What
are the effects on musical performance? In this survey article we illustrate our recent research about
the use of vibrotactile feedback in three digital instrument interfaces, using tools that we developed
over several years and made available to the community in open-source form. These interfaces span
a wide range of familiarity and gestural opportunities, enabling us to explore the impact of haptic
feedback on different types of digital instruments. We conducted experiments with professional
musicians to assess the impact of vibratory cues on both the perceived quality of the instrument and
the playing experience, as well as on musical performance. Particular attention was paid to scientific
rigor and repeatability of the results, so as to serve as a reference for researchers and practitioners
of the musical haptics community. Our results suggest a significant role of vibrotactile feedback in
shaping the perception of digital musical instruments, although the effects on musical performance
varied depending on the interfaces tested.
Keywords: vibration; musical instrument; digital musical interface; quality perception; performance
assessment; musical practice
1. Introduction
During instrumental performance, musicians are exposed to auditory, visual, and also
somatosensory cues. This multisensory experience has been studied since long ago Camp-
bell (2014); Cochran (1931); Galembo and Askenfelt (2003); Hodges et al. (2005); Palmer
et al. (1989); Turner (1939); however, the specific interaction between sound and vibrations
in musical instrument playing has only been the object of systematic research since the
1980s Askenfelt and Jansson (1992); Fontana et al. (2017); Keane and Dodd (2011); Marshall
(1986); Saitis (2013); Suzuki (1986); Wollman et al. (2014), with an increasing recognition
of the prominent role of tactile and force feedback cues in the complex perception–action
mechanisms involved O’Modhrain and Gillespie (2018). More recently, research on the
somatosensory perception of musical instruments has been consolidated, as testified by
the emerging “musical haptics” topic Papetti and Saitis (2018a). This growing interest is
in part driven by the increased availability of accurate sensors capable of recording touch
gestures and vibratory cues, and affordable and efficient actuators rendering vibrations
or force. Touch sensors and actuators can be employed first to investigate the joint role
of the auditory and somatosensory modalities in the perception of musical instruments,
and then to realize novel musical interfaces and instruments, building on the lessons of
the previous investigation Marshall and Wanderley (2006); O’Modhrain and Chafe (2000);
Overholt et al. (2011). Through this process, richer or even unconventional feedback cues
Arts 2023, 12, 143. https://doi.org/10.3390/arts12040143 https://www.mdpi.com/journal/arts