Controlling piano tone by varying the « weight » applied on the key Caroline Traube, Manon Moulin and Felipe Verdugo with the collaboration of Justine Pelletier, Marine Blassel, Sébastien Bel, Aude Van Laere and Denis Brassard Laboratoire de recherche sur le geste musicien (LRGM) — OICRM / CIRMMT / BRAMS — Faculté de musique, Université de Montréal Context For a long time, pianists and acousticians have had divergent views about the possibility to control the timbre of a piano through the variation of touch. While acousticians underline the strong inter-dependence between timbre and intensity, pianists have always claimed that it is possible to realize different timbre nuances by modifying the way the key is depressed. In particular, the pedagogical approach proposed by the professors of piano at the Faculty of music of University of Montreal recommends to act on the double escapement action of grand pianos by varying the amount of weight applied on the key. When the pianist uses more weight, the key is pressed to the bottom of the keyboard and the pianist feels a friction when passing the escapement point (a “bump” is felt after a 7 millimetre descent). The tone is said to be “fuller” and “rounder”. When the pianist uses less weight, he or she plays more at the surface of the keyboard. No friction is felt. The tone is said to be “brighter” and “lighter”. Aims 1) understand the impact of the variation of weight (applied on the key) on the double escapement grand piano action 2) evaluate if the variation of weight has an impact on the sound of the instrument, independently of the intensity of the tone 3) Identify the dimensions of piano timbre which vary with weigh Behaviour of a piano demonstration key Study 1 Method Task Two pianists played a short musical phrase varying several control parameters and at several intensity levels. Beethoven sonata in F minor op. 2 n. 1, 2 d mov. mes. 10-11 with C half note on left hand removed and right hand transposed down to one octave. • with more weight and with less weight • with pedal and without pedal • right hand only, left hand only, the two hands together • at four dynamic levels : p, mf, f, ff Each condition is played 3 times. Equipment • A computer-controlled acoustic grand piano Bösendorfer CEUS – located at BRAMS – equipped with sensors on keys and hammers, allowing the precise capture of key position and hammer velocity (40µm accuracy ). • A pair of high quality microphones directed towards the sound board allowing the recording of the excerpts. Workflow of data analysis Segmentation of individual notes from sound recordings using the onset times from MIDI data Computation of spectral and temporal timbre descriptors on each note with the Matlab MIR Toolbox Computation of mean values for each condition Principal component analysis (PCA) of the data set Study 2 Method Task Two pianists played isolated notes (key : middle D, 293 Hz) • with more weight and with less weight • with pedal and without pedal • right hand only, left hand only, the two hands together • at four dynamic levels : p, mf, f, ff Each condition is played 10 times. Equipment • A acoustic grand piano (Kawai brand) from which several key mechanisms have been removed to allow the placement of a cell phone camera capturing the double escapement grand piano action. • A pair of high quality microphones directed towards the sound board allowing the recording of the excerpts. The video monitoring confirms that the grand piano action behaves differently depending on the amount of weight. More weight on the keystroke provokes a sliding motion of the jack along the hammer knuckle and a friction sensation at the level of the key. Without weight : jack stays aligned With weight : jack slides -6 -4 -2 0 2 4 6 -4 -3 -2 -1 0 1 2 3 4 Dim2 :23.3889% expliquée Dim1 :64.0333% expliquée Left hand only Hands together Right hand only With pedal Without pedal Without weight With weight flatness spread brightness entropy attackS RMS attackT kurtosis skewness attackL Dynamic level Attack time When the key is pressed "with weight", the jack slides along the hammer knuckle (which is covered with leather) and produces a sensation of friction. When the key is pressed "without weight", the jack stays aligned with the hammer knuckle and no friction occurs between these two pieces of the piano action. jack hammer knuckle Brightness Results Without weight, the tone displays an increased brightness. camera (cell phone) 4 key mechanisms were removed Video monitoring of the jack / hammer knuckle interaction Workflow of data analysis Segmentation of individual notes from sound recordings Computation of spectral and temporal timbre descriptors on each note with the Matlab Timbre Toolbox (as well as RMS value) Computation of mean values for each condition Principal component analysis (PCA) of the data set Results PCA analysis of data without pedal along the first two dimensions. Conclusion The analysis of the audio data shows that piano tones produced with and without weight weight (with and without friction) differ along several timbre-related acoustical descriptors (temporal and spectral features). This confirms that it is possible to change timbre independently from dynamic level on a piano and that timbre change can be induced by varying weight applied on one key. These timbral differences might be partly due to the different behaviour of the grand piano action mechanisms (jack / hammer knuckle interaction). Acknowledgments Pianists (now graduated doctoral students in performance) : Felipe Verdugo, Justine Pelletier and Marine Blassel Master students in electrical engineering : Manon Moulin (Studies 1 an 2) and Aude Van Laere (hammer velocities) Doctoral student in musical acoustics : Sébastien Bel (PCA initial Matlab coding on Study 1) Piano tuner and technician : Denis Brassard (piano preparation and tuning) … and the wonderful team of piano teachers from the Faculty of music at University of Montréal who inspired this project. References Bel, S. & Traube C. Corrélats acoustiques de cinq nuances de timbre au piano. Actes des Journées d’informatique musicale, Montréal, Qué bec, Canada, 2015. Bernays, M. The expression and production of piano timbre : gestural control and technique, perception and verbalisation in the context of piano performance and practice. PhD thesis, Faculté de musique, Univeristé de Montréal, 2013. Bernays, M. & Traube C. Expressions of piano timbre : verbal description and gestural control. Ed. Delatour-France, 2013. Bernays, M. & Traube C. Investigating pianists’ individuality in the performance of five timbral nuances through patterns of articulation, touch, dynamics, and pedaling. Frontiers in Psychology - Cognitive science, vol. 5, article 157, pp. 35-53, 2014. Giordano, B. L., McAdams S., Misdariis N., Peeters G., & Susini P. The Timbre Toolbox : Extracting audio descriptors from musical signals. Technical report, Ircam/CIRMMT, March 2011. Goebl, W. The Role of Timing and Intensity in the Production and Perception of Melody in Expressive Piano Performance. Doctoral thesis, Institut für Musikwissenschaft, Karl-Franzens-Universität, Graz, Austria. Goebl, W., Bresin, R. & Fujinaga I. Perception of touch quality in piano tones. Journal of the Acoustical Society of America, 2014. Goebl, W., Bresin, R. & Galembo, A. Touch and temporal behaviour of grand piano actions, JASA, 118, p. 1154, 2005. Lartillot O. MIRtoolbox I.6.I, User’s Manual. Aalborg University, Denmark, Dept. of Architecture, Design and Media Technology, 2014. MacRitchie J. The art and science behind piano touch: a review connecting multi-disciplinary literature. Music Sci 19(2):171–190, 2015. McPherson A. P. & Kim Y. E., Multidimensional Gesture Sensing at the Piano Keyboard, Proceedings of the ACM Conference on Human Factors in Computing Systems CHI 2011, Vancouver, BC, Canada, May 7–12, 2011. Moulin M. Etude de l’effet du toucher pianistique sur les caractéristiques acoustiques du timbre au piano. MA thesis. U. of Mons, 2016. Parncutt, R., & McPherson, G. E. (Eds.). The science and psychology of music performance: Creative strategies for teaching and learning. New York: Oxford University Press, 2002. Repp, B. H. Acoustics, perception, and production of legato articulation on a computer-controlled grand piano, Journal of the Acoustical Society of America, 102(3), pp. 1878–1890, 1997. Traube, C. An Interdisciplinary Study of the Timbre of the Classical Guitar. Ph.D. thesis, McGill University, Montreal, 2004. Traube, C. La notation du timbre intrumental : noter la cause ou l’effet dans le rapport geste-son. In Circuit : musiques contemporaines, volume 25, pages 21 – 37. 2015. Without weight, the tone displays an increased spectral centroid. With weight, the tone displays a longer attack. Spectral envelope of piano tones (after the attack) with weight and without weight. Full = With weight Empty = Without weight Size = dynamic level With weight, the tone displays a longer attack. Attack time Centroid (Thanks to Prof. Mickael Begon for his help in the analysis the high-speed video images)