Sonic Sweetener Mug Signe Lund Mathiesen Aarhus University signelma@food.au.dk Derek Victor Byrne Aarhus University derekv.byrne@food.au.dk Qian Janice Wang Aarhus University qianjanice.wang@food.au.dk 1. INTRODUCTION Eating is one of the most sensory of all activities that we take part in. Apart from tasting, it involves both the food and the environment. The multitude of different sensory inputs (from the smell of the food and the colour of the plate, to the lighting in the room and the ambient sound- scape) all affect the way we think about and perceive our food [1]. Much like eating, listening is a fundamental part of most lives; and similar to the role of food, music can modulate our feelings, our mood, and our experiences in life. This demo explores the common link between these two phenomena, specifically the way in which what we taste can be influenced by what we listen to. 2. BACKGROUND Associations between sensory input across all the senses exist, and a substantial body of research on the topic of multisensory perception has shown that individuals expe- rience the phenomenon of “crossmodal correspondences” which can be explained as seemingly arbitrary and unre- lated associations between sensory features from different sensory modalities [2]. For example, the combination of a figure spatially positioned upwards and a high pitched sound is often perceived as related and constitutes an ex- ample of a crossmodal correspondence [3]. Recent research now demonstrate how music and sound can influence our eating experiences and taste and flavour perceptions. For instance, evidence shows that individu- als are able to systematically map sonic attributes (such as pitch, articulation, tempo, harmony, etc) and basic tastes [4]. For instance, high-pitched sounds are often associ- ated with sweet or sour tastes, whereas low-pitch sounds tend to be associated with bitter tastes [5–7]. By making use of such crossmodal correspondences, pieces of music can be chosen or composed to correspond with the taste, mouthfeel, or flavour of a particular food or drink [8]. Fur- thermore, research within the emergent field of “sonic sea- soning” has shown that such soundtracks can actually alter people’s actual taste experience [9]. To further explore the phenomenon of sonic seasoning, we have constructed a system where the way in which Copyright: c  2019 Signe Lund Mathiesen et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License , which permits unre- stricted use, distribution, and reproduction in any medium, provided the original author and source are credited. people pick up a mug triggers the onset of different taste- corresponding soundtracks to accompany the drinking ex- perience. This demo turns the act of drinking into a form of embodied interaction with music, while high-lighting how the same beverage can taste very differently depend- ing on the music that happens to be playing at the same time. By building such new interactive systems, we ex- plore the extent to which our distal senses can influence flavor perception, and at the same time use sound to raise people’s awareness of their own eating behavior. 3. SOUNDTRACK CREATION Two soundtracks were created for this demo, one designed to bring out sweetness, and the other to emphasise bit- terness. Previous research has demonstrated how high- pitched, consonant, and legato-articulated music is associ- ated with sweet tastes, whereas low-pitched and dissonant music is associated with bitter taste [10–12]. The sweet soundtrack is composed of a fairly high-pitched, slow legato clarinet melody on top of a single C major chord piano arpeggio strum. A string pad is introduced to create a softer timbre. The bitter soundtrack consists of a low-pitched brass drone with a slightly syncopated cello string figure in the low register. Furthermore, an arpeggiated synth as well as a pizzicato string ensemble adds a percussive element (static pulse) to the track. Both soundtracks were created in Logic Pro X with the program’s preinstalled software instruments. 4. SETUP The main components and relationships of the Janus Mug are schematised in Figure 1. Each of the two handles of the mug is connected to an Arduino microprocessor (Makey Makey LLC) connected to a laptop. When the user picks up one of the handles, they complete an electrical circuit that trigger a specific key press on the laptop. A program built with the Processing software then reads in the key press and plays the appropriate taste soundtrack. This en- sures that the user can hear either the sweet or bitter sound- track as they are drinking from the mug, depending on which handle of the mug they are holding onto. When the user is finished drinking, setting the mug down on the ta- ble triggers a different key press, which then stops music playback.