A tutorial on immersive three-dimensional sound technologies Craig T. Jin School of Electrical and Information Engineering, University of Sydney, Building J03, Maze Crescent, Darlington, Sydney, NSW, 2006 Australia Abstract: There is renewed interest in virtual auditory perception and spatial audio arising from a technological drive toward enhanced perception via mixed-reality systems. Because the various technologies for three-dimensional (3D) sound are so numerous, this tutorial focuses on underlying principles. We consider the rendering of virtual auditory space via both loudspeakers and binaural headphones. We also consider the recording of sound fields and the simulation of virtual auditory space. Special attention is given to areas with the potential for further research and development. We highlight some of the more recent technologies and provide references so that participants can explore issues in more detail. Keywords: 3D sound technology, Virtual auditory space, Spatial hearing PACS number: 43.38.Md, 43.38.Vk, 43.10.Ln, 43.10.Sv [doi:10.1250/ast.41.16] 1. INTRODUCTION There is renewed interest in three-dimensional (3D) sound technologies driven in large part by the technolog- ical drive towards enhanced perception via mixed reality systems. A few commercial examples, as of 2018, are Microsoft 3D Soundscape and the HoloLens, Google’s Resonance Audio, Sony Playstation headset with 3D sound, the Oculus headset and its support for 3D sound spatialization. There is also increasing support and aware- ness for 3D sound by broadcasting companies. A few broadcasting examples are the BiLi project in France, the Binaural Project by the BBC in the United Kingdom, the Orpheus European project, the NHK Super Hi-Vision theatre support for 3D sound with a 22.2 multichannel system. At the current time, the technologies for 3D sound are growing exponentially and so this tutorial will focus on fundamental principles. To begin, we should clarify our definition of 3D sound. By 3D sound, we refer to an immersive experience in which the listener has a clear and extended perception of a 3D sound space — that is sound objects and sound events clearly positioned relative to the listener and to each other in some ambient space that encompasses both the listener and the sound objects. It requires and involves something more than a transient illusion or perception of sound direction; it requires an extended and believable perceptual experience of auditory space that supports some version of reality. This tutorial focuses on three primary technological areas: loudspeaker sound reproduction, binaural sound reproduction using earphones or headphones, and sound field recording. A related tutorial introduction is [1]. To a lesser extent we consider sound field simulation — the art of using engineering design to create a virtual sound environment. We will assume familiarity with the funda- mental processes underlying human 3D sound perception. In particular, we assume familiarity with interaural time difference cues, interaural level difference cues, monaural spectral cues and the necessity for head-tracking when rendering 3D sound over headphones. We also assume familiarity with the following mathematical or signal processing terms: head-related impulse response filters, binaural room impulse responses and head-related transfer functions. 2. LOUDSPEAKER SOUND REPRODUCTION Four loudspeaker reproduction methods shall be con- sidered: vector-base amplitude and intensity panning, transaural cross-talk cancellation, ambisonics, and wave- field synthesis. Loudspeaker reproduction methods for 3D sound can be classified according to the listening con- ditions. Vector-base amplitude panning can provide a relatively stable acoustic image across a moderate-sized listening area with some compromises in spatial fidelity. Wave-field synthesis can provide higher spatial fidelity, but requires a substantially more dense loudspeaker array and is often limited in the frequency range for which high spatial fidelity can be achieved. The ambisonics method e-mail: craig.jin@sydney.edu.au 16 Acoust. Sci. & Tech. 41, 1 (2020) #2020 The Acoustical Society of Japan INVITED TUTORIAL