D. Forsyth, P. Torr, and A. Zisserman (Eds.): ECCV 2008, Part II, LNCS 5303, pp. 568 – 581, 2008. © Springer-Verlag Berlin Heidelberg 2008 Shape-Based Retrieval of Heart Sounds for Disease Similarity Detection Tanveer Syeda-Mahmood and Fei Wang IBM Almaden Research Center, 650 Harry Road, San Jose, CA 95120 {stf,wangfe}@almaden.ibm.com Abstract. Retrieval of similar heart sounds from a sound database has applica- tions in physician training, diagnostic screening, and decision support. In this paper, we exploit a visual rendering of heart sounds and model the morphologi- cal variations of audio envelopes through a constrained non-rigid translation transform. Similar heart sounds are then retrieved by recovering the correspond- ing alignment transform using a variant of shape-based dynamic time warping. Results of similar heart sound retrieval are demonstrated for various diseases on a large database of heart sounds. Keywords: Sound pattern analysis, audio retrieval, curve analysis, healthcare application. 1 Introduction The field of pattern recognition is becoming increasingly applicable to new modali- ties. In this paper we explore the application of computer vision techniques to an important class of audio signals, namely, heart sounds. Heart auscultation, i.e., listen- ing to the sounds produced by the heart, is a common practice in the screening of heart disease. Although different diseases produce characteristic sounds, forming a diagnosis based on sounds heard through a stethoscope is a skill that takes years to perfect. Numerous studies have shown that as much as 87% of patients referred to cardiologists for evaluation are as a result of false alarms [2]. Thus software diagnos- tic tools that aid physicians in their diagnosis of heart sounds are needed. Auditory discrimination of heart sounds is inherently difficult as these sounds are faint and lie at the lower end of the audible frequency range [1]. Although there are tools to visually render the sounds [3], we recently observed that the visual represen- tation of heart sounds actually brings out the differentiating characteristics of various diseases more readily than the audio signal. Figure 1 illustrates this by recording the visual appearance of the audio signal within a single heart beat duration, from patients with various diseases. As can be seen, different diseases show characteristically dif- ferent shape patterns. Further, it is also easier to spot the similarity in the sounds across patients with similar diseases through their visual representations. Figure 2 illustrates this for different patients diagnosed with the same disease. From these examples, it appears plausible that the disease similarity can be inferred by develop- ing a measure for capturing the visual similarity of audio signals.