A local approach based on a Local Binary Patterns variant texture descriptor for classifying pain states Loris Nanni a, * , Sheryl Brahnam b , Alessandra Lumini a a DEIS, IEIIT—CNR, Università di Bologna, Viale Risorgimento 2, 40136 Bologna, Italy b Computer Information Systems, Missouri State University, 901 S. National, Springfield, MO 65804, USA article info Keywords: Image medical analysis Texture descriptors Local Binary Patterns Neonatal pain detection Support vector machine abstract This paper focuses on the use of image-based techniques for classifying pain states, in particular we com- pare several texture descriptors based on Local Binary Patterns (LBP), and we proposed some novel solu- tions based on the combination of new texture descriptors: the Elongated Ternary Pattern (ELTP) and the Elongated Binary Pattern (ELBP). ELTP is the best performing descriptor in our experiments. The ELBP descriptor combines characteristics of the Local Ternary Pattern (LTP) and ELTP. These two variants of the standard LBP are obtained by considering different shapes for the neighborhood calculation and dif- ferent encodings for the evaluation of the local gray-scale difference. The resulting extracted features are used to train a support vector machine classifier. Extensive experiments are conducted using the Infant COPE database of neonatal facial images. Our results show that a local approach based on the ELTP feature extractor produces a reliable system for classifying pain states. Ó 2010 Elsevier Ltd. All rights reserved. 1. Introduction At the end of the century, the US congress passed into law a dec- laration that the first decade of the new millennium would be des- ignated as the ‘‘Decade of Pain Control and Research.” As a result, pain is now considered the 5th vital sign, and there has been a movement to devise better methods of pain assessment. One development has been to move away from the gold standard of self-assessment and include objective measurements (Schiavenato, 2008). For many patient populations, for example, the elderly and neonates, pain must be assessed through proxy judgments. Current pain assessment instruments developed for newborns utilize physiological and behavioral information. Common physio- logical indicators of pain include changes in heart and respiratory rates, blood pressure, vagal tone, and palmar sweating (Coffman et al., 1997). Major behavioral indicators of pain include body movement, crying, and facial expressions (McGrath, 1989). Diag- nosing pain using only physiological measures is often difficult. The physiological parameters associated with pain are often indis- tinguishable from those produced by other stressful events (Van Cleve, Johnson, & Pothier, 1996) and physiological responses in in- fants vary widely (McGrath, 1989). The best source of information for inferring pain is to examine the infant’s facial expressions. Fa- cial responses to pain are more specific and consistent than behav- ioral and physiologic responses (Craig, 1998). For this reason, the majority of pain assessment instruments developed for newborns, including COMFORT (Ambuel, Hamlett, Marx, & Blumer, 1992), CRIES (Krechel & Bilder, 1995), FLACC (Merkel, Voepel-Lewis, Shayevitz, & Malviya, 1997), MIPS (Buchholz, Karl, Pomietto, & Lynn, 1998), and CFACS (Gilbert et al., 1999), require careful obser- vations of facial activity. Fig. 1 illustrates some of the known facial patterns that are strongly associated with pain—prominent fore- head, eye squeeze, naso-labial furrow, taut tongue, and an angular opening of the mouth (Grunau, Grunau, & Craig, 1987). Even though facial activity indicative of pain is clearly discern- able, instruments that have relied on facial information have pro- ven unsatisfactory because of problems with observer bias (Prkachin, Solomon, Hwang, & Mercer, 2001). There are several sources of observer bias: personality of the observer, attitude to- wards the measure, the context, and desensitization due to re- peated exposure to suffering (Stevens, Johnston, & Gibbins, 2000; Xavier Balda et al., 2000). An excellent method for reducing bias would be to produce a machine system capable of detecting neona- tal pain. Several researchers have begun investigating machine assessment of some neonatal pain indicators. Lindh, Wiklund, and Håkansson (1999), for instance, have succeeded in detecting pain in heart rate variability, and Petroni, Malowany, Johnston, and Stevens (1995) have trained neural networks to discriminate differences in neonatal cries, including a cry in response to pain. However, relying solely on these systems is impractical as the neo- nates would need to be tethered to sensors. In many hospitals cam- eras are now in place for mothers to observer their infants. A more 0957-4174/$ - see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.eswa.2010.04.048 * Corresponding author. E-mail addresses: loris.nanni@unibo.it (L. Nanni), sbrahnam@missouristate.edu (S. Brahnam), alessandra.lumini@unibo.it (A. Lumini). Expert Systems with Applications 37 (2010) 7888–7894 Contents lists available at ScienceDirect Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa