Getting to Know Someone: Familiarity, Person Recognition, and Identification in the Human Brain Gyula Kovács Abstract ■ In our everyday life, we continuously get to know people, dom- inantly through their faces. Several neuroscientific experiments showed that familiarization changes the behavioral processing and underlying neural representation of faces of others. Here, we propose a model of the process of how we actually get to know someone. First, the purely visual familiarization of unfamiliar faces occurs. Second, the accumulation of associated, nonsensory infor- mation refines person representation, and finally, one reaches a stage where the effortless identification of very well-known per- sons occurs. We offer here an overview of neuroimaging studies, first evaluating how and in what ways the processing of unfamiliar and familiar faces differs and, second, by analyzing the fMRI adap- tation and multivariate pattern analysis results we estimate where identity-specific representation is found in the brain. The available neuroimaging data suggest that different aspects of the informa- tion emerge gradually as one gets more and more familiar with a person within the same network. We propose a novel model of familiarity and identity processing, where the differential activa- tion of long-term memory and emotion processing areas is essen- tial for correct identification. ■ INTRODUCTION The identification of the surrounding objects is an important ability of any living being. For social animals, such as the Homo sapiens, the identification of conspecific individuals is unarguably one of the most frequently accomplished and most important tasks in human life, which has obvious evolutionary benefits. In our everyday life, we continuously and effortlessly detect, discriminate, categorize at different levels, and, finally, recognize and identify other human beings. We do so by evaluating simultaneously many differ- ent static and dynamic sensory features, such as body shape, movement, voice, smell, and, above all, the face of other individuals. Person identification, therefore, is a multimodal process (Blank, Wieland, & von Kriegstein, 2014), involving various sensory but also semantic and contextual informa- tion as well. Still, most of our knowledge about person identification comes from facial identification, and corre- spondingly, the most influential models are also built on cognitive models of face identification. Therefore, in the following pages, we largely concentrate on the process of face-based person identification. Determining the facial identity of a person is computa- tionally a very challenging endeavor. This is because of the facts that (1) every face shares the very same features, which is why identifying the faces of different individuals requires special processing mechanisms (for a review, see Peterson & Rhodes, 2005), and (2) the facial features of a given individual show an enormous variability across time and space because of changes in illumination, viewpoint, facial expressions, hairstyles, makeup, and age, making the task of grouping the faces of the same individual together very difficult (Andrews, Jenkins, Cursiter, & Burton, 2015; Jenkins, White, van Montfort, & Burton, 2011). Reflecting these difficulties, an extensive network of subcortical and cortical areas has been allocated to face processing in the human brain (Figure 1; for reviews, see Rapcsak, 2019; Duchaine & Yovel, 2015; Gobbini & Haxby, 2007; Calder & Young, 2005; Haxby, Hoffman, & Gobbini, 2000). In this review, we modify the classification of the influential model of familiar face processing (Gobbini & Haxby, 2007), which describes recognition as the result of the interaction of a core network and an extended system. Currently, it is considered that occipital, fusiform, and superior temporal areas are parts of the core face process- ing system. According to most face processing theories, the core network is largely responsible for encoding visual appearances and is activated by any face-like stimulus. The face-sensitive visual areas of the inferior occipital gyrus (occipital face area [OFA]) are thought to be the gate- way of early, view-specific face processing steps (Pitcher, Walsh, & Duchaine, 2011; but for a different conclusion, see Ambrus, Amado, Krohn, & Kovács, 2019; Ambrus, Dotzer, Schweinberger, & Kovács, 2017; Ambrus, Windel, Burton, & Kovács, 2017; Rossion, 2014). The fusiform face area (FFA) seems to have a higher-order, more image- invariant face processing and is currently allocated to an intermediate level in face identification. The STS is Friedrich Schiller University Jena © 2020 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 32:12, pp. 2205–2225 https://doi.org/10.1162/jocn_a_01627 Downloaded from http://direct.mit.edu/jocn/article-pdf/32/12/2205/1862277/jocn_a_01627.pdf by guest on 12 October 2021