Identification of the source and target domains in a visual metaphor and the role of perceptual similarity therein: An eye-movement study Amitash Ojha (amitashojha@research.iiit.ac.in) Cog Sci Lab, International Institute of Information Technology Hyderabad, India Bipin Indurkhya (bipin@iiit.ac.in) Cog Sci Lab, International Institute of Information Technology Hyderabad, India Abstract In two eye movement studies we try 1) to shed some light on the issue of identification of target and source in a visual metaphor and 2) try to explore the role of perceptual similarity in metaphor interpretation. In first experiment we found that viewer must notice the incongruity in order to come up with a metaphorical interpretation when he is presented with a visual metaphor (where only one object is depicted). We also show, using eye movement data, that when viewers fixate on the incongruous object they identify the target of metaphor correctly and if they do not focus on incongruous object, the target is identified erroneously. In second experiment we hypothesize that perceptual similarity plays a key role in forming a metaphorical interpretation. we found that as far as pictorial similes are concerned, attention must be focused on the perceptual similarities between the two images in order for a viewer to come up with a metaphorical interpretation. Moreover, the attention is shared between the perceptually similar regions of the two images in a symmetric way, suggesting that they are being compared during metaphorical interpretation. Keywords: Pictorial metaphor; Incongruity; Fixation, Saccade; Attention Introduction and Background The origin of the view that metaphor plays a key role in cognition can be attributed to I.A. Richards (1936): “Thought is metaphoric, and proceeds by comparison, and the metaphors of language derive therefrom.” However, much of the contemporary research on metaphor has remained focused on metaphors of language. More recently, Lakoff and Johnson (1980) argued with numerous examples that “metaphor is primarily a matter of thought and action and only derivatively a matter of language.” Though this work spawned a whole new generation of metaphor researchers, the focus still remained largely on verbal manifestations of the underlying metaphoric thought. There have been only a few studies on non-linguistic modes of metaphor, and they have been mostly concerned with visual metaphors ( Blackwell, 1998; Forceville, 1996; Kennedy, 1982, 1997;Lundmark, 2005, Noel, 1994) Even then, there is no consensus on such basic questions as: What are the source and the target domains of a visual metaphor and how are they identified? How do the two domains interact to generate the metaphorical meaning? If features are mapped, then what features of the source domain are mapped onto the target domain? (Forceville 2002.) Our goal in this research is to shed some light on these issues using eye- movement studies. Here we report on the results of two pilot experiments towards this goal. This study focuses on two kinds of visual metaphors. One is what is called homospatial metaphor by Carroll (1994) and contextual metaphor by Forceville (2007). Two examples of such metaphors are shown in Figure 1. In the first one, the head of the boy is replaced with a loudspeaker. The metaphor may be paraphrased as “the boy’s mouth is a loudspeaker.” Here the target of the metaphor, the boy’s head is absent, and is strongly implied by the context. The source of the metaphor, the loudspeaker, is used to replace the head, so homospatiality triggers the interaction between the source and the target. (A verbal counterpart of such metaphors might be “the butcher operated on the patient”, where the words ‘operated’ and ‘patient’ suggest the concept of ‘surgeon’, which is the target of the metaphor.) One other characteristic necessary for visual metaphors that Carroll posited is incongruity, namely that the source, which is the object replacing the target, should be incongruous with the rest of the picture. Based on this, we hypothesize that the viewer must notice the incongruity (pay more attention to it) in order to come up with a metaphorical interpretation. It is this hypothesis that we empirically investigate in the first experiment. The other kind of visual metaphor that we chose for this study is what Forceville calls pictorial simile. An example of this is shown in Figure 5, which can be paraphrased as “The sun is like an apple” (or “The apple is like sun”, for there is no implied directionality in the visual juxtaposition.) Here two images are juxtaposed and the viewer may or may not form a metaphorical mapping between them. Our hypothesis is that perceptual similarity plays a key role in forming a metaphorical interpretation (Indurkhya 2006, Sec. 8.3). However, the problem in testing this hypothesis is how to determine perceptual similarity without causing the participants to form metaphors inadvertently. To get out of this impasse, we decided to use a computer-based image-search system called FISH to establish perceptual similarity (Indurkhya, Kattalay, Ojha and Tandon 2008). In the second experiment we investigate the effect of perceptual similarity (as implemented in FISH