IN THE EYE OF THE BEHOLDER: THE PERCEPTION OF INDETERMINATE ART Christian Wallraven, Kathrin Kaulard, Cora Kürner 1 , Robert Pepperell 2 1 Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany. E-mail: <chris- tian.wallraven@tuebingen.mpg.de> 2 University of Wales Institute Cardiff, Cardiff CF24 0SP, UK. E-mail: <rpepperell@uwic.ac.uk> Introduction How do we interpret an object, a scene, or a painting? Perception research and art illuminate from different views how the vast amount of information in our visually perceived environment is proc- essed by the viewer to form a coherent and consistent interpretation of the world. Using paintings by the artist Rob- ert Pepperell, this paper outlines some scientific research that attempts to con- nect these different world views. Pep- perell's paintings may at first glance appear as baroque frescoes, expressionist still-lifes, or cubist collages. Taking a closer look, however, these concrete interpretations vanish and we are left with indeterminate images [1]. Using psychophysical experiments and eye tracking measures, in this work we seek to illuminate the visual processing of information in Pepperell's paintings. More specifically, we investigated how the pattern of fixations — the loci of interest — change as a function of the task (e.g. 'What is depicted in this scene?' vs. 'Does this image contain peo- ple?') and of the image content. The in- terpretation of the experimental results in the context of perceptual research offers insights into the perception of indetermi- nate art. At the same time, the results may contribute to the development of the artist’s project, as they provide some measurable validation of his intentions. Eye tracking and Behavioural Experiments As is known from the early experiments by Yarbus [2], eye fixations are critically dependent on the task, i.e. how one looks at an image differs depending on what is being looked for. In our experiment, we were interested to investigate the differ- ence between local and global process- ing of Pepperell's paintings. To do this we asked participants to view a set of paintings and to solve two different vis- ual tasks while recording their eye activ- ity. In order to test local information processing strategies we asked whether a person could be found on the painting (person task), whereas global informa- tion processing was tested by asking subjects to categorize the painting into one of seven pre-defined categories (categorization task). Methods A total of 30 indeterminate drawings and paintings were shown to 20 participants. These 30 works of art were split into two sets of 15 paintings that were used in each task. The first 10 participants saw the first set for the person task and the second set for the categorization task, whereas for the remaining 10 partici- pants we changed the sets for each task in order to counterbalance for possible order effects. Afterwards, a behavioural task was set in which a question relating to each of the two tasks was shown on the screen and participants had to answer by selecting the appropriate item from a list. For the person task, the question was 'Was there a person in this image?' and for the categorization task the question was 'To which of the following catego- ries does this image belong?' Here the choice consisted of seven categories: 'Biblical scenes', 'Landscapes with per- son', 'Landscape without a person', 'Por- trait', 'Still-life', 'Battle scene', and 'None of the above', these categories having been determined in prior pilot experi- ments. Their response time to answering these questions was also measured. Overall Results In general, the behavioural data shows that participants were able to consis- tently categorise some images whereas some images could not be classified con- sistently at all. The results showed that overall the paintings, indeed, correspond closely to Pepperell's artistic intention of providing indeterminacy. In addition, participants used the 'none-of-the-above' category for 26% of all responses, again showing that some paintings defy easy categorization. Taken together, these results provide a rough, yet very consis- tent validation of Pepperell's indetermi- nate paintings in terms of their category membership. The eye tracking data shows a differ- ence in activity between the two tasks — person spotting and scene categoriza- tion. The top pair of images in Fig. 1 shows a much more centered distribution of fixations when undertaking the person spotting task, whereas the bottom pair in Fig. 1 shows a broader, more localized spread when doing the categorization task [3]. The processed images on the right, showing the fixation clusters, demonstrate this effect even more clearly. The average number of fixation clusters in the categorization task is 288, whereas for the person detection task it is much higher at 413 unique clusters. In summary, the averaged fixation maps and the cluster analysis have shown that global information seems to be more important for the categorization task than local information. Fig. 1 Averaged eye tracking fixation maps (left pair) and fixation clusters (right pair) for both tasks (person spotting and catego- rization) on all the images in the study [3]. (© C. Wallraven) Image Analysis In the following sextion we will take a closer look at three selected paintings — Fig. 2 A, B and C — from the 30 used in the study that are representative of the general trends we observed in the data and interpret the fixation patterns and behavioral results for each [3]. Fig. 2A. Categorization task: This image was classified as a landscape (with or without a person) in the categorization task. The fixation pattern clearly reflects the search strategy discussed above: during categorization participants scanned nearly all of the area of the painting compared to the person task which shows a much more localized distribution. Person task: 7 out of 10 participants recognized a person in this painting. As this painting was unanimously classified as a landscape scene, we assume that participants looked for persons in the image that were 'at a distance', i.e., ob- jects that occupied only a small image area. The center of the image, which gathered the largest amount of fixations in this task, does indeed contain a few possible items of interest. This area is