Memory modulates color appearance Thorsten Hansen, Maria Olkkonen, Sebastian Walter & Karl R Gegenfurtner We asked human observers to adjust the color of natural fruit objects until they appeared achromatic. The objects were generally perceived to be gray when their color was shifted away from the observers’ gray point in a direction opposite to the typical color of the fruit. These results show that color sensations are not determined by the incoming sensory data alone, but are significantly modulated by high-level visual memory. Visual perception is inherently constructive and creative in nature, and frequently, additional assumptions and constraints are necessarily involved in generating a specific visual sensation 1 . Color is a particu- larly ill-posed problem, because the light entering our eyes is the product of the illumination and the surface reflectance of the object and therefore changes considerably when the illumination changes. Still, we are able to reliably perceive the color of objects. There are several known mechanisms involved in this process. Some mechanisms act on a local scale, computing cone excitation ratios across edges, whereas others act on a more global scale, computing the average color over larger image regions 2 . What these mechanisms have in common is that they are driven solely by the visual signals arising from the retina. Here we investigated whether the known color of objects also affects color appearance. Many natural and man-made objects have a typical or diagnostic color, which is termed their memory color. This knowl- edge could be used to guide our visual system in the process of assigning colors to objects and in discounting the illuminant. Previous studies have demonstrated that the memory color can be distinct from the measured color of the objects and is typically more saturated (refs. 3–9; and A. Hurlbert & Y. Ling, J. Vis. 5, 787a, 2005). However, the results of these studies were highly variable, and, because the subjects in these studies could never directly change the colors of the objects under investigation, it is not clear whether the effects were of a perceptual nature. We presented digitized photographs of natural fruit objects on a uniform gray background. The color of the fruit objects could be interactively manipulated to produce, for example, bananas of any arbitrary color. This, of course, poses the question: what does a blue banana look like? We used an adjustment method in which we started with the original color image and then allowed the subject to inter- actively scale and rotate the whole distribution of pixels in color space (Fig. 1 and Supplementary Methods online). This method satisfies the constraint that a gray banana under a neutral illumination should be achromatic: all pixels vary only along a luminance axis but not in their chromaticity. For all other settings, the banana was characterized by a whole distribution of chromaticities, and the relationship between relative saturation and luminance was kept stable. Our subjects had to adjust the color of the fruit objects until they appeared gray. In a different set of experiments, we asked the same subjects to adjust the color of the fruit objects until they appeared natural. It was evident that the settings for the banana (Fig. 2a) deviated from the neutral gray adaptation point at the origin of the color space, in the direction opposite to the typical setting (t 13 ¼ 6.915, P o 0.001). In actual fact, subjects adjusted the banana to a slightly bluish hue—its opponent color—in order for it to appear neutral gray. At the point where the banana was actually achromatic, at the origin of the color space, it still appeared yellowish. A similar effect was obtained for the other fruit objects we investigated (Fig. 2b). As a control, we asked our subjects to adjust uniform spots of light and random noise patches (Supplementary Methods), which do not have an association with a typical color. The settings for these stimuli (Fig. 2b) did not differ significantly from the neutral gray background (t 13 ¼ 1.39, P 4 0.05), but the difference between the controls and the fruit setting was significant (t 13 ¼ 4.22, P o 0.001). To quantify the memory color effect, we determined how far from the neutral control stimuli the subjects adjusted the fruits in the direction away from their typical setting. We then normalized the memory color index by dividing through the distance between the control settings and the fruit’s typical –10 –5 0 5 10 –90 –45 0 45 90 (L + M) – S (% cone contrast) a b L – M (% cone contrast) Figure 1 The chromatic adjustment method. (a) The distribution of chromaticities in the original photograph of the banana (yellow) in the isoluminant plane of a color space spanned by an L – M axis and an (L + M) – S axis. Subjects could adjust the color of the stimulus in two dimensions. This was achieved by rotating and scaling the whole distribution of chromaticities (for example, toward magenta). The black cross indicates the mean of the distribution and how it changes when it is rotated by 901 and its amplitude scaled to 70%. (b) Stimuli corresponding to the two chromatic distributions shown in a. Received 1 September; accepted 27 September; published online 15 October 2006; doi:10.1038/nn1794 Abteilung Allgemeine Psychologie, Justus-Liebig-Universita ¨t Giessen, Otto-Behaghel-Str. 10F, 35394 Giessen, Germany. Correspondence should be addressed to K.R.G. (gegenfurtner@uni-giessen.de). NATURE NEUROSCIENCE VOLUME 9 [ NUMBER 11 [ NOVEMBER 2006 1367 BRIEF COMMUNICATIONS © 2006 Nature Publishing Group http://www.nature.com/natureneuroscience