740 nature neuroscience • volume 2 no 8 • august 1999 articles Many neurons in the primary visual cortex are tuned to orien- tation, and their responses as a function of orientation, known as tuning curves, are typically bell-shaped (Fig. 1a). From these tuning curves, one can predict how neurons will respond, on average, to a given orientation. However, neurons are noisy, and a neuron whose average response to a grating at a particular ori- entation is 20 spikes per second might respond at 18 spikes per second on one trial and 23 spikes per second on the next. This variability is evident when we plot the activity of a population of neurons produced by a grating presented at 90 degrees. If the activity of each neuron is plotted on one trial as a function of its preferred orientation, the resulting pattern looks like a noisy hill (Fig. 1b). On another trial, the presentation of the same stimulus would lead to a similar hill, but each cell would respond slightly differently. The task faced by the brain is to estimate, on each trial, the orientation of the grating from this noisy hill. The task of estimating encoded variables is not specific to ori- entation; many sensory and motor variables are encoded through the activity of large populations of neurons with bell-shaped tun- ing curves 1,2 . How does the brain perform this estimation, and how well can it do? Several methods, also known as estimators, have been proposed to ‘read out’ these noisy hills, that is, to extract the encoded variable or variables based on the observed activity 3 . One such method is the population vector estimator 2 , which assigns to each neuron a vector whose length is proportional to the neuron’s activity and whose direction corresponds to its pre- ferred orientation, sums all the individual vectors to form a pop- ulation vector, and then estimates the orientation from the angle of the population vector. This is mathematically equivalent to finding the cosine function that best fits through the pattern of activity and using the position of the peak of the cosine as the esti- mate of direction 4,5 (Fig. 1c). This method has received consider- able attention recently, primarily because of its mathematical simplicity. Is it, however, the optimal method? A natural way to answer this question is to present the same orientation repeated- ly and compute the mean and variance of the estimate. Recall that the hill of activity changes from trial to trial because of the neu- ronal noise, even if the orientation stays the same. As a result, the estimate also changes from trial to trial. An optimal estimator should be right on average; that is, the mean estimate should equal the presented orientation. An estimator that is right on average is referred to as unbiased, and it is the only type we consider in this paper. An optimal estimator should also have minimum vari- ance; that is, the estimate should be as similar as possible from trial to trial when the orientation is held fixed. It is possible to derive a lower bound on the variance of the estimator if one knows the structure of the neuronal noise, and an estimator is said to be optimal if its variance is equal to this lower bound. Although the population vector estimator is unbiased, its vari- ance is typically well above the lower bound dictated by the noise; thus, it is not optimal. The problem with the population vector can be seen in Fig. 1c—the cosine function is not the right tem- plate for this particular activity pattern. Instead, one should fit a template derived from the tuning curves of the cells, as illus- trated in Fig. 1d. Fitting the optimal template is known as max- imum likelihood, and this type of estimator reaches the lower bound dictated by the noise (at least for the case considered here, in which the noise exhibited by each of a large number of neu- rons is independent of the noise exhibited by the others 4–6 ). An estimator that reaches the lower bound is often referred to as an ideal observer, because it performs as well as possible given the noise. Because an ideal observer provides an objective yardstick against which one can measure the performance of an animal, it has been used in several recent studies to relate neuronal vari- ability to behavioral variability 7–9 . A natural question is whether biologically plausible networks can implement a maximum likelihood estimator. We show here that the answer is yes, provided that the level of neuronal noise is independent of firing rate. In this case, recurrent networks of nonlinear units with broad tuning curves—the kind of networks found throughout cortex—can achieve maximum likelihood. When the neuronal noise is more Poisson-like, so that the vari- ance increases with mean activity as observed in cortical neu- rons 10–12 , then the type of network considered in this paper is a close approximation to maximum likelihood. To illustrate these results, we simulated a recurrent network Reading population codes: a neural implementation of ideal observers Sophie Deneve 1 , Peter E. Latham 2 and Alexandre Pouget 1 1 Brain and Cognitive Science Department, University of Rochester, Rochester, New York 14627, USA 2 Department of Neurobiology, University of California, Los Angeles, Los Angeles, California 90095-1763, USA Correspondence should be addressed to A.P. (alex@bcs.rochester.edu) Many sensory and motor variables are encoded in the nervous system by the activities of large popula- tions of neurons with bell-shaped tuning curves. Extracting information from these population codes is difficult because of the noise inherent in neuronal responses. In most cases of interest, maximum likeli- hood (ML) is the best read-out method and would be used by an ideal observer. Using simulations and analysis, we show that a close approximation to ML can be implemented in a biologically plausible model of cortical circuitry. Our results apply to a wide range of nonlinear activation functions, suggest- ing that cortical areas may, in general, function as ideal observers of activity in preceding areas. © 1999 Nature America Inc. • http://neurosci.nature.com © 1999 Nature America Inc. • http://neurosci.nature.com