Abstract—Hopfield model of associative memory is studied in this work. In particular, two main problems that it possesses: the apparition of spurious patterns in the learning phase, implying the well-known effect of storing the opposite pattern, and the problem of its reduced capacity, meaning that it is not possible to store a great amount of patterns without increasing the error probability in the retrieving phase. In this paper, a method to avoid spurious patterns is presented and studied, and an explanation of the previously mentioned effect is given. Another technique to increase the capacity of a network is proposed here, based on the idea of using several reference points when storing patterns. It is studied in depth, and an explicit formula for the capacity of the network with this technique is provided. Keywords—Associative memory, Hopfield network, Network capacity, Spurious patterns. I. INTRODUCTION SSOCIATIVE memory has received much attention for the last two decades. Though numerous models have been developed and investigated, the most influential is Hopfield Associative Memory [1], based on his studies of collective computation in neural networks. Hopfield’s model consists in a fully-interconnected series of bi-valued neurons (outputs are either -1 or +1). Neural connection strength is determined in terms of weight matrix W, j i w , representing the synaptic connection between neurons i and j. This matrix is fixed, that is, once the learning phase (an application of Hebb’s postulate of learning [2]) has finished, no further synaptic modification is considered. Two main problems are found in this model: the apparition of spurious patterns and its low capacity. Spurious patterns are local minima of the corresponding energy function and not associated to any stored pattern. The capacity parameter α is usually defined as the quotient between the maximum number of patterns to load into the network and the number of used neurons that achieve an Manuscript received July 15, 2005. D. López-Rodríguez is with the Department of Applied Mathematics, at the University of Málaga, Spain. (Corresponding author. Phone: +34 952 132 866. e-mail: dlopez@ ctima.uma.es). E. Mérida-Casermeiro is also with the Department of Applied Mathematics, at the University of Málaga, Spain. (e-mail: merida@ ctima.uma.es). J. M. Ortiz-de-Lazcano-Lobato is with the Department of Computer Science and Artificial Intelligence, at the University of Málaga, Spain. (e- mail: jmortiz@lcc.uma.es). acceptable error probability in the retrieving phase. It has been shown that this constant is approximately 15 . 0 = α for Hopfield’s model. This value means that if the net is formed by N neurons, a maximum of N K α ≤ patterns can be stored and retrieved with very little error probability. McEliece [3] showed that the asymptotic capacity of the network is at most 2 log N N , if most of the prototype patterns are to remain as fixed points. This capacity decreases to 4 log N N if every pattern must be a fixed point. In this work, a technique to avoid the apparition of spurious states in Hopfield’s model is explained in terms of the decrease of the energy function associated to state vectors. The main contribution of this paper consists in an extension of this model as associative memory to overcome the problem of its reduced capacity. The organization of the paper is as follows: in Sec. II, a description of Hopfield model is given, putting special emphasis on its application as content-addressable memory. In Sec. III, the method to avoid the apparition of spurious patterns is presented. In Sec. IV, the associative memory model is extended by the use of multiple reference points, and in Sec. V, a study of the capacity of this new model is presented, similar to that presented in [4], followed by several consequences of importance. Finally, in Sec. VI some final remarks and conclusions are given, as well as possible future research lines. II. HOPFIELD’S MODEL A. The Network Hopfield’s model consists in a net formed by N neurons, whose outputs (states) are either -1 or +1. Thus, the state of the net at time t is completely defined by a N-dimensional state vector N N t V t V t V t V } 1 , 1 { )) ( , ), ( ), ( ( ) ( 2 1 − ∈ = K . Associated to every state vector there is an energy function that determines the behavior of the net: , 1 1 1 1 ( ) 2 N N N ij i j i i i j i EV w VV V θ = = = =− + ∑ ∑ ∑ (1) where j i w , is the connection weight between neurons i and j, and i θ is the threshold corresponding to i-th neuron. Since Hopfield Network as Associative Memory with Multiple Reference Points Domingo López-Rodríguez, Enrique Mérida-Casermeiro, and Juan M. Ortiz-de-Lazcano-Lobato A World Academy of Science, Engineering and Technology International Journal of Mathematical and Computational Sciences Vol:1, No:7, 2007 324 International Scholarly and Scientific Research & Innovation 1(7) 2007 scholar.waset.org/1307-6892/6850 International Science Index, Mathematical and Computational Sciences Vol:1, No:7, 2007 waset.org/Publication/6850