J. Phys. zyxwvutsrq A: Math. Gen. 24 (1991) 2645-2654. Printed in the UK zyxwv Are spin-glass effects relevant to understanding realistic auto-associative networks? Alessandro Treves Department ol Experimental Psychology, Universtty of Oxford, South Parks Road, Oxford 0x1 3UD. UK Received 9 January 1991, in final form 22 February zyxwv 1991 Abstract. Elementary units characterized by zyxwv a threshold-linear (graded) response have been argued to model single neurons in auto-associative networks more realistically than binary units. The different way local activity is constrained in the two representations is z shown here to have important consequences For the spin-glass-like properties of otherwise equivalent systems. In particular, in contrast with their binary counterparts, the threshold- linear Sherrington~Kirkpatrick model is stable with respect to replica symmetry-breaking zy (nse), while threshold-linear fully connected neural networks with covariance learning are RSB unstable only in a very restricted region ol their phase diagram. Whether or not spin-glass effects dominate attractor dynamics is suggested to affect considerably, among other things, the ability al auto-associative memories to encode new information. 1. Introduction Since Amit, Gutfreund and Sompolinsky (AGS) [l] analysed the Little-Hopfield [2,3] model for auto-associative memory by adapting methods originally developed for studying spin-glasses, the appearance of spin-glass effects has been considered one of the typical features of the low-noise long-time limit behaviour of associative networks with feedback. In networks characterized by symmetric interactions, spin-glass freezing occurs, in a low noise phase, when the associated energy landscape is very ‘rough at the microscopic level. This roughness, induced by the quenched disorder in the interactions, may, if fast, ‘thermal‘ noise is low, carry over to the free-energy landscape. Then the system becomes unable to escape one of an exponentially large number of disorderly placed tiny valleys, and a pure thermodynamic state is characterized by a probability distribution confined to a few configurations. Spin-giass freezing unaoubtediy aifects the retrieval dynamics ofrhe memory. More importantly, it has been considered to undermine its ability to select and store meaning. ful incoming information. Parisi [4] has argued that if the network freezes into a spin-glass state while subject to an external stimulus varying in time (to be interpreted as meaningless), just as it freezes into a retrieval state when subject to a steady (and therefore meaningful) one, it will mistakenly store in the synaptic connections the irrelevant firing pattern characterizing that particular spin-glass slate. He has then suggested that this may be avoided if the interactions are asymmetric and, as a result, the system is less prone to freeze into restricted portions of phase space. This suggestion has raised the issue of how sensitive networks with asymmetric connections are to spin-glass effects, an issue which has been addressed both in the context of purely disordered systems [5] and in the specific one of auto-associative memories [6,7]. 0305-4470/Y1/112645+ 10$03.50 zyxwvut 0 1991 IOP Publishing Ltd 2645