Implementation of Memristive Neural Networks with Spike-rate-dependent Plasticity Synapses Yide Zhang, Zhigang Zeng and Shiping Wen Abstract— The property of changing resistance according to applied currents of memristors makes them candidates for emulating synapses in artificial neural networks. In this paper, we introduce a memristive synapse design into neu- ral network circuits. Combined with modified integrate-and- fire (I&F) complementary metal-oxide-semiconducter (CMOS) neurons, the memristive neural network shows similarities to its biological counterpart, in respect of biologically realistic, current-controlled spikes and adaptive synaptic plasticity. Then, the spike-rate-dependent plasticity (SRDP) of the synapse, an extended protocol of the Hebbian learning rule, is originally implemented by the circuit. And some advanced neural ac- tivities including learning, associative memory and forgetting are realized based on the SRDP rule. These activities are comprehensively validated on a neural network circuit inspired by famous Pavlov’s dog-experiment with simulations and quan- titative analyses. I. I NTRODUCTION T HE BRAIN has excellent parallel computation capa- bility and ultra high integration density, outperforming today’s most advanced computer in the world. For this rea- son, building a new-type computer to emulate the brain has always been a target for scientific and engineering research [1]. However, due to von Neumann architecture’s occupation in computer industries, hardware and software in modern computers are principally designed for serial computing, with separated data storage and computation units. Several decades ago, artificial neural networks, the novel computing architecture emulating the brain structure, were proposed. Neural networks are able to conduct parallel computing, and perform data storage and computation simultaneously on a single component, the synapse. To make use of these advantages, attempts have been proposed to build neural networks on silicon chips [2], [3], [4], [5]. The features of neural networks, especially the synaptic plasticity, however, are not easy to be implemented by solely utilizing CMOS transistors. Although designs of artificial synapses in analog VLSI circuits have been presented [6], the circuits are gener- ally too complex to be integrated in large scale, diminishing the performance of the brain-like system. Yide Zhang, Zhigang Zeng and Shiping Wen are with the School of Automation, Huazhong University of Science and Technology, and Key Laboratory of Image Processing and Intelligent Control of Education Min- istry of China, Wuhan 430074, China (email: edwardchang@hust.edu.cn, hustzgzeng@gmail.com, wenshiping226@126.com). This work was supported by the Natural Science Foundation of China under Grant 61125303, National Basic Research Program of China (973 Program) under Grant 2011CB710606, the Program for Science and Tech- nology in Wuhan of China under Grant 2014010101010004, the Program for Changjiang Scholars and Innovative Research Team in University of China under Grant IRT1245. Recently, following HP Labs’ demonstration of T iO 2 memristor [7] (short for memory resistor [8]), a plenty of researches have been focused on the device [9], [10], [11], [12]. One research direction is concentrated on its characteristic of changing internal state following current or voltage excitation. The feature is analogous to plasticity of synapses, whose weight varies according to action potentials of adjunct neurons. The device has also been proven to be suitable for neural network applications theoretically due to its high nonlinearity and dynamic property [13], [14]. Since the designs of CMOS synapses are not suitable for large scale integration due to their complexity, it is reasonable to adopt memristors into the design of artificial synapses. The combination of memristive synapses and CMOS neurons is defined as the memristive neural network, which has richer dynamic behaviors than an ordinary artificial neural network [15]. In this emerging area, pioneering works have been conducted [16], [17], [18], [19]. Some important features of neural networks, such as synaptic multiplication [20], spike- timing-dependent plasticity (STDP) [21], unsupervised learn- ing [22], [23] etc., have been thoroughly studied and realized on memristive neural networks. Besides these features, there is another basic characteristic of neural networks, the spike- rate-dependent plasticity (SRDP), which claims the synaptic plasticity’s weight is dependent on pre-synaptic spiking rate [24]. The SRDP rule is captured by the BCM model [25], and it is an extended protocol of the Hebbian learning rule [26], [27]. Li et al. realized SRDP on a single synaptic device, the Ag/conducting polymer/Ta memristor [24]. Wang et al. also implemented the protocol on a memristor, whose material is amorphous InGaZnO [28]. However, their works only considered the SRDP protocol on a single memristive device with precisely designed signals emulating action potentials. The research about the implementation of SRDP in a neural network, where advanced neural activities happen, is scarcely discussed. In this paper, we propose a new design of memristive neu- ral networks consisting of memristive synapses and CMOS neurons, where the SRDP protocol and some advanced neural activities are originally implemented. SPICE simulations are conducted to show bio-inspired features of these models. The neuron model capable of generating biologically realistic spikes is a modification of the leaky integrate-and-fire (I&F) model derived from Mead’s work [2], [29]; the relationship between the neuron’s spiking frequency and input current is established quantitatively. The synapse model is based on HP’s T iO 2 memristor, in which a tunable nonlinear window function with two controlling parameters is included; 2014 International Joint Conference on Neural Networks (IJCNN) July 6-11, 2014, Beijing, China 978-1-4799-1484-5/14/$31.00 ©2014 IEEE 2226