IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE (IN-PRESS) 1 Unlocking the potential of two-point cells for energy-efficient training of deep nets Ahsan Adeel 1,2,3* Adewale Adetomi 2 Khubaib Ahmed 2 Amir Hussain 4 Tughrul Arslan 5 W.A. Phillips 6 Abstract—Context-sensitive two-point layer 5 pyramidal cells (L5PC) were discovered as long ago as 1999. However, the potential of this discovery to provide useful neural computation has yet to be demonstrated. Here we show for the first time how a transformative L5PC-driven deep neural network (DNN), termed the multisensory cooperative computing (MCC) architecture, can effectively process large amounts of heterogeneous real- world audio-visual (AV) data, using far less energy compared to best available ‘point’ neuron-driven DNNs. A novel highly- distributed parallel implementation on a Xilinx UltraScale+ MPSoC device estimates energy savings up to 245759 × 50000 μJ (i.e., 62% less than the baseline model in a semi-supervised learning setup) where a single synapse consumes 8e -5 μJ. In a supervised learning setup, the energy-saving can potentially reach up to 1250x less (per feedforward transmission) than the baseline model. This remarkable performance in pilot experi- ments demonstrates the embodied neuromorphic intelligence of our proposed L5PC based MCC architecture that contextually selects the most salient and relevant information for onward transmission, from overwhelmingly large multimodal information utilised at the early stages of on-chip training. Our proposed approach opens new cross-disciplinary avenues for future on- chip DNN training implementations and posits a radical shift in current neuromorphic computing paradigms. I. I NTRODUCTION Conventional point neuron [1][2] inspired DNNs have demonstrated ground-breaking performance improvements in a wide range of real-world problems, ranging from image recognition [3] to speech processing [4][5][6]. Scientists have also designed point neuron inspired sophisticated computer architectures e.g., Intel’s Loihi [7], IBM’s TrueNorth [8], SpiNNaker [9], Neurogrid [10], BrainSclaseS [11], MNIFAT [12], DYNAP [13], DYNAP-SEL [14], ROLLS [15], Spirit [16], DeepSouth [17], Tianjic [18], ODIN [19], and Intel SNN chip [20]. However, point neuron-driven technologies are often economically, technically, and environmentally un- sustainable [21][22]. Their unrealistically high computational demand and complexity scale so rapidly that the technology becomes burdensome [21]. When a single leaky integrate- and-fire (LIF) point neuron fires, it consumes significantly more energy compared to the equivalent computer operation, and an unnecessary fire not only affects the neurons it is directly connected to, but also others operating under the same energy constraint [23]. The unnecessary neural firing * 1 CMI Lab, University of Wolverhampton, Wolverhampton. 2 Oxford Com- putational Neuroscience, Nuffield Department of Surgical Sciences, University of Oxford, Oxford. 3 deepCI.org, 20/1 Parkside Terrace, Edinburgh. 4 School of Engineering, University of Edinburgh, Edinburgh. 5 Edinburgh Napier University, Edinburgh. 6 Department of Psychology, University of Stirling, Stirling. Email: ahsan.adeel@deepci.org leads to unnecessary information transmission that creates a huge demand on energy consumption by the system as a whole. Yet, such models can learn, sense and perform complex tasks continuously, but at energy levels that are currently unattainable for modern processors. The fundamental problem is attributed to the simplified LIF neural structure that processes every piece of information it receives, irrespective of whether or not the information is useful to other neurons or the long-term benefit of the whole network [29]. This approach increases the overall neural activity or contradictory messages to high perceptual levels, leading to energy-inefficient and hard to train DNNs [29]. Fur- thermore, the lack of dynamic cooperation between neurons make these DNNs intolerant of faults. A simple illustration of point neuron and point neuron based neural network is presented in Fig. 1. The point neuron integrates all incoming streams in an identical way i.e., simply summing up all the excitatory and inhibitory inputs, with an assumption that they have the same chance of affecting the neuron’s output [1]. In contrast, biologically inspired two-point neurons transmit information only when the received information is relevant 1 to the task at hand, and not otherwise [29]. Recent neurobiological breakthroughs [31][32] have dis- covered neocortical neurons with two functionally distinct points of integration (apical and basal) in thick-tufted layer 5 pyramidal cells of the mammalian neocortex. However, it has not been demonstrated until now how these cells can provide useful neural computation. Although a few machine learning experts such as G. Hinton [33], T.P. Lillicrap [34], R. Naud [35] and Y. Bengio [36] have been inspired by the discovery of two-point L5PC, their papers have focused predominantly on learning. In contrast, our work uses context to guide both ongoing processing and learning [29]. Specifically, it is shown how the apical zone receives input from diverse cortical and subcortical sources as a context that selectively amplifies and suppresses the transmission of coherent signals and conflicting signals, respectively. This style of biologically plausible cooperative context-sensitive style of information processing is shown to have information processing capa- bilities of the kind displayed by the neocortex, performing better than current ‘point’ neuron-driven deep learning (DL) algorithms [29]. Building on this work, here we demonstrate the energy-efficiency of these new forms of context-sensitive machine learning algorithms. The main contributions of this paper are as follows: 1 Relevant (coherent) information refers to the portion of input information being logical and consistent with other portions of input information from the source data. arXiv:2211.01950v2 [cs.NE] 14 Nov 2022