1 Input Conditioned Subranging and Skewed Quantisation of MACs in IMC Ashwin Balagopal Sundar 1 , Graduate Student Member, IEEE, Janakiraman Viraraghavan 2 , Member, IEEE, Balaji Vijayakumar 3 , Graduate Student Member, IEEE Integrated Circuits and Systems Group, Department of Electrical Engineering, Indian Institute of Technology Madras-600036 Email: ee17d200@smail.iitm.ac.in 1 , janakiraman@ee.iitm.ac.in 2 , ee19d202@smail.iitm.ac.in 3 Abstract—In-Memory Computation(IMC) of Neural- Network(NN) inference is done by performing the Multiply- ACcumulate(MAC) operation in the analog domain. Parallelly digitising MAC voltages by fitting Analog to Digital Converters(ADCs) within dense memory-pitches is a fundamental challenge for IMC engines. IMC works thus far rely on clipping the MAC-PDF to reduce the dynamic range, reducing the per data-line(DL) ADC precision requirement. In this work, we show that the per-DL ADC precision can be reduced even further by focusing on quantising the input Conditioned MAC-PDF(CMPDF), which spans a sub-range in the total MAC-PDF. We demonstrate on hardware a technique to locate the CMPDF in one-shot by tracking its mean. We show that quantisation levels about the CMPDF mean can be skewed to only span the portion of CMPDF that yields positive ReLU inputs, provided MACs are implemented as complete sums. Compared to symmetrically spanning CMPDF, this requires 20% to 40% fewer references at iso-accuracy for the investigated neural network layers. Hardware measured results for Fully-Connected NN inference on MNIST yielded < 1% accuracy drop when MAC-voltages were quantised with 4 bit references about the CMPDF mean as compared to full-range 6 bit ADC sensing the MAC-voltages. MATLAB evaluation using 3.5 to 4.2 bit CMPDF quantisation for MNIST-FCNN, CIFAR-10 Resnet-20 and VGG-11 inference yielded < 1% accuracy drop as compared to full-range 6 to 7 bit MAC quantisation. Index Terms—Multi-Bit Activations, In-Memory Comput- ing (IMC), SRAM, Subranging, Skew Quantisation, Dot- Product, Multiply-Accumulate(MAC), Analog computing, Edge- Computing, Machine Learning (ML). I. I NTRODUCTION D EEP neural networks(DNN) are at the forefront of Ma- chine Learning(ML) algorithms for performing Artificial Intelligence inference tasks and several architectures have emerged that have been successful in Image Recognition [1], [2], Speech Recognition [3], [4] and Natural Language Processing [5], [6]. DNNs have already started to proliferate into our daily lives, with dedicated Neural Processing Units in mobile phones [7], conversational AI [8] and automated delivery robots [9], to name a few examples. Despite sparsifi- cation efforts [10]–[12], DNN operations remain quite data intensive. Thus, Von-Neumann(VN) deployments of DNNs must deal with the memory bottleneck [13], which is that memory accesses dominate latency and energy costs [14] in the Fetch-Execute flow. The memory bottleneck issue can prove quite challenging for edge-devices as they are resource constrained. This work was supported by SERB project number ECR/2018/000760 . DAC 1 DAC 2 DAC N w b 0,1 x 1 x 2 x N Σ i=1 N BL 0 BL k 2 -k Σ N w i x i BL B W -1 DATALINE Power of Two Summing WORD-LINE DRIVERS i=1 w b 0,2 w b 0,N w b k,1 w b k,2 w b k,N w b BW-1 ,1 w b k,i ADC V MAC α x i w b BW-1 ,2 w b BW-1 ,N Fig. 1: Dataflow architectural representation of column-major analog-domain IMC on one data-line One non-VN architecture that has attracted significant research interest in the last few years is In Memory Computing(IMC) [15], in which the guiding principle is to move input data to memory, where weights are stored, and per- form inference computation within the memory itself. Several works have been published detailing various on-Silicon im- plementations of IMC engines. These works can be classified based on the method in which Multiply-ACcumulate(MAC) is performed: current domain [16]–[20], charge domain [21]– [37] or fully digital [38] IMC. Additionally, some Non-CMOS [39]–[41] realisations of IMC have also been attempted. In both current and charge domain techniques, the MAC opera- tion is performed within the memory in an analog manner and results in a voltage that is proportional to the MAC. A dataflow architectural representation of Column-Major [23] IMC is shown in Fig. 1. Inputs(x i ) are driven through Digital to Analog Converters(DACs) onto word-lines(WL) and get multi- plied with weight bits(w b k,i ) to get bit-wise products(w b k,i x i ) that are summed, yielding a voltage(V MAC ) proportional to the MAC: V MAC ∝ ∑ N i=1 w b k,i x i . The conversion of these