Area–Time Performances of Some Neural Computations Valeriu Beiu, Member, IEEE, Jan A. Peperstraete, Joos Vandewalle, Fellow, IEEE, and Rudy Lauwereins Katholieke Universiteit Leuven, Department of Electrical Engineering Division ESAT, Kardinaal Mercierlaan 94, B-3001 Heverlee, Belgium Abstract—The paper aims to show that VLSI efficient im- plementations of Boolean functions (BFs) using threshold gates (TGs) are possible. First we detail depth-size tradeoffs for COM- PARISON when implemented by TGs of variable fan-in (∆); a class of polynomially bounded TG circuits having O ( lgn ∕ lg∆29 depth and O ( n ∕ ∆29 size for any 3 ≤ ∆ ≤ clgn, improves on the pre- vious known size O ( n29 . We then proceed to show how fan-in in- fluences the range of weights and of thresholds, and extend these results to Fn,m, the class of functions of n variables having m groups of ones. We conclude that the fan-in could be used by VLSI designers for tuning the area-time performances of neural chips. Index terms—neural networks, threshold circuits, Boolean functions, Fn,m, fan-in, area-time complexity, VLSI. I. INTRODUCTION I N this paper we shall consider neural networks (NNs) made of linear threshold gates (TGs), each one computing a Boolean function (BF) f : {0,1} n ➞ {0,1}. If the input vector is Z = (z 0 , …, z n-1 29 ∈ {0,1} n : f (Z29 = sgn ( ∑ n - 1 i = 0 w i z i + t29 , where w i are the synaptic weights, t is known as the threshold, and sgn is the signum function. A feedforward NN will be a feedforward TG circuit having as cost functions: (i) depth (number of layers), and (ii) size (number of TGs). These are linked to T = delay (depth) and A = area (size) of a VLSI chip. But TGs do not closely follow this proportionality as: (i) the area of the connections counts [1]; (ii) the area of one TG is related to its associated weights and threshold [2], [3]. Bes- ide, there are also sharp VLSI limitations: (i) the maximal value of the fan-in cannot grow over a limit [4]; (ii) the max- imal ratio between the largest and the smallest weight [5]. The maximum fan-in will be denoted by ∆ (notation from [5]) and is defined to be the largest number of inputs that con- nect to any single TG. This maximum fan-in of a three layer neural network has been called “order” (of a network) [6]. Al- though this measure has been largely neglected by the current literature, it has a long history going back to the mid 60s when TGs have been intensively studied. In Section II we analyze depth-size tradeoffs of COMPARI- SON as it is a Fn,1 function (it has just one group of ones in its truth table). As parameter which dictates such tradeoffs we have chosen the maximum fan-in, but we shall prove that it also determines the range of weights and of thresholds. These will be used to estimate the area-time performance of a VLSI implementation. By defining a more precise estimate of the area (which takes into account the values of the weights and the thresholds), closer estimates of AT 2 are presented and com- pared with that of the best known depth-size solution for COM- PARISON. Putting together these results with similar ones for MAJORITY functions, we are able to develop in Section III a systematic solution for Fn,m functions (BFs of n variables hav- ing m groups of ones [7]). Section IV presents several con- clusions and further directions of research. II. COMPARISON A. Depth and Size X and Y are binary numbers of n bits each: X = x n-1 …x 0 , Y = y n-1 …y 0 , COMPARISON [8], [9], [10], [11] being defined: C n > (≥29 (X,Y29 =    1 if X > Y (X ≥ Y29 0 if X ≤ Y (X < Y29 . (1) These are isobaric function [12], [13], and: Cn ≥ (X,Y29 = Cn > (X + 1,Y29 = Cn > (X,Y - 129 . (2) It is known [8], [14], [15], that with unbounded fan-in TGs COMPARISON: (i) cannot be computed by a single TG with polynomially bounded integer weights; (ii) can be computed by a single TG but having exponential large weights; (iii) can be computed in a depth-2 NN with O(n 4 ) TGs and polynomi- ally bounded weights; (iv) can be computed in a depth-3 NN of size O(n29 with polynomially bounded weights. In the following we will show that COMPARISON can be decomposed in trees where the leaves are COMPARISONs on a reduced number of bits and the nodes of these trees combine partial results of COMPARISONs (Lemma 1). Second we will prove that the nodes implement linearly separable functions In P. Borne, T. Fukuda, S.G. Tzafestas (eds.): SPRANN’94, GERF EC, Lille, France, April 1994. Manuscript received October 1, 1993. This work was partly carried out in the framework of a Concerted Research Action of the Flemish Community, entitled: “Applicable Neural Networks,” and partly supported by a Doctoral Scholarship Grant offered to Valeriu Beiu by the Academic Board of the Katholieke Universiteit Leuven. Valeriu Beiu is on leave of absence from “Politehnica” University of Bucharest, Department of Computer Science, Spl. Independentei 313, RO- 77206 Bucharest, România. Rudy Lauwereins is Senior Research Associate of the Belgian National Fund for Scientific Research. 664