On issues related to VLSI implementations of neural networks Sorin Draghici (1) , Valeriu Beiu (2) (1) Vision and Neural Network Laboratory, Department of Computer Science, 431 State Hall, Wayne State University, Detroit, MI 48202 Email: sod@cs.wayne.edu (2) Los Alamos National Laboratory, Division NIS-1, Mail Stop D466 Los Alamos, NM 87545, USA Email: beiu@lanl.gov If neural networks are to be used on a large scale in everyday life applications, they have to be suited to hardware implementation. This is because the hardware implementation satisfies best requirements related to reliability, size and speed and is also the most cost effective solution for large scale use. One problem related to this foreseeable transition towards hardware is that the actual software simulations of neural networks use floating point arithmetic and high precision weights. Storing so many bits for each weight and implementing floating point operations would make any hardware implementation unnecessarily bulky and expensive. Limited precision weights neural networks are better suited for such purposes because they require fewer bits for storing the weights and also simpler computations. In turn, this determines a decrease in size of the VLSI chip and therefore, a lower cost for the same performance or, alternatively, a better performance for the same price. This paper deals with adapting neural network algorithms and architectures to VLSI implementations in the context of classification problems. It has been shown that neural networks using limited integer weights can solve any classification problem if the weight range is chosen appropriately for the given problem. More specifically, [Draghici, 1997] has established a quantitative relationship between the necessary weight range and some problem specific values such as the minimum distance between patterns of opposite classes and the radius of the hypersphere including all patterns. Also, it has been shown that an entropy-based lower bound can be calculated for the number of bits (in the information entropy sense) used by the neural architecture. At the same time, it has been shown that a neural network is VLSI optimal if it respects certain conditions related to the weight range and the fan-in of the units [Beiu, 1997]. This paper presents a methodology for calculating the minimal size of a VLSI optimal network for a given problem. The method is based on existing theoretical results which are further detailed and completed. It is shown that the number of weights is bounded by: ( 29 ( 29 1 / 2 log 2 / 2 log min min max + ⋅ ⋅ ⋅ ⋅ d d d n m where d min is the minimum distance between patterns of opposite classes, d max is the maximum distance between any patterns, m is the number of patterns and n is the number of dimensions. A constructive algorithm using limited precision integer weights is used to construct and train neural networks for several problems. For each problem, the theoretical approach is used to calculate the necessary weight range and the lower bound for the number of bits necessary to solve the problem in the worst case. The actual weight range and number of bits are compared with the theoretical values given by the method presented in order to test the validity and the practical usability of the approach. References: [Draghici, 1997] - Draghici S., Sethi, I.K. - On the possibilities of the limited precision weights neural networks in classification problems, in Biological and Artificial Computation: From Neuroscience to Technology, Lecture Notes in Computer Science, J. Mira, R. Moreno-Diaz, J. Cabestany (Eds.), pp. 753-762, Springer Verlag, 1997 [Beiu, 1997] - Beiu V., Draghici S., Makaruk H.E. - On Limited Fan-In Optimal Neural Networks, In D.L. Borges and W. Martins (Eds.): Proc. IV SBRN, Goiana, Brazil, 3-5 December, IEEE CS Press, Los Alamitos, 1997.