Proceedings of International Joint Conference on Neural Networks, Orlando, Florida, USA, August 12-17, 2007 LogTOTEM: A Logarithmic Neural Processor and its Implementation on an FPGA Fabric. P. Lee, E. Costa, S. McBader, L. Clementel, and A. Sartori Abstract- This paper describes the design of a neural network architecture optimised for use with the Reactive Tabu Search (RTS) training algorithm. The neural network is built using the hybrid-Logarithmic Number System (hybrid-LNS) instead of the traditional fixed-point methods for the multiply- accumulate (MAC) unit contained in each neuron. The circuits have been designed and implemented using between 4 and 8 bits of fractional precision for the logarithmic representation of the weights and the data. The architecture is based on the existing TOTEM VLSI chip and contains 32 neurons each having a 256 x 10-bit weight RAM. The device has been implemented on a Virtex XCV600 device where it consumed less 6025 slices with 4 bits of fractional precision of the logarithms and 6280 slices with 5 bits of fractional precision. At 4-bits This represents a (45%) reduction in logic resources required by the neuron array and an overall reduction of 10% of the FPGA resources when compared to the SoftTOTEM device built using the same technology. I. INTRODUCTION T HE TOTEM neural network architecture has been successfully applied to a number of ANN problems since it was first developed in 1995 [1] to accelerate the training of Multi-Layer Perceptrons (MLPs) using combinatorial optimisation methods based on the Reactive Tabu Search (RTS) . In recent years the original TOTEM architecture has undergone a series of enhancements from its original custom VLSI implementation and has recently been ported onto FPGA technology [2]. In [3] Lee et al considered the possibility of using logarithmic signal processing for implementing a "multiplierless" MAC unit for the TOTEM architecture and showed that this method had potential for providing faster computation and lower power dissipation than the equivalent linear arithmetic MAC unit used in the original devices. Theoretical simulations confirmed that the use of low-precision logarithmic representations for the weights and for performing the signal processing did not impact significantly on the performance of the system. A test VLSI Manuscript received February 12, 2007. (Write the date on which you submitted your paper for review.) This work was supported in part by Neuricam SpA, INFN and the University of Trento. P. Lee and E. Costa are with the Electronics Department at the Univerity of Kent, Canterbury, Kent, CT2 7NT UK. (phone: +44-1227-823251; fax: +44-1227-824066; e-mail: P.Lee 7, kent.ac.uk). S. McBader was with the University of Kent. She is now with Semtech Ltd, Unit 2-3 Park Court, Romsey, Hampshire SO51 9AQ, UK (e-mail: S.McBader osemtech.com). L. Clementel, and A. Sartori are with Neuricam SpA, Via Grazioli 71, 38100 Trento, Italy (e-mail: sartori oneuricam.com). version of the MAC unit has subsequently been designed [4]. Recent advances in FPGA technology have now made it possible to consider the use of a new logarithmic TOTEM architecture on a reconfigurable device. The use of the Logarithmic Number System (LNS) for building neural networks is not new and has been reported in the past by Arnold [5,6], Kwan [7] and Richfield [8]. Both These architectures use LNS for performing multiplication (division) and addition (subtraction) in the log domain. The alternative architecture used for the LogTOTEM processor and described in this paper is commonly known as the hybrid-logarithmic or hybrid-LNS system where multiplication (division) is performed in the log domain and addition (subtraction) is performed in the linear domain. This technique requires the use of efficient Lin2Log and Log2Lin converter algorithms to translate numbers from one domain to the other. As with addition in the LNS, conversion in hybrid-LNS arithmetic can become prohibitively expensive at resolutions of 12 bits and above. For the architecture in this paper both the weights and data have been calculated with between 4 and 8 bits of fractional precision for the logarithmic representation. As shown in [3], this is sufficient for acceptable performance of the TOTEM neural network. The paper shows that it is efficient to implement a classical MAC unit using the hybrid-LNS method at these levels of precision in a modern FPGA. The paper begins with a brief overview of the Reactive Tabu Search in Section II followed by a review of the TOTEM architecture in Section III. Section IV introduces logarithmic signal processing and describes the hybrid-LNS neuron developed for the LogTOTEM system. Section V reports the results achieved when implementing a LogTOTEM architecture on a Xilinx Virtex device. The paper concludes with an overview of further work to be carried out. II. THE REACTIVE TABU SEARCH (RTS). The Reactive Tabu Search (RTS) is a search algorithm that belongs to the framework of Tabu Search methods described by Glover [9]. A Tabu search optimises a given cost function, f, by using an iterative "greedy" component (modified local search) to bias the search toward points of low f values, while using prohibition strategies to avoid the occurrence of cycles. In Tabu Search algorithms the task is transformed into a combinatorial optimisation problem. The weights (N) of the network are treated as one long bit string of length L 1-4244-1 380-X/07/$25.00 ©2007 IEEE