IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 46, NO. 3, MARCH 1998 301 Fast Sequential Implementation of “Neural-Gas” Network for Vector Quantization Clifford Sze-Tsan Choy and Wan-Chi Siu, Senior Member, IEEE Abstract— Although the “neural-gas” network proposed by Martinetz et al. in 1993 has been proven for its optimality in vector quantizer design and has been demonstrated to have good performance in time-series prediction, its high computational complexity ( ) makes it a slow sequential algorithm. In this letter, we suggest two ideas to speedup its sequential realization: 1) using a truncated exponential function as its neighborhood function and 2) applying a new extension of the partial distance elimination method (PDE). This fast realization is compared with the original version of the neural-gas network for codebook design in image vector quantization. The comparison indicates that a speedup of five times is possible, while the quality of the re- sulting codebook is almost the same as that of the straightforward realization. Index Terms— Neural-gas network, partial distance elimina- tion, vector quantization. I. INTRODUCTION V ECTOR QUANTIZATION (VQ) [1] is a process through which a vector space is partitioned, such that an input in each partition is represented by a vector called a codeword or a codevector. Suppose in a -dimensional Euclidean space , the probability density function (pdf) of vector is given by The distortion introduced by replacing a vector with a codeword is denoted by , and the squared distortion measure is a popular one, i.e., (1) where and are the values of the th dimensions of and , respectively. Given codewords, (which are collectively referred to as a codebook), the vector space is partitioned into regions satisfying s.t. , and s.t. The objective is to design a codebook which minimizes the expected distortion introduced. The generalized Lloyd algorithm (GLA) [2] is a well- established approach for designing a codebook in vector quantization. The GLA requires a set of training vectors, which approximates the distribution of vectors from the source to be vector-quantized. An initial codebook of codewords is randomly selected, which partitions the training set into Paper approved by E. Ayanoglu, the Editor for Communication Theory and Coding Application of the IEEE Communications Society. Manuscript received February 15, 1997; revised August 15, 1997. This work was supported by the Research Grant Council of the University Grant Committee under Grant HKP152/92E (PolyU340/939). The authors are with the Department of Electronic Engineering, Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong (e-mail: stchoy@en.polyu.edu.hk; enwcsiu@hkpucc.polyu.edu.hk). Publisher Item Identifier S 0090-6778(98)02116-3. disjoint subsets using the nearest neighbor criterion as described in the previous paragraph. The centroid of each subset is then determined, which forms a new codebook. This reduces the expected distortion introduced on this training set. This process is iterated until either there is no reduction in expected distortion or a prespecified number of iterations has completed. Since the GLA designs the codebook based on the training set, if the distribution of vectors from the source changes with time, a new training set has to be defined and the GLA be reapplied to define a new codebook. Hence, the GLA is considered as an offline or batch approach which is not suitable for designing a codebook which adaptively tracks source characteristics. The -means algorithm [3] can be considered as an online or adaptive version of the GLA, in which vectors sampled from the source are used to adaptively modify the codebook and, hence, can adapt to changes in vector distributions. However, the quality of the codebooks from both approaches depends highly on the initial codebooks, which means that they are very likely to produce poor quality codebooks. Recently, a number of neural networks have been suc- cessfully applied to codebook design [4]–[7]. These neural approaches are characterized by their adaptive nature and are highly insensitive to initial conditions. The “neural-gas” net- work proposed by Martinetz et al. [7] is particularly important since it was proven that in the limit of large the density function of codevectors is proportional to when the squared distortion measure (1) is used. This is equivalent to the density distribution of codevectors of an optimal codebook in the asymptotic limit of large with the same measure [8]. In addition, it was demonstrated by Martinetz et al. [7] that the neural-gas network takes a smaller number of learning steps to converge than that of three other algorithms they tested, including the -means algorithm [3]. Furthermore, the neural-gas network has also been applied to time-series prediction which outperforms two other neural approaches. Other researchers have also demonstrated various applications of the neural-gas network [9], [10]. Despite all of these advantages, the neural-gas network suffers from high computational complexity for its sequential realization—a complexity of [7]. Although highly parallel implementation can improve its processing speed, sequential implementation is still more economical, practi- cal, and flexible, especially with the recent wide availability of powerful processors including digital signal processors (DSP’s). It is obvious that with a fast sequential realization, the neural-gas is more feasible in solving problems with larger (less error in VQ and prediction). In next section the neural- gas algorithm is briefly reviewed. Then in Section III we will 0090–6778/98$10.00  1998 IEEE