614 zyxwvutsrqponml IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, zyxwv VOL. ASSP-35, NO. 5, MAY 1987 On “Soft” Bit Allocation Abstract-If a random variable zyxwvutsr X with variance rs ’ is quantized op- timally, being mapped into zyxwvutsr s discrete levels, the quantization error is roughly proportional to zyxwvutsrq 6’/s2. In many applications in speech coding and image digitization, we are given sets of random variables and we have to quantize them in some way so as to represent their realizations with as few discretized levels (bits) as possible. This has to be done while also minimizing the total quantization error. The quantization level or bit allocation process should therefore be the result of a com- promise between the total discretization error and the number of bits used to represent the realizations of random vectors. This paper pre- sents a solution of this problem, extending the classical bit allocation methods in which zyxwvutsrqp a fixed, prescribed number of bits have to be allo- cated to minimize the quantization error. The general soft bit alloca- tion process is useful in designing variable rate (adaptive) coders as opposed to the classical bit allocation procedures that were devised for information transmission over communication channels requiring con- stant rate encoding. I.INTRODUCTION S UPPOSE we are given zyxwvutsr M independent random vari- ables, X , , X,, * , X, with variances of 0: zyxwvut 2 ui 2 u3 2 * * - 2 ui, and we separately quantize them using si quantization levels for Xi. Then the total quantization error, with optimal quantization,is approximately pro- portional to 2 M and the total number of bits needed to code the IIE , si possible discretized levels for the vector (Xl, zyxwvuts X,, - , X,) is M M We note that the quantization error €or uni€ormly dis- tributed random variables is exactly proportional to the expression given by (1). The expression a2/s2 is also a very good approximation of the error in quantizing a ran- dom variable of variance u2 to s levels, and therefore, (1) is a good approximation for the total error incurred by independent quantization of M random variables. The classical bit allocation or integer quantization level allocation problem is to choose the natural numbers si so as to meet the requirement that BT = B (a certain prede- termined integer), while minimizing EQ. The motivation for such a bit allocation process may be the requirement Manuscript received April 18, 1986; revised July 26, 1986. The author is with the Department of Electrical Engineering, Tech- IEEE Log Number 8613495. nion-Israel Institute of Technology, Haifa, 32000, Israel. A. M. BRUCKSTEIN 0096-35 18/87/0500-0614$01 .OO zyxwvu 0 1987 IEEE to transmit information on successive realizations of the vector (X,, X,, - , X,) over aJixed capacity commu- nication channel.We note that the optimal E, and the corresponding bit-count B, do not correspond to a point on the rate-distortion curve for aninformation source pro- ducing the vector of random variables. The assumption of independent quantization of the elements in the source vector necessarily makes this coding scheme a suboptimal one. It is, however,a very popular block quantization process, widely used in speech coding and imagepro- cessing applications. The bit allocation process, as described above, does not allow any flexibility in either reducing the bit rate when the quantization error is smaller than required, or in in- creasing it when it becomes temporarily important to im- prove the precision of describing X,, X,, - - - , X,. With present technology it is not a problem to dynamically al- locate channel capacity in order to accommodate a non- stationary information source, and improved performance may certain!y be expected to result from adaptation of the quantization process to slowly varying statistical proper- ties of the sources or to changing demands on precision. In this paper we propose a “soft” bit allocation pro- cedure. It will be assumed that two functions, CQ( .) and z C,( e), that measure the cost incurred by quantization er- ror and by bit usage, respectively, are given, and we shall determine an integer quantization level allocation {si that minimizes their sum for a source vector characterized by the given set of variances { u:, a;, - - , a;}. The total cost to be minimized, over all feasible allo- cations {si } , is therefore the function / M where cQ and GD are monotone increasing and smooth (differentiable) functions. The minimization of this func- tion has to be carried out under the constraint that si are natural numbers greater than one. 11. THE OPTIMAL LEVEL ALLOCATION PROCESS The optimal integer quantization level allocation prob- lem defined in the previous section is an integer program- ming problem. We shall first assume that the si may take any real value in the interval [ 1, 03 ), and solve the prob- lem of minimizing the cost function (3), under this con- straint alone. Then the integers closest to the optimal so- lution may be chosen as the integer level allocation, or a search process around the optimal solution may be done.