+Ds+D9, D+D’+D’, D+D2+D3+D6+D7+D8+D9] Every permutation of the columns of this matrix yields a matrix thus y = 9 and the Corollary to Theorem 3 gives only the weaker whose row space contains [l,O, l,O] or [O,l,O, 11. bound n(p+2)-2=42. In the example above, the Corollary to Theorem 3 was a little ACKNOWLEDGMENT disappointing in that it gave a bound of 42 whereas more careful The authors wish to thank M. K. Simon and J. G. Smith for examination yielded L f 16 (even 16 may be too high, for a cursory examination of the bit pattern associated with the basic bringing this problem to their attention and for suggesting generator for C given above indicates that 13 may be the several possible approaches. answer). When k = n - 1 it is clear from Theorem 3 that encoders REFERENCES do exist for which the bound given by the Corollary is tight. In [I] G. D. Fomey, Jr., “Convolutional codes I: Algebraic structure,” IEEE general there are minimal encoders whose codes have no infinite Tram. Inform. Theory, vol. IT-16, pp. 720-738, Nov. 1970. (See also alternating run but do possess codewords with finite alternating correction: samejournal, May 1971, page 360). runs of length np + k + 1 which compares reasonably well with the bounds given by the Corollary. For example, consider the (n, k) convolutional encoder G= [ I 1 0 1 0. *.0/p q p q ... where I is an identity matrix or order k - 1 and 0’ is a k - 1 by n-k + 1 matrix of zeros. A Note on Optimal Quantization JAMES A. BUCKLEW AND NEAL C. GALLAGHER, JR., MEMBER, IEEE Herep=p(D)=l+D+D’and,forneven,q=q(D)=1+D2 Abshzcz-For a genehd class of optimal quantinss the variance of the +D” (~>3) while for n odd q(D)=l+D3+Dp (~24). G is outputislessthanthatoftheinput.AlsothemeanvalueIsprrservedby obviously basic and minimal. Further Theorem 1 guarantees that the quantizfng operation. no codeword generated by G contains an infinite run of altemat- ing symbols. That G generates a codeword with a run of alter- I. INTRODUCTION nating symbols of length n + k + 1 can be confirmed by select- ing the inputs xc’), . . . ,x(6 J. Max [l] is generally credited with being the first to consider properly. For example, let n=8, the problem of designing a quantizer to minimize a distortion k = 4, and p = 3, then the bit pattern associated with the bottom measure given that the input statistics are known. Max derives row of G is necessary conditions for minimizing the mean square quantiza- 00011111 00010101 00001010 00011111. tion error. These results are summarized in the following equa- tions: So if x(~)=I+D~+D~ (=lOllO~~~) and x(~)=D+D~+D~+ D4 with x(l) = xc3)= 0 the codeword generated by G is yj’ x, s xf(x) dX/p(Xj-l <X Gxj) (1) x,-l 00011111 01010101 01010101 01010101 010111~~~ Yj +Yj+ I which, starting with its 8th symbol, has an alternating run of - = 3 2 length 29 = 8 -3 + 5. Obviously XC’), * . * , xck-‘) can always be adjusted to fill in the first k - 1 symbols of each block of n where f(x) is the probability density of the variable to be symbols in the proper fashion. So the input xck) is the critical quantized and P(xi-, <x <xi) is the probability that x lies in the one. For n even, k even, and p odd, xck)= 1 + D2 + D4 interval (xi-,,xj]. The y, are output levels and the xi are the + 1. . + DP-’ + D”. Similar formulas exist for the other cases- break points where an input value between xj-, and xi is when n is odd these vary with p modulo 4. quantized to yj. Fleisher [2] later gave a sufficient condition for As final examples consider the NASA Planetary Standard Max’s equations to be the optimal set. encoders of rates l/2 and l/3. Here G=[gi,g$] or [gl,g2,g3] Typically, the above equations are intractable except for sim- with g,=l+D2+D3+D5+D6, g2=l+D+D +D3+D6, g3 ple input densities, causing some researchers to derive approxi- = 1 + D + D2 + D 4+ D 6. These both are basic minimal encoders mate formulae for some common densities. Roe [3] derives an which do not possessinfinite alternating runs in any codeword approximation for the input interval endpoints assuming that the widths of these intervals are small, i.e., the number of output as Theorem 1 easily shows. (Note that [ g,,g3,g2] and [ g2,g3,gl] do possesssuch runs, thus if infinite alternating runs are to be levels is large. Wood [4] derives a result which states, in effect, avoided the outputs in [ g,,g2,g3] must be interleaved properly). that the variance of the output of a minimum mean-square error For the rate l/2 code the Corollary of Theorem 3 yields L < 2.8 quantizer should be less than the input variance. He also states -2= 14, and Theorem 3 itself guarantees the existence of finite that the significance of his result is that the signal and noise are codewords with alternating runs in this case. The rate l/3 code dependent and that no pseudo-independence of the sort consid- has a dual generator F given by ered by Widrow [4] is possible. However, Wood’s derivation assumes the input density to be F= D [ l+D2+D3 l+D+D2+D3 , h (D )=l+D I five times differentiable and that the quantizer input intervals be 1+D3 D3 l+D+D’ , h(D)=O. very small in order to truncate various Taylor series expansions. Furthermore, the derived expression for the output variance is Apply Theorem 3 to the first row of F. Here s = 11 so L SLS + n - dependent upon the input interval lengths and the input proba- 2= 12. A finite codeword with an alternating run of length 12 is bility density function evaluated at the midpoints of these inter- generated from G by the input xc’) = 1 + D + D2 + D4 + D 7 vals. (= * . * 0111010010~ . . ); so this bound is achieved. In this note we derive a generalization of Wood’s results that Note: It is easy to see that, for k= 1 (n >2), it is always eliminates a number of his approximations and generalizes the possible to rearrange the columns of a basic generator matrix to results to apply to more than just Max quantizers. avoid infinite alternating runs. However, this is not true in general. Consider a basic (4,2) convolutional code whose genera- Manuscript received May 5, 1978; revised September 5, 1978. This work tor matrix modulo 1 + D is was supported in part by the National Science Foundation under Grant ENG-7682426 and in Dart the Air Force Office of Scientific Research. Air 1 0 1 0 1 Force Systems Comma;ld, USAF under Grant AFOSR-78-3605. 0 0 11’ The authors are with the School of Engineering, Purdue University, West Lafayette, IN 47907. IEEETRANSACTIONS ON INFORMATION THEORY,VOL. IT-25, NO. 3, MAY 1979 365 0018-9448/79/0500-0365$00.75 01979 IEEE