IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 44, NO. 5, SEPTEMBER 1998 1981 The Effect of Quantization on the Performance of Sampling Designs Karim Benhenni and Stamatis Cambanis Abstract— The most common form of quantization is rounding-off, which occurs in all digital systems. A general quantizer approximates an observed value by the nearest among a finite number of representative values. In estimating weighted integrals of time series with no quadratic mean derivatives, by means of samples at discrete times, it is known that the rate of convergence of the mean-square error is reduced from to when the samples are quantized. For smoother time series, with quadratic mean derivatives, it is now shown that the rate of convergence is reduced from to when the samples are quantized, which is a very significant reduction. The interplay between sampling and quantization is also studied, leading to (asymptotically) optimal allocation between the number of samples and the number of levels of quantization. Index Terms—Covariance function, quantization, random integral, rate of convergence, sampling design, trapezoidal estimator. I. INTRODUCTION AND SUMMARY In several areas of communicatons, information theory, and signal processing, it is of interest to estimate a weighted random integral from observations of the process at a finite number of sampling points rather than over an observation interval. This problem was widely studied by Sacks and Ylvisaker [10]–[12], Cambanis [5], Su and Cambanis [14] for processes having or quadratic mean derivatives; and by Benhenni and Cambanis [2], [3] for processes having quadratic mean derivatives, where the rate of convergence for the mean-square error (MSE) is of order More recently, Istas and Loredo [7] as well as Stein [13] extended the results to processes satisfying a H¨ older condition of the type (1). We consider in this correspondence the case of estimating random integrals from quantized observations. The most com- mon form of quantization is rounding-off which occurs in all digital systems. We show that for processes whose covariance function can be expanded around up to order , as in (1), the rate of convergence for the mean-square error in the approximation of the random integral by trapezoidal estimators constructed from quantized observations, is of order In particular, it is known from Bucklew and Cambanis [4] that for processes having quadratic mean derivatives, i.e., , then the rate of convergence is reduced from to when quantized data are used. For any , the appropriate estimators called adjusted trape- zoidal estimators in Benhenni and Cambanis [2] with quantized observations are used, and it is shown that the corresponding rate of convergence for the mean-square error is of order Thus there is a significant reduction in the rate of convergence from Manuscript received October 7, 1996; revised January 15, 1998. This work was supported by the Air Force Office of Scientific Research under Contract F49620 92J 0154 , by the Army Research Office under Grant DAAL03 92G 0008, and by the National Science Foundation. The material in this correspondence was presented at the IMS Annual Meeting, Chapel Hill, NC, June 1994. K. Benhenni is with LABSAD, BSHM, Universit´ e Pierre Mend` es France, BP 47 X, 38040 Grenoble, France. S. Cambanis is with the Department of Statistics, University of North Carolina, Chapel Hill, NC 27599-3260 USA. Publisher Item Identifier S 0018-9448(98)05126-8. to when the samples are quantized for processes having quadratic mean derivatives, i.e., We consider regular sequences of sampling designs where the sampling points represent the percentiles of a continuous positive density on Let a stationary and Gaussian process with mean and variance , and define the quantized observation process where is an -level quantizer with when where and , cf. Lloyd [8] and Max [9]. Let and that the covariance function of the Gaussian process can be expanded around as follows: (1) where and corresponds to , respectively, or where Then the covariance of the process can be expanded around as follows (see Appendix III): (2) where Moreover, we assume that the covariance function has two continuous derivatives away from . The trapezoidal predictor used to approximate the random integral is (3) The precision of the approximation is measured by the mean-square error It is known that in quadratic mean, see Benhenni and Cambanis [2], and thus Putting 0018–9448/98$10.00  1998 IEEE