ACCURATE IDENTIFICATION OF BIASED MEASUREMENTS UNDER SERIAL CORRELATION R. KONGSJAHJU, D. K. ROLLINS and M. B. BASCUN Ä ANA Departments of Chemical Engineering and Statistics, Iowa State University, Iowa, USA C hemical process data are often correlated over time (i.e., auto or serially correlated) due to recycle loops, large material inventories, sampling lag, dead time, and process dynamics created by high-order systems and transportation lag. However, many approaches that attempt to identify gross errors in measured process variables have not addressed the issue of serial correlation which can lead to large inaccuracies in identifying biased measured variables. Hence, this work extends the unbiased estimation technique (UBET) of Rollins and Davis 1 to address serial correlation. The serially correlated gross error detection study of Kao et al. 2 is used as a basis for setting up this study and comparison. In their work, the type of autocorrelation was assumed known (ARMA(1,1)), and the measurement test (MT) was used for the identi®cation of the measurement bias. While Kao et al. 2 used prewhitening of the data and variances of measured variables derived from knowledge of the time correlation structure, this work presents two prewhitening methods and a different identi®cation strategy based on the UBET. Results of the simulation study show the UBET has higher perfect identi®cation rates and lower type I error rates over the MT. Keywords: serial correlation; autocorrelation; time series; gross error detection; data reconciliation INTRODUCTION As measurement and modelling technology continues to improve, along with advancements in computer technology, the amount of available process data will continue to grow. However, the bene®ts of these large amounts of data cannot be fully harnessed if data are inaccurate. For the past four decades, active research in the area of gross error detection or GED has aspired to detect, identify, and correct these measured process variables with signi®cantly large errors. Most GED methods proposed in the literature for the detection of large process measurement errors are derived from the assumption of serially independent process data that are iid (i.e., identically and independently distributed). This iid assumption is not always a realistic view of the condition of real processes. From a statistical point of view, serial correlation occurs when a process measurement variable is correlated over time. This kind of correlation can occur when there is model-mismatch in the dynamics that was not accounted for in either the process or measurement model. In most GED literature, serial correla- tion is differentiated from process dynamics and is used to refer strictly to correlation between measurement errors. In chemical processes, serial correlation may occur because of a number of physical factors such as process dead time, process dynamics, process control (e.g., feedback control), as well as factors related to measuring instruments. Kao et al. 2 proposed one of the ®rst GED methods to address serial correlation. They prewhitened residuals of measured variables and used the measurement test (MT) to identify gross measurement errors. The authors’ evaluation of this approach indicated a high rate of false identi®cation consistent with other MT methods involving non-serially correlated process data (Rollins, et al. 3 ). The concept of prewhitening process measurements for handling serial correlation is also used in this work to extend the capabilities of the unbiased estimation technique (UBET) developed by Rollins and Davis 1 . Over the years the UBET has been extended to address a variety of conditions including unknown variances and covariances of measurement errors (Rollins and Davis 4 ), bilinear con- straints (Rollins and Roelfs 5 , Kuiper et al. 6 ), dynamic processes (Rollins et al. 7 , Devanathan 8 ) and for automati- cally controlled processes (Rollins et al. 9 , Manuell et al. 10 ). However, this is the authors’ ®rst attempt to extend the UBET to serially correlated data. There are two major contributions in this article. First, two ways of prewhitening serially correlated data are presented: (1) directly on the measured variables and (2) on the nodal mass balances, both of which are different from Kao et al. 2 . The merits of both strategies are then investigated. The second major contribution is the mod- i®cation of the UBET test statistics and hypothesis tests to correctly address the use of prewhitened transformed data. This paper is organized into three sections. The ®rst section presents the process measurement model that includes the serial correlation structure used in this study and a note on the scope of this work. This section is followed by a reproduction of some Kao et al. 2 ’s results and a discussion of the limitations of the measurement test (MT) in dealing with serial correlation. The last section discusses the enhancement of the UBET to serially correlated process measurements. This section compares results of the UBET and the MT. 1010 0263±8762/00/$10.00+0.00 q Institution of Chemical Engineers Trans IChemE, Vol. 78, Part A, October 2000