Identification of Equation Error Models from Small Samples using Compressed Sensing Techniques Satheesh K. Perepu ∗ Arun K. Tangirala ∗ ∗ Department of Chemical Engineering, IIT Madras, Chennai, Tamilnadu 600036, India (e-mail: satheesh841@gmail.com & arunkt@iitm.ac.in). Abstract: System identification (SI), especially from small samples, is a challenging problem and of interest in several applications. Standard prediction-error minimization methods (PEM), under these conditions, generally result in estimates with higher variance. Moreover, in the identification of parametric models, one often needs prior knowledge of the input-output delay, obtaining estimates of which, is not possible using classical methods when the delay is either comparable or greater than the sample size. In this work, we develop a compressed sensing (CS)-based method for identifying sparse equation-error models that includes both auto-regressive eXogenous (ARX) and AR moving average eXogenous (ARMAX) structures with large delays, small orders and small delays with large orders, but with missing coefficients. The outcome is an iterative basis pursuit de noising (IBPDN) algorithm for solving non-linear CS problems. In addition, we propose a semi-rigorous method to lower the mutual coherence of the regressor matrix so as to obtain lower variance parameter estimates with the CS techniques. Errors in parameter estimates are computed using the bootstrapping method. Simulation studies on three diverse examples are presented to demonstrate the efficacy of the proposed methodology. Keywords: system identification; ARX models; ARMAX models; non-linear compressed sensing; mutual coherence. 1. INTRODUCTION Parametric system identification is concerned with developing models of a specific structure from input-output data. Most of the existing techniques including the widely used prediction- error minimization (PEM) methods (of which least squares (LS) and maximum likelihood (ML) are special cases) the- oretically yield efficient and consistent estimates only under asymptotic (large sample) conditions (Ljung, 1999). Further- more, identification of parametric models require prior spec- ification of input-output delay and model polynomial orders. Time-delay estimation is typically carried out using impulse and frequency response methods (Bjorklund and Ljung, 2003; Selvanathan and Tangirala, 2010). Alternatively, delay may be also treated as an additional parameter and estimated simultane- ously with model parameters. Regardless, both delay estimation techniques are known to work efficiently only in the presence of large samples, i.e., when the delay d ≪ M , where M is the sample size. Order determination is usually carried out mostly using information-theoretic criteria such as Akaike information and Bayesian information criteria, respectively, with prelimi- nary guesses generated using ideas from subspace identification (SSID). Once again the information-theoretic measures, which use the ML as their basic engine, and the SSID technieus are devised for large sample situations. However, in several appli- cations, only data sets of limited or small size are available. On- line estimation, set-point oriented process are a few examples of these situations. Isaksson (1991) used maximum likelihood estimation (MLE) to estimate parameters of ARX model using small number of samples. The complexity of these algorithms increase inversely with number of samples available (Bohlin, 1971). Yang et al. (2012) developed a method to identify a bioethanol plant using small number of samples. This technique is based on an orthogonal least squares algorithm and a new resampling method called output jittering. The complexity of this algorithm also increases inversely with number of sam- ples available. Vanli and Castillo (2007) used pseudo-linear regression to estimate closed loop Box-Jenkins (BJ) models as ARMAX models from small samples. This method requires knowledge of delay and order of the process prior to estima- tion. To the best knowledge of authors there are no effective techniques for the estimation of parametric models, especially those that can also automatically estimate delay and order, from small samples. This work is concerned with a sub-class of such models, namely, the equation-error or the ARMAX models. A regular ARMAX model with known model order and delay is described by the equation, A(q −1 )y[k]= B(q −1 )u[k]+ c(q −1 )e[k] (1a) A(q −1 )=1+ a 1 q −1 + a 2 q −2 + ··· + anaq na (1b) B(q −1 )= b d q −d + ··· + bn b ′ q −d−n b (1c) C(q −1 )=1+ c 1 q −1 + c 2 q −2 + .... + cnc q nc (1d) where u[k] and y[k] are the input and measured output at sampling instant k respectively, e[k] ∼ N (0,σ 2 e ), d is the input-output delay and n b ′ = n b + d. An ARX model is a special case of ARMAX model with all the past terms of the innovations of e[k] i.e. c 1 = c 2 = ··· = c nc = 0 set to zero. ARX models give rise to linear predictors, thereby permitting the use of linear LS methods to generate unique parameter estimates. ARMAX models, on the other hand, as evident from (1a), result in non-linear predictors. A non-linear LS estimator has to be thus employed, wherein the optimum is searched numerically and one has to be usually content with a local optimum. As remarked earlier, these estimators, linear or non-linear, require the user to specify a prior the time-delay and order. More importantly, the non-linear estimator is efficient only under asymptotic conditions. Preprints of the 9th International Symposium on Advanced Control of Chemical Processes The International Federation of Automatic Control June 7-10, 2015, Whistler, British Columbia, Canada TuPoster2.3 Copyright © 2015 IFAC 796