The Cost of Complexity in Identification of
FIR Systems
⋆
Cristian R. Rojas
∗
, M¨ arta Barenthin
∗∗
, James S. Welsh
∗
,
H˚ akan Hjalmarsson
∗∗
∗
School of Electrical Engineering and Computer Science, The
University of Newcastle, NSW 2308, Australia (e-mail:
cristian.rojas@studentmail.newcastle.edu.au).
∗∗
School of Electrical Engineering, KTH, 100 44 Stockholm, Sweden
(e-mail: marta.barenthin@ee.kth.se)
Abstract: In this paper we investigate the minimum amount of input power required to estimate
a given linear system with a prescribed degree of accuracy, as a function of the model complexity.
This quantity is defined to be the ‘cost of complexity’. The degree of accuracy considered is
the maximum variance of the discrete-time transfer function estimator over a frequency range
[-ω
B
,ω
B
]. It is commonly believed that the cost increases as the model complexity increases.
The objective of this paper is to quantify this dependence. In particular, we establish several
properties of the cost of complexity. We find, for example, a lower bound for the cost asymptotic
in the model order. For simplicity, we consider only systems described by FIR models and assume
that there is no undermodelling.
1. INTRODUCTION
The purpose of system identification is to construct math-
ematical models of dynamical systems from experimental
input/output data. To this end, a judicious choice of
the input signal is crucial. This has motivated substan-
tial interest in the topic of optimal experiment design.
Indeed, many results have appeared on this topic, both
in the statistics literature [Cox, 1958, Kempthorne, 1952,
Fedorov, 1972] and in the engineering literature [Mehra,
1974, Goodwin and Payne, 1977, Zarrop, 1979, Jansson,
2004].
A key point as to why system identification can work in
practice lies in the nature of the input signal: it is noted
that experiment design can emphasize system properties
of interest, while properties of little or no interest can be
‘hidden’ [Hjalmarsson, 2005, Hjalmarsson et al., 2006]. As
remarked in [Hjalmarsson et al., 2006], some properties
can be more easily estimated than others, in the sense that
the amount of input power needed to estimate them with a
given level of accuracy does not depend on the complexity
of the model considered. However, some properties do
depend on the model order. For example, it has been
shown that the cost of estimating the transfer function at
a particular frequency, or one non-minimum phase zero, is
independent of the model order [Hjalmarsson et al., 2006].
This paper can be considered as an extension of the study
of this phenomenon. Here we investigate the minimum
amount of input power needed to estimate a given linear
system with a prescribed degree of accuracy, as a function
of the model complexity. This quantity is defined to be the
‘cost of complexity’. The degree of accuracy considered
is the maximum variance of the discrete-time transfer
function estimator over a frequency range [-ω
B
,ω
B
].
For simplicity, we restrict the model class to systems
described by FIR models. Also, we assume that there is
no undermodelling, i.e. that the true system belongs to
the model structure.
⋆
This work was supported by the Swedish Research Council.
The contribution of this paper consists of establishing sev-
eral properties for the dependence of the cost on the model
complexity. We believe that these results can provide
a better understanding of the relationship between the
amount of information that we ask to be extracted from a
system, and the sensitivity of the cost of the identification
with respect to the model complexity. This appears to be
a key for understanding why system identification works
for complex systems.
In order to study the problem posed in this paper, we
employ a semidefinite optimization approach [Hildebrand
and Gevers, 2003, Jansson and Hjalmarsson, 2005, Bom-
bois et al., 2006]. In particular, the input design problem is
formulated in terms of Linear Matrix Inequalities (LMIs)
and the problem reduces to studying the positivity of a
specific Toeplitz matrix.
This paper is organised as follows. The problem is for-
mulated in Section 2. The main results are presented in
Section 3 and a numerical example is provided in Section 4.
Section 5 concludes the paper.
2. PROBLEM SET-UP
Consider the FIR system with input u(t) and output y(t),
y(t)=[θ
o
no
]
T
Λ
no
(q)u(t)+ e
o
(t)= G(q,θ
o
no
)u(t)+ e
o
(t),
where Λ
no
(q) := [ 1 q
-1
··· q
-no
]
T
with q
-1
denoting the
backward time shift operator and θ
o
no
=[ b
o
0
··· b
o
no
]
T
.
Furthermore, e
o
(t) is zero mean white noise with variance
σ
2
o
, and the input signal is considered to be wide-sense
stationary. The model to be fitted to this system is given
by
y(t)=
n
k=0
b
k
u(t - k)+ e(t)=[θ
n
]
T
Λ
n
(q)u(t)+ e(t).
where n ≥ n
o
. Consider the following autocovariance
representation for the power spectrum of u(t):
Proceedings of the 17th World Congress
The International Federation of Automatic Control
Seoul, Korea, July 6-11, 2008
978-1-1234-7890-2/08/$20.00 © 2008 IFAC 11451 10.3182/20080706-5-KR-1001.3107