Journal of Mathematics and Statistics 10 (3): 414-420, 2014
ISSN: 1549-3644
© 2014 S. Saleh, This open access article is distributed under a Creative Commons Attribution
(CC-BY) 3.0 license
doi:10.3844/jmssp.2014.414.420 Published Online 10 (3) 2014 (http://www.thescipub.com/jmss.toc)
414
Science Publications JMSS
MODEL SELECTION VIA ROBUST VERSION OF R-SQUARED
Shokrya Saleh
Institute of Mathematical Sciences, University of Malaya, 50603 Kuala Lumpur, Malaysia
Received 2014-06-05; Revised 2014-07-08; Accepted 2014-09-17
ABSTRACT
R-squared (R
2
) is a popular method for variable selection in linear regression models. R
2
based on Least
Squares (LS) regression minimizes the sum of the squared residuals; LS is sensitive to outlier observation.
Alternative criterion based on M-estimators, which is less sensitive to outlying observation has been
proposed. In this study explicit expression for such criterion is obtained when the Least Trimmed Squares
(LTS) estimator is used. The influence function of R
2
is also discussed. In our simulation study, the
performance of proposed criterion is compared to the existing criteria based on M-estimators (R
2
M
) and to
the classical non-robust based on least squares estimators (R
2
LS
). We observe that the proposed (R
2
LTS
)
selects more appropriate models in the case of bad leverage points (outliers in the X-direction) are present.
Keywords: Robust R
2
-Coefficents, Least Trimmed Squares, Influence Function
1. INTRODUCTION
The use of squared multiple correlation coefficient,
R
2
, in choosing model is a common goal in
econometrics analysis; it is a classical model selection
criterion which has been widely used for centuries and
it is still popular till today. Hahn (1973), Kvålseth
(1985), Willett and Singer (1988) as well as
Anderson-Sprecher (1994) have expounded on R
2
.
Consider a multiple linear regression model:
T
i i
y X α β ε = + + (1)
where, X = (x
i1
,...x
ip
)
T
is a vector containing p
explanatory variables, i = 1,...,n,y
i
is the response
variable, β is a vector of p parameters, a is the intercept
parameter and ε
i
is an independently and identically
distributed (iid) random error with mean 0 and variance
σ
2
. The distribution of errors satisfying Fσ(x) = F
0
(x/σ),
where σ is the residual scale parameter and F
0
is
symmetric with a strictly positive density function. With
( 29
2
1
n
i
i
SSE r
=
=
∑
, where
T
i i LS LS
r y X β α = - +
, the residual
from the Least Squares (LS) fit, the classical R
2
coefficient is given by:
2
1
LS
SSE
R
SST
= - (2)
where,
( 29
2
1
i
n
i
i
y y SST
=
- =
∑
with
i
y is the sample mean
of the dependent variable. Selecting models on the basis
of maximizing
2
LS
R is equivalent to minimizing the
residual mean square.
Note that the numerator in Equation 2 approximates
the scale of the residuals in the full model, while the
denominator is the scale of the residuals in the following
reduced model:
0 i i
y α ε = + (3)
In fact the LS estimator of α
0
in Equation 3 equals
i
y . Then analogous to Equation 2 is:
2
var( )
1
var( )
LS
full
R
reduced
= - (4)
where, var is defined according to principles guide
estimation: