Volume 6 • Issue 5 • 1000264 J Biom Biostat ISSN: 2155-6180 JBMBS, an open access journal Research Article Open Access Ghadban and Iguernane, J Biom Biostat 2015, 6:5 DOI: 10.4172/2155-6180.1000264 Research Article Open Access Keywords: Linear regression model; Multicollinearity; Ridge regression estimators; Simulation study Mathematics Subject Classifcation: Primary 62J07; Secondary 62J05 Introduction Consider the standard multiple linear regression model; Y X e = + , (1) where Y is an ( 1) n × vector of responses, X is an ( ) n p × matrix of the explanatory variables of full rank p, β is a ( 1) p × vector of unknown regression coefcients, and fnally, 2 ~ (0, ) e N I σ is an ( 1) n × vector of error terms. Te OLS estimator is ofen used to estimate the regression coefcients β as: 1 ˆ ( ) XX XY β − ′ ′ = . (2) Te standard assumption in the linear regression analysis is that all the explanatory variables are linearly independent. When this assumption is violated, the problem of multicollinearity enters into the data and it infates the variance of an ordinary least squares estimator of the regression coefcient. Obtaining the estimators for multicollinear data is an important problem in the literature. In fact, when the problem of multicollinearity is present in the measurement error ridden data, then an important issue is how to obtain the consistent estimators of regression coefcients. One of the most popular estimator for combating multicollinearity is the ridge estimator, originally proposed by Hoerl et al. [1]. Tey suggested a small positive number (k>0) to be added to the diagonal elements of the XX ′ matrix from the multiple regression and the resulting estimators are obtained as: 1 ˆ () ( ) , k XX kI XY β − ′ ′ = + (3) which is known as a ridge regression estimator. For a positive value of k, this estimator provides a smaller MSE compared to the OLS estimator, i.e., ˆ ˆ ( ( )) ( ) MSE k MSE β β < . Most of the later eforts in this area have concentrated on estimating the value of the ridge parameter k. Many diferent techniques for estimating k have been proposed by diferent researchers, for example, Hoerl et al. [1], Hoerl et al. [2] Dempster et al. [3], Gibbons [4], Kibria [5], Khalaf et al. [6], Alkhamisi et al. [7], Khalaf [8] and Khalaf [9]. Te plan of the paper is as follows: in Section 2, we present diferent methods for estimating the parameter of ridge regression together with our proposed estimators. A simulation study has been conducted in Section 3. Te simulation results are discussed in Section 4. In Section 5 we give a brief summary and conclusions. Te Proposed Ridge Regression Parameter In case of ordinary ridge regression, many researchers have suggested diferent ways of estimating the ridge parameter. Hoerl et al. [1] showed, by letting max β denote the maximum of the i β , that choosing; 2 2 max ˆ ˆ ˆ HK k σ β = , (4) implies that ˆ ˆ ( ( )) ( ) MSE k MSE β β < . Te ridge estimator using ˆ HK k will be denoted by HK. Hoerl et al. [2] suggested that, the value of k is chosen small enough, for which the MSE of ridge estimator is less than the MSE of OLS estimator. Tey showed, through simulation, that the use of the ridge with biasing parameter given by: 2 ˆ ˆ , ˆ ˆ HKB p k σ ββ = ′ (5) has a probability greater than 0.50 of producing estimator with a smaller MSE than the OLS estimator, where 2 ˆ σ is the usual estimator of 2 σ , defned by 2 ˆ ˆ ( )( ) ˆ 1 Y X Y X n p β β σ ′ − − = − − . Te ridge estimator using Eq. (5) will be denoted by HKB. Te purpose of this study is to modify the approaches of estimating k mentioned in Hoerl and Kennard [1] and Hoerl et al. [2] given in *Corresponding author: Ghadban AK, Department of Mathematics, Faculty of Science, King Khalid University, Saudi Arabia, Tel: 0172418000; E-mail: albadran50@yahoo.com Received November 23, 2015; Accepted December 02, 2015; Published December 09, 2015 Citation: Ghadban AK, Iguernane M (2015) The Traditional Ordinary Least Squares Estimator under Collinearity. J Biom Biostat 6: 264. doi:10.4172/2155- 6180.1000264 Copyright: © 2015 Ghadban AK, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. The Traditional Ordinary Least Squares Estimator under Collinearity Ghadban AK* and Iguernane M King Khalid University, Saudi Arabia Abstract In a multiple regression analysis, it is usually diffcult to interpret the estimator of the individual coeffcients if the explanatory variables are highly inter-correlated. Such a problem is often referred to as the multicollinearity problem. There exist several ways to solve this problem. One such way is ridge regression. Two approaches of estimating the shrinkage ridge parameter k are proposed. Comparison is made with other ridge-type estimators. To investigate the performance of our proposed methods with the traditional ordinary least squares (OLS) and the other approaches for estimating the parameters of the ridge regression model, we calculate the mean squares error (MSE) using the simulation techniques. Results of the simulation study shows that the suggested ridge regression outperforms both the OLS estimator and the other ridge-type estimators in all of the different situations evaluated in this paper. Journal of Biometrics & Biostatistics J o u r n a l o f Bi o m e t r i c s & B i o s t a t i s t i c s ISSN: 2155-6180