Criticality of predictors in multiple regression Razia Azen* Department of Educational Psychology, University of Wisconsin-Milwaukee, USA David V. Budescu Department of Psychology, University of Illinois at Urbana-Champaign, USA Benjamin Reiser Department of Statistics, University of Haifa, Israel A new method is proposed for comparing all predictors in a multiple regression model. This method generates a measure of predictor criticality, which is distinct from and has several advantages over traditional indices of predictor importance. Using the bootstrapping (resampling with replacement) procedure, a large number of samples are obtained from a given data set which contains one response variable and p predictors. For each sample, all 2 p 2 1 subset regression models are ®tted and the best subset model is selected. Thus, the (multinomial) distribution of the probability that each of the 2 p 2 1 subsets is `the best’ model for the data set is obtained. A predictor’s criticality is de®ned as a function of the probabilities associated with the models that include the predictor. That is, a predictor which is included in a large number of probable models is critical to the identi®cation of the best-®tting regression model and, therefore, to the prediction of the response variable. The procedure can be applied to ®xed and random regression models and can use any measure of goodness of ®t (e.g., adjusted R 2 , C p , AIC) for identifying the best model. Several criticality measures can be de®ned by using different combinations of the probabilities of the best-®tting models, and asymptotic con®dence intervals for each variable’s criticality can be derived. The procedure is illustrated with several examples. 1. Introduction Multiple regression (MR) models are used to predict a single criterion variable from several predictors. Consider the MR model Y 5 Xb 1 «, where Y is an n 3 1 data vector (the criterion), X is an n 3 ( p 1 1) full-rank data matrix (the predictors); « is an n 3 1 vector of unobservable `error’ terms and b is a ( p 1 1) 3 1 vector of parameters, estimated from the data by à b 5 (X 9 X) 21 X 9 Y. Here p represents the British Journal of Mathematical and Statistical Psychology (2001), 54, 201±225 Printed in Great Britain © 2001 The British Psychological Society 201 * Requests for reprints should be addressed to Dr Razia Azen, Department of Educational Psychology, University of Wisconsin-Milwaukee, PO Box 413, Milwaukee, WI 53201, USA.