Extensions to Quantile Regression Forests for Very High Dimensional Data Nguyen Thanh Tung 1 , Joshua Zhexue Huang 2 , Imran Khan 1 , Mark Junjie Li 2 , and Graham Williams 1 1 Shenzhen Key Laboratory of High Performance Data Mining. Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China. 2 College of Computer Science and Software Engineering, Shenzhen University. tungnt@wru.vn, zx.huang@szu.edu.cn, imran.khan@siat.ac.cn, jj.li@szu.edu.cn, Graham.Williams@togaware.com Abstract. This paper describes new extensions to the state-of-the-art regression random forests Quantile Regression Forests (QRF) for appli- cations to high dimensional data with thousands of features. We propose a new subspace sampling method that randomly samples a subset of fea- tures from two separate feature sets, one containing important features and the other one containing less important features. The two feature sets partition the input data based on the importance measures of fea- tures. The partition is generated by using feature permutation to produce raw importance feature scores first and then applying p-value assessment to separate important features from the less important ones. The new subspace sampling method enables to generate trees from bagged sample data with smaller regression errors. For point regression, we choose the prediction value of Y from the range between two quantiles Q0.05 and Q0.95 instead of the conditional mean used in regression random forests. Our experiment results have shown that random forests with these ex- tensions outperformed regression random forests and quantile regression forests in reduction of root mean square residuals. Keywords: Regression Random Forests, Quantile Regression Forests, Data Mining, High Dimensional Data 1 Introduction Regression is a task of learning a function f (X)= E(Y |X) from a training data L = {(X,Y )=(X 1 ,Y 1 ), ..., (X N ,Y N )}, where N is the number of objects in L, X ∈ R M are predictor variables or features and Y ∈ R 1 is a response variable or feature. The regression model has the form Y = E(Y |X)+ ǫ (1) where error ǫ ∼ N (0,σ 2 ). A parametric method assumes that a formula for conditional mean E(Y |X) is known, for instance, linear equation Y = β 0 + β 1 X 1 ,...,β M X M . The linear