Proceedings of the IConSSE FSM SWCU (2015), pp. MA.8–12 ISBN: 978-602-1047-21-7 SWUP MA.8 Parameter estimation of kernel logistic regression Riska Yanu Fa’rifah * , Suhartono, Santi Puteri Rahayu Department of Statistics, Institut Teknologi Sepuluh Nopember, Sukolilo, Surabaya 60111, Indonesia Abstract Logistic regression (LR) is a nonlinear classification method, often used for binary data sets. Over fitting the training data sets may arise in LR, especially when the data sets are used have high-dimensional. One of approaches to reduce over fitting is through regularized LR method. Regularized LR can be defined as log-likelihood function of LR adding with regularization parameter. There are regularized optimization problem, because the Loss function (deviance) in regularized LR nonlinear. To minimize this problem, need a linear combination method of regularized LR, known as kernel logistic regression (KLR). KLR is a nonlinear classifier. KLR provide higher classification accuracy of small to medium sample size data sets when compared with LR. With truncated newton method, estimation KLR using maximum likelihood estimation (MLE) can be optimum. Keywords kernel logistic regression, logistic regression, MLE, regularized logistic regression, truncated Newton 1. Introduction Regression is one of statistical method that described the causal relationship between response and predictors (Draper & Smith, 1998). If the response is categorical data (nonmetric), then the analysis which can be used is a classification method, such as logistic regression (LR). Over fitting the training data sets may arise, especially if the data sets have a high-dimensional (Hosmer & Lemeshow, 2000). One of approaches to reduce over fitting is a quadratic regularization, known as regularized LR (Maalouf, 2009). Regularized LR can be formed from adding a regularization parameter on log-likelihood function of LR. If the analysis using the small to medium sample size, loss function (deviance) produced not minimum, because deviance is nonlinear in its parameters. This situation due to parameter estimation of KLR using MLE is not optimum. It can be solved by making a linear combination of regularized LR, known as kernel logistic regression (Maalouf, 2009). KLR is a nonlinear classifier method. KLR is combination of regularized LR and kernel. Parameter estimation of KLR using MLE not close form, so to optimize the parameter estimation use numerical method. This method named newton raphson (Minka, 2003). However, newton raphson does not provides an optimum estimation. It caused by high- dimensional of hessian matrix. Thus, MaaLouf et al. (2010) adding conjugate gradient algorithm on truncated newton method. This method was first used (Komarek & Moore, 2005) to get the parameter estimation method regularized LR. * Corresponding author. Tel.: +62 812 4944 4853; E-mail address: riska.yanu@gmail.com