131 New Radial Basis Function Interior Point Method Learning Rule for Rough Set Reduct Detection George Meghabghab Roane State Department of Computer Science Technology Oak Ridge, TN, 3 7830 email: gmeghab*hotmail.com Abstract - Rough set based approach for knowledge discovery variables to 8 variables per rules. These variables can have a employs a greedy algorithm technique for reducing search space different number of sampling values. Results show that finding in order to extract a reduct of a given decision table. Given than rules of the form X =>Y that are reducts are computationally an exhaustive search over all possible attribute combinations will bounded to O((mom1/N)log(miN)), where mo is the number of input require time that is exponential in the number of attributes, it nodes(equal to the number of variables) and m1 is the number of may not be computationally feasible to find a reduct. We will hidden units(smaller than N), and N is the number of training turn reduct detection into a classification problem since in the examples. These results are better than the O(mO2N2) already elementary set approximation of an unknown concept for known in the literature. example, an elementary set is mapped to the positive region of an INTRODUCTION unknown concept if its degree of membership is greater than a user defined threshold. The latter idea leads us to consider a RBF neural networks emerged as a viable architecture to generalized RBF neural network which uses radial basis function implement neural network solution to many problems. RBF based on Cover's Theorem of 1965 which states that a "non . . . linearly separable pattern classification problem in a high- .neral inetwor sa determinist globalenon-lne dimension space is more likely to be linearly separable than in a minim zation methods. These methods detect sub-regions not low-dimensional space- the reason for making the dimension of the containing the global minimum and exclude them from further hidden layer in RBF network high". The generalized RBF neural consideration. In general, this approach is useful for problems network has a number of nodes at the hidden layer equal to M, requiring solution with guaranteed accuracy. These are where M is smaller than the number of training patterns N. At the computationally very expensive. The mathematical basis for output layer, the linear weights associated and the position of the RBF network is provided by Cover's Theorem ([1]) which centers of the radial basis functions and the norm weighting matrix states that a non linearly separable pattern classification associated with the hidden layer are all unknown parameters that have to be learned. We used RBF with an Interior Point pro blem in a hig -dimension a spacei iel toabe Method(IPM) to evaluate the promise of interior point method to learly separable than in a low-dmensional space- the reason radial basis functions. We trained the centers, spreads and the for making the dimension of the hidden layer in RBF network weights using interior point methods as follows. The learning error high. RBF uses a curve fitting scheme for learning, that is, function E can be approximated by a polynomial P(x) of degree 1, learning is equivalent to finding a surface in a multi- in a neighborhood of xi: dimensional space that represents a best fit for the training E_ P(x) _ E(x,) + gT(x_ xi) (1) data ([2]). The approach considered here is a generalized RBF N neural network ([2]) where the number of nodes at the hidden g =VxE=l VxEj layer is M, where M is smaller than the number of training j=1 patterns N. At the output layer, the linear weights associated So the linear programming problem for a given search direction has and the position of the centers of the radial basis functions and as a solution: m gT the norm weighting matrix associated with the hidden layer subjecttoi-ooz x-x_oio (2) are all unknown parameters that have to be learned. The where o is the vector of all ones with the same dimension as the state approach considered here is a generalized RBF neural network vector x and oc>O is a constant. In this research we rely on the IPM having a structure similar to that of Figure 1. A generalized method of Meghabghab and Nasr (1999) and apply it to search for RBF has the following characteristics: the direction that will minimize the error corresponding to the 1- Every radial basis function (pi at the hidden layer has a weights, the error corresponding to the centers of the neurons, the center position ti and a width or a spread around the error corresponding to the spreads of the neurons. center. It connects to the output layer through a weight The linear interior point method and its dual can be expressed as value wi. follows: T T 2- The number of radial basis functions at the hidden layer is minimize g x maximize b y of size M, where M is smaller than the number of input subject to x+z-b subject sx=c-y .0 (3) training patters N. x,z . 0 s, =-y .0 3- The linear weights wi which link an RBF node to the where x-x,+oho=u and b=2o7o. We apply this new RBF 1PM learning rule to the Heart disease output layer, the position of the centers of the radial basis training provided by the RSES of the Warsaw Institute of Poland. functions, and the width of the radial basis functions are The training is made out of N=8000 samples of m0=13 variables all unknown parameters that have to be learned. represented in 80 examplars corresponding to rules from 3 1 -4244-0363-4/06/$20.OO ©2006 IEEE