An optimal weight learning machine for handwritten digit image recognition Zhihong Man a,n , Kevin Lee a , Dianhui Wang b , Zhenwei Cao a , Suiyang Khoo c a Faculty of Engineering and Industrial Sciences, Swinburne University of Technology, Melbourne, VIC 3122, Australia b Department of Computer Science and Engineering, La Trobe University, Melbourne, VIC 3087, Australia c School of Engineering, Deakin University, Geelong, VIC 3122, Australia article info Article history: Received 29 February 2012 Received in revised form 31 May 2012 Accepted 16 July 2012 Available online 1 August 2012 Keywords: Neural networks Optimal weight learning Feature space Handwritten digit recognition Regularization abstract An optimal weight learning machine for a single-hidden layer feedforward network (SLFN) with the application to handwritten digit image recognition is developed in this paper. It is seen that both the input weights and the output weights of the SLFN are globally optimized with the batch learning type of least squares. All feature vectors of the classifier can then be placed at the prescribed positions in the feature space in the sense that the separability of all nonlinearly separable patterns can be maximized, and a high degree of recognition accuracy can be achieved with a small number of hidden nodes in the SLFN. An experiment for the recognition of the handwritten digit image from both the MNIST database and the USPS database is performed to show the excellent performance and effectiveness of the proposed methodology. & 2012 Elsevier B.V. All rights reserved. 1. Introduction Neural network-based pattern classification techni- ques have been widely used for handwritten digit image recognition over the last 20 years [1–5]. The merits of the neural classifiers for the image recognition are attributed to (i) their powerful learning abilities, through trainings, from a large amount of training data, (ii) their capability of accurately approximating the unknown functions with complex dynamics embedded in the training data, and (iii) their parallel structures to perform fast and efficient parallel computing during the training as well as in the process of image recognition. It has been noted that most neural pattern classifiers for the handwritten digits recognition are designed with multi- layered neural networks, trained with the recursive gradient- based back-propagation (BP) algorithms, to perform the pre- filtering, feature extraction and pattern recognition. Because the BP training process is time-consuming with a slow convergence [6–10], these types of neural classifiers are hard to be used in many practical applications where a fast on-line training is required. In addition, most existing neural classi- fiers require the large number of hidden nodes in their hidden layers in order to obtain the highly separable features in the feature space. Such a requirement, in fact, will greatly increase the physical size of the neural classifiers’ hardware as well as the training time in practice. In view of all the above issues, the researchers in the areas of pattern classification and computational intelli- gence have been exploring the new types of neural classifiers with a single hidden layer, a small number of hidden nodes, and the fast training algorithms to fulfill the industrial requirements such as small size hardware and easy implementation in industrial environments. One of the state-of-the-art neural classifiers is a single hidden layered feedforward neural network based classifier developed in [11–13] where the input weights and the hidden layer biases are randomized and the output weights are computed by using a generalized inverse of Contents lists available at SciVerse ScienceDirect journal homepage: www.elsevier.com/locate/sigpro Signal Processing 0165-1684/$ - see front matter & 2012 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.sigpro.2012.07.016 n Corresponding author. Tel.: þ61 3 9214 5175. E-mail addresses: zman@swin.edu.au (Z. Man), kklee@swin.edu.au (K. Lee), dh.wang@Latrobe.edu.au (D. Wang), zcao@swin.edu.au (Z. Cao), khooyang@yahoo.com (S. Khoo). Signal Processing 93 (2013) 1624–1638