Abstract—Estimation of model parameters is necessary to predict the behavior of a system. Model parameters are estimated using optimization criteria. Most algorithms use historical data to estimate model parameters. The known target values (actual) and the output produced by the model are compared. The differences between the two form the basis to estimate the parameters. In order to compare different models developed using the same data different criteria are used. The data obtained for short scale projects are used here. We consider software effort estimation problem using radial basis function network. The accuracy comparison is made using various existing criteria for one and two predictors. Then, we propose a new criterion based on linear least squares for evaluation and compared the results of one and two predictors. We have considered another data set and evaluated prediction accuracy using the new criterion. The new criterion is easy to comprehend compared to single statistic. Although software effort estimation is considered, this method is applicable for any modeling and prediction. Keywords—Software effort estimation, accuracy, Radial Basis Function, linear least squares. I. INTRODUCTION ODELING of a system is critical to understand and to predict its behavior. In software development due to intangible nature of software and there is no manufacturing, each software produced is unique. We only make copies of the software which is done in a short time. As the software engineering field is not yet matured like conventional engineering fields there is no established hand book. There is no standards certification for all the software. The problem becomes more complicated as the size measurement is also not universally standardized. In spite of all these problems managers and software engineers have to develop a plan using estimation techniques. Generally Lines of Code (LOC) or Function Point (FP) is used as basic size measure. Methods of varying complexity are proposed for software effort estimation. They are expert based [1], analogy based [2], analytical [3], and machine learning based [4]. Among the machine learning methods, neural networks play a major role in Software Development Effort Estimation (SDEE) [5]. One can design Radial Basis Function network (RBF) by changing only one parameter, function width (spread) which is also S.K.Pillai is with the Electrical and Electronics Engineering Department, Noorul Islam University, Noorul Islam Centre for Higher Education, Kumaracoil, Tamil Nadu, India (Mob: +919840783711; Fax: +914651257266; e-mail: skpillai50@gmail.com). M.K. Jeyakumar is with the Computer Applications Department, Noorul Islam University, Noorul Islam Centre for Higher Education, Kumaracoil, Tamil Nadu, India (Mob: +919443281133; Fax: +914651257266; e-mail: jeyakumarmk@yahoo.com). known as impact factor [6]. RBF is frequently used for Software Development Effort Estimation and it is shown that RBF performs better [7]–[9]. This motivates the authors to use RBF for estimating small projects. The estimate is essential at early stages of a project to plan manpower, schedule and cost. Underestimates may lead to poor quality and reducing the scope or even may lead to cancellation of the project. This can happen even to fit the project to budget due to management pressure. On the other hand overestimation can lead to underutilization of staff or an organization may lose the project in bidding itself. Both the cases are deterrent to an organization. One has to estimate effort as accurately as possible. Here lies the real problem, the definition of accuracy [10]. A new method of evaluation of accuracy based on linear least squares is proposed. A linear relationship between actual effort and predicted effort for test data is made. We have used mainly the data given in [11] for our studies. The paper is organized as follows: The next section reviews the related work followed by description of radial basis function neural network. Experimental evaluations using the new method are provided in the next section. Conclusions are given at the end followed by references. II. RELATED STUDIES SDEE or any prediction (forecasting) accuracy depends on the input data, algorithm used, and criteria used for accuracy computation. Generally historical data is divided into training (verification) set and testing (validation) set. Training data is used to build the model. This model is used for validation using test data. SDEE is a function of input where size of software projects plays an important role. For small projects effort required is also small. Lopez-Martin [11] used fuzzy logic model based on two independent variables New & Changed (N&C) code and Reused (R) code. He has compared the performance of fuzzy model with multiple regression model. The results indicate that there is no difference between these two models. Two fuzzy logic models Mamdani and Takai-Sugeno are studied in [12]. The evaluation of these methods with linear regression showed that Takai-Sugeno fuzzy system performs better. None of these works compares SDEE using one and two independent variables. We have used error characteristics to compare the performance of the two models as explained in [10]. We have followed the guidelines suggested in the literature to conduct statistical tests [13]. Commonly used accuracy evaluation criteria are Mean Magnitude of Relative Error (MMRE), PRED which are defined as below [10], [14]. Evaluation of Model Evaluation Criterion for Software Development Effort Estimation S. K. Pillai, M. K. Jeyakumar M World Academy of Science, Engineering and Technology International Journal of Computer and Information Engineering Vol:9, No:1, 2015 118 International Scholarly and Scientific Research & Innovation 9(1) 2015 scholar.waset.org/1307-6892/10000667 International Science Index, Computer and Information Engineering Vol:9, No:1, 2015 waset.org/Publication/10000667