A Preliminary Performance Study on Nonlinear Regression Models using the Jaya Optimisation Algorithm Panagiotis D. Michailidis Abstract—Parameter estimation in nonlinear regression mod- els (NRMs) represents a major challenge for various scientific computing applications. In this study, we briefly consider a recent population-based metaheuristic algorithm named Jaya, which is used in estimating the parameters of NRMs. The algorithm is experimentally tested on a set of benchmark regression problems of various levels of difficulty. We show that the algorithm can be used as an alternative means of parameter estimation in NRMs. It is efficient in computational time and achieves a high success rate and accuracy. Index Terms—Nonlinear regression models, Parameter esti- mation, Optimisation, Metaheuristics, Jaya algorithm. I. I NTRODUCTION Regression analysis is an important statistical method for modelling the relationship between two or more variables using a data set. It has been used extensively in various areas of human and scientific activity to describe social and econometric phenomena [1]. Two major types of regression models exit: linear and nonlinear. In linear (LRM) and nonlinear (NRM) regression models, the regression function is linear and nonlinear, re- spectively, with respect to the parameters [2]. The parameter estimation problem in an LRM can be solved optimally using the method of ordinary least squares. By contrast, the same problem in NRMs cannot be solved easily. It is also a difficult task for traditional optimisation methods such as Gauss - Newton, and Levenberg - Marquardt. This difficulty of parameter estimation in NRMs is mainly due to the increased functional complexity [1]. The parameter estimation problem in NRMs is reduced to an optimisation problem. More specifically, it involves min- imising the nonlinear least squares. Not only can some classic optimisation methods be used to find the optimal values of parameters in NRMs but some population-based modern metaheuristic algorithms. The major problem of classic op- timisation methods is the trapping local minimal (e.g. Gauss - Newton) [3] as well as the required use of considerable mathematical operations such as matrix operations, gradient operation and Jacobean matrix calculation (e.g. Gauss - Newton and Levenberg - Marquardt) [4], [5]. Therefore, metaheuristic methods can be an alternative to nonlinear re- gression parameter estimation. These methods can mainly be classified into two categories: evolutionary algorithms (EA) (e.g. genetic algorithms (GA)) and swarm intelligence based algorithms (SI) (e.g. particle swarm optimisation (PSO)) [6]. Manuscript received June 15, 2018; revised October 5, 2018. P. Michailidis is with the Department of Balkan, Slavic and Oriental Studies, University of Macedonia, 54636 Thessaloniki, GREECE, e-mail: pmichailidis@uom.gr. Recently, Rao [6] introduced a population-based metaheuris- tic, known as Jaya. It is based on the idea that for a given problem the best solution can be obtained while avoiding the worst solution. In this study, we used the Jaya algorithm for the nonlinear regression optimisation problem because it is a straightforward and reliable optimisation algorithm which uses few parameters and is easy to implement [7], [8]. The Jaya algorithm prevents solutions from becoming trapped in local optima, giving it a notable advantage over other population-based optimisation methods [8]. Several studies have proposed using GA and PSO methods to address the parameter estimation problem of NRMs such as [9], [10], [1], [2]. In this study, we consider using the Jaya algorithm for the parameter estimation of NRMs. The Jaya algorithm is evaluated on 14 known nonlinear regression tasks having various levels of difficulty. Experimental results show that the algorithm is stable and reliable in solving the parameter estimation problem. The remainder of the paper is organized as follow. In Sec- tion II, we briefly describe the Jaya optimization algorithm for nonlinear regression. Section III presents the numerical results and comparisons of the Jaya optimisation algorithm with well-known NRMs. Finally, the conclusions of our study are presented in Section IV. II. SEQUENTIAL JAYA OPTIMISATION ALGORITHM Originally proposed by Rao [6], [11], the Jaya algorithm is a population-based metaheuristic for solving optimisation problems. The basic idea of the Jaya algorithm is that it consistently (i.e. at every iteration) tries to improve the solution by avoiding the worst possible solution. Through this mechanism, the algorithm aims to be successful or ‘victorious’ (‘jaya’ is a Sanskrit word meaning ‘victory’) [6] by finding the best possible solution. Algorithm 1 presents the framework of the Jaya algorithm. First, the algorithm accepts the control input parameters, such as the population size n, number of parameters m, lower and upper limits of the parameters (X min ,X max ), and the maximum number of iterations max iter. The algorithm also accepts the objective function f (x) of the optimisation problem. The algorithm then begins by initial- ising the population in the search space which represents candidate solutions (line 2). The population of the solutions is represented by an n × m matrix X i,j , where n is the population size, or number of candidate solutions, and m is the number of parameters. The population matrix for the algorithm is expressed as follows: IAENG International Journal of Applied Mathematics, 48:4, IJAM_48_4_09 (Advance online publication: 7 November 2018) ______________________________________________________________________________________