Sobol’s Sequence Based Method for Fitting Nonlinear Mixed Effects Model: A Comparative View M. Rashedul Hoque 1 and A.H.M. Mahbub Latif 2 1, 2 Institute of Statistical Research and Training (ISRT), University of Dhaka, Dhaka- 1000, Bangladesh 1 Corresponding author: M. Rashedul Hoque, e-mail: rhoque@isrt.ac.bd Abstract Nonlinear mixed effects models are mixed effects models in which some of the fixed and random effects parameters enter nonlinearly to the model function. Nonlinear models are parsimonious so that we can capture the nonlinear variation with a minimum number of parameters. Due to their great importance, fitting of these models are also of crucial matters. A number of methods for fitting nonlinear mixed effects model are available in literature, most of the methods require approximating wither the model function or the likelihood function. A new method is proposed which numerically evaluate the integrations involved in the likelihood function with Monte Carlo integration using Sobol's sequence. The methods are compared using simulation studies and the method based on Laplace approximation is found to fit the nonlinear mixed effects model the best. The proposed Sobol's sequence based method performs better than some of the existing methods, especially in some cases; it produces good result in estimating random effects parameter. Thus, the Sobol's sequence based proposed method is very much compatible with the existing ones as well as the approximation based methods are quite handy. Keywords: Laplace approximation, Lindstrom and Bates approximation, intractable integrations, quasi Monte Carlo integration. 1. Introduction In nonlinear models responses are expressed as a nonlinear function of explanatory variables and are preferred over linear models due to parsimony by capturing nonlinear behavior of the system. There are some fixed effects parameters as well as some random effects parameters in mixed effects model. Such models can easily handle unbalanced repeated-measures data, can allow for flexible variance-covariance structures of the response vector, and are intuitively appealing. Thus nonlinear mixed effects model have properties of both nonlinear model and mixed effects model. There are two types of parameters in such models. These are the regression parameters associated with the fixed effects and the variance component parameters associated with the random effects. Regression parameters have subject-specific interpretation while variance component parameters have population average interpretation. Through maximum likelihood and restricted maximum likelihood several estimation methods have been proposed for mixed effects models where the later is generally adopted for linear mixed effects models (Harville 1977). In recent years several different nonlinear mixed-effects models have been proposed (Sheiner and Beal 1980; Lindstrom and Bates 1990; Davidian and Gallant 1992; Vonesh and Carter 1992). Most commonly used one is proposed by Lindstrom and Bates (1990). There are different estimation methods for the parameters in the nonlinear mixed effects model (Davidian and Giltinan 1993) and there is an ongoing debate in the literature about the most adequate methods. One of the reasons for this variety of estimation methods is related to the numerical Proceedings 59th ISI World Statistics Congress, 25-30 August 2013, Hong Kong (Session CPS201) p.5069