DOI 10.1140/epjp/i2016-16425-7 Regular Article Eur. Phys. J. Plus (2016) 131: 425 T HE EUROPEAN P HYSICAL JOURNAL PLUS The nonlinear dispersive Davey-Stewartson system for surface waves propagation in shallow water and its stability Ehab S. Selima 1,2, a , Aly R. Seadawy 3,4, b , and Xiaohua Yao 1 1 Faculty of Mathematics and Statistics, Central China Normal University, Wuhan 430079, China 2 Mathematics Department, Faculty of Science, Menoufia University, Shebin El-kom 32511, Egypt 3 Mathematics Department, Faculty of science, Taibah University, Al-Ula, Saudi Arabia 4 Mathematics Department, Faculty of Science, Beni-Suef University, Beni Suef, Egypt Received: 31 August 2016 / Revised: 10 November 2016 Published online: 9 December 2016 – c  Societ` a Italiana di Fisica / Springer-Verlag 2016 Abstract. The three-dimensional (3-D) nonlinear and dispersive PDEs system for surface waves propagat- ing at undisturbed water surface under the gravity force and surface tension effects are studied. By applying the reductive perturbation method, we derive the (2 + 1)-dimensions form of the Davey-Stewartson (DS) system for the modulation of 2-D harmonic waves. By using the simplest equation method, we find exact traveling wave solutions and a general form of the multiple-soliton solution of the DS model. The dispersion analysis as well as the conservation law of the DS system are discussed. It is revealed that the consistency of the results with the conservation of the potential energy increases with increasing Ursell parameter. Also, the stability of the ODEs form of the DS system is presented by using the phase portrait method. 1 Introduction In the past decades, the nonlinear evolution equations of the mathematical physical models have been widely applied in many natural sciences such as hydrodynamics, the theory of turbulence, shallow water waves theory and other dynamical systems [1–5]. Wave propagation in an incompressible fluid is a classical problem in mathematical physics. In particular, in surface gravity waves, this subject was studied in two separated domains: shallow water [6–8] and deep water [9,10], and the theory for shallow water is more widely known. The nonlinear waves are propagating in many fields such as fluid, plasma, granular materials, etc. [11–15]. To solve these models, the following PDEs are generally used: Boussinesq, Korteweg de Vries (KdV), modified KdV (mKdV), Burgers, mKdV-Burger, Kadomtsev Petviashvili, Camassa Holm (CH), nonlinear Schr¨ odinger (NLS), and DS system [16–18]. The reductive perturbation method was used for obtaining these equations in plasma [19, 20] and, recently, in water wave problems [21–23]. In recent years, many powerful methods for finding the exact solutions to these PDEs and their stability analysis have been proposed such as Hirota’s bilinear method, the direct algebraic method, the Exp-function method, the complex tanh function method and its extensions, the complex amplitude ansatz method, the bilinear and B¨ acklund transformation, the simplest equation method and so on [24–34]. The water wave problem for incompressible and inviscid fluids has been studied in the presence of constant gravity acceleration and surface tension by Dullin et al. [6, 7] in one dimension and the CH and the KdV equations were obtained by using the Kodama transformation, but Demiray [21] studied it for the shallow water model without including the surface tension effect using the multiple time scaling method to obtain a set of KdV equations. The effect of surface tension and gravity force has been included in the study of Iizuka and Wadati [2] which applied the reductive perturbation method to the Rayleigh-Tayler problem for the interface between light and heavy inviscid incompressible fluids, and the stable and unstable NLS equations and the nonlinear diffusion equation were derived without finding their solutions. The same problem was revisited by Abourabia et al. [11, 17, 23] for the nonlinear surface waves to obtain the KdV and NLS equations which were solved analytically to illustrate the behavior of traveling solitary waves. a e-mail: es.selima@yahoo.com b e-mail: aly742001@yahoo.com