Continuum Mech. Thermodyn. https://doi.org/10.1007/s00161-018-0640-z ORIGINAL ARTICLE Maryam Ghodousi · Majid Shahgholi · Gholamhassan Payganeh Analysis of nonlinear vibrations and stability of rotating asymmetrical nano-shafts incorporating surface energy effects Received: 28 April 2017 / Accepted: 1 March 2018 © Springer-Verlag GmbH Germany, part of Springer Nature 2018 Abstract The objective of the present work is to investigate the nonlinear vibrations of the rotating asym- metrical nano-shafts by considering surface effect. In order to compute the surface stress tensor, the surface elasticity theory is used. The governing nonlinear equations of motion are obtained with the aid of variational approach. Bubnov–Galerkin is a very effective method for exploiting the reduced-order model of the equa- tions of motion. The averaging method is employed to analyze the reduced-order model of the system. For this purpose, the well-known Van der Pol transformation in the complex form and angle-action transformation are utilized. The effect of surface stress on the forward and backward speeds, steady state responses of the system, fixed points, close orbits and stability of the solutions is examined. The preliminary results of the research show that the absolute values of forward and backward whirling speeds in the presence of surface effect with positive residual surface stress are higher than those of regarding the system without surface effect and in the presence of surface effect with negative residual surface stress. In addition, it is seen that the undamped rotating asymmetrical nano-shaft, for specified value of detuning parameter, in the absence or presence of surface effect has various number of stable and unstable periodic solutions. Besides, there is different number of separatrix (homoclinic orbit type). Furthermore, bifurcations, number of solutions and their stability for damped rotating asymmetrical nano-shaft are investigated. Also, the above results have been obtained for rotating symmetrical nano-shaft. Keywords Surface effect · Asymmetrical rotating shaft · Van der Pol transformation · Nonlinear vibration · Stability 1 Introduction In order to study the mechanical behaviors of nano- and microstructures, the surface energy is one of the important fields which are investigated by researchers. A surface elasticity theory which has been generally used to comprehend the influence of surface energy on static and dynamic behaviors of nanomaterial was represented in [1–3]. In [4–6], and several other researchers have been made contribution to the further development of the surface energy theory. For example, Sharma et al. [7] investigated the effect of surface energy theory on the size-dependent elastic state of nano-inhomogeneities. The validation of a general thin plate theory in nanoscale thickness for size-dependent nonlinear response of thin elastic films that including surface effects was verified by Lu et al. [8]. In Ref. [9], the existence of mathematical solution of the initial-boundary problems in the linear elasticity with the surface stresses was studied. In Ref. [10], it has been shown that all equations of the linearized Gurtin–Murdoch model of surface elasticity can be derived, in a straightforward way, from a Communicated by Andreas Öchsner. M. Ghodousi · M. Shahgholi (B) · G. Payganeh Department of Mechanical Engineering, Shahid Rajaee Teacher Training University, P.O. Box 16788-15811, Tehran, Iran E-mail: Majid.Shahgholi@srttu.edu