PHYSICAL REVIEW E 99, 032221 (2019) Unveiling the chaotic structure in phase space of molecular systems using Lagrangian descriptors F. Revuelta, 1, 2, * R. M. Benito, 1 , † and F. Borondo 2, 3 , ‡ 1 Grupo de Sistemas Complejos, Escuela Técnica Superior de Ingeniería Agronómica, Alimentaria y de Biosistemas, Universidad Politécnica de Madrid, Avda. Puerta de Hierro 2-4, 28040 Madrid, Spain 2 Instituto de Ciencias Matemáticas (ICMAT), Cantoblanco, 28049 Madrid, Spain 3 Departamento de Química, Universidad Autónoma de Madrid, Cantoblanco, 28049 Madrid, Spain (Received 13 August 2018; revised manuscript received 25 January 2019; published 27 March 2019) We explore here the feasibility of using the recently introduced Lagrangian descriptors [A. M. Mancho et al., Commun. Nonlinear Sci. Numer. Simul. 18, 3530 (2013)] to unveil the usually rich dynamics taking place in the vibrations of molecular systems, especially if they are floppy. The principal novelty of our work is the inclusion of p norms in the definition of the descriptors in this kind of system, which greatly enhances their power to discern among the different structures existing in the phase space. As an illustration we use the LiCN molecule described by realistic potentials in two and three dimensions, which exhibits chaotic motion within a mixed phase space in the isomerization between the two wells corresponding to the linear isomer stable configurations, LiNC and LiCN. In particular, we pay special attention to the manifolds emerging from the unstable fixed point between the corresponding isomer wells, and also to the marginally stable structures around a parabolic point existing near the LiNC well. DOI: 10.1103/PhysRevE.99.032221 I. INTRODUCTION The rich dynamics usually exhibited by generic nonlinear systems is strongly influenced by the structures existing in their phase spaces [1]. Some of them, like invariant tori, confine trajectories in particular specific regions, where the motion is regular. Contrary, other trajectories have a more complex, yet chaotic, behavior, thus being much more com- plicated to compute and characterize, as Poincaré early dis- covered [2]. Still, their behavior is strongly influenced by invariant manifolds, which attract or repel motion toward or apart from different regions in phase space. The phase-space structure of conservative Hamiltonian systems with two degrees of freedom (2-dof) can be very well characterized using Poincaré surface of sections (PSOS). In these systems, the phase space is four dimensional, and the motion is always confined in the three-dimensional energy shell. If the value of one of the coordinates and the conjugate momentum are measured simultaneously on a certain plane, i.e., that defining the PSOS, the dynamics is then easily visu- alized and characterized. PSOS can then be used to identify invariant tori (rendering curves in the PSOS picture), which always correspond to regular motion, and those regions where chaotic motion takes place, which is shown as a dense sea of points, typically with no recognizable pattern at all. These chaotic regions emerge when invariant tori break down, as the celebrated Kolmogorov-Arnold-Moser (KAM) and the Poincaré-Birkhoff theorems [1] dictate. Nevertheless, the chaotic regions of phase space are not free from organization and hierarchy. Indeed, underlying * fabio.revuelta@upm.es † rosamaria.benito@upm.es ‡ f.borondo@uam.es structures exist, such as unstable periodic orbits (POs) and their associated invariant manifolds. In general, they present an infinite number of intersections, the so-called homoclinic tangle first described by Poincaré, whose infinite crossings and recrossings are responsible for all the complexity of the chaotic dynamics. Actually, these tangles, which can be either homoclinic (intersections of the invariant manifolds associ- ated with a single PO) or heteroclinic (intersections associated with the invariant manifolds of different POs) are cornerstones for the dynamics of nonlinear systems. Thus, their identifica- tion is key for a correct dynamical characterization of this kind of system. Unfortunately, PSOSs are difficult to visualize in systems with more than 2-dof because of their higher dimension. For example, the Poincaré map in a system with 3-dof is four dimensional. As a consequence, new visualization tools were developed for this purpose (see, for example, Refs. [3–7]). Also, other indicators of chaos and regular motion, measuring the stability of trajectories, were defined and used. Lyapunov exponents [8], measuring the stability of trajectories, are one of them. However, they are often difficult to compute, since their convergence is, in general, only achieved after an extremely long propagation in time [8]. Other short-time alternatives, such as fast Lyapunov indicators (FLI) [9,10] and their variants [11], circumvent the previous problem, being then much better suited for the task. Also, the small alignment index (SALI) [12,13], and the mean exponential growth factor of nearby orbits [14] are also efficient alternatives, much less demanding computationally than the Lyapunov exponents. Very recently, another powerful indicator of chaos, known as Lagrangian descriptors (LDs), has been introduced. These descriptors focus on the phase-space structures that are em- bedded in the chaotic regions of phase space. They were first introduced by Madrid and Mancho [15] under the name of 2470-0045/2019/99(3)/032221(10) 032221-1 ©2019 American Physical Society