Quasistable two-dimensional solitons with hidden and explicit vorticity in a medium with competing nonlinearities Hervé Leblond, 1 Boris A. Malomed, 2 and Dumitru Mihalache 3 1 Laboratoire POMA, UMR 6136, Université d’Angers, 2 Bd Lavoisier, 49000 Angers, France 2 Department of Interdisciplinary Studies, School of Electrical Engineering, Faculty of Engineering, Tel Aviv University, Tel Aviv 69978, Israel 3 Department of Theoretical Physics, Institute of Atomic Physics, P.O. Box MG-6, Bucharest, Romania Received 25 August 2004; published 15 March 2005 We consider basic types of two-dimensional 2Dvortex solitons in a three-wave model combining quadratic 2 and self-defocusing cubic - 3 nonlinearities. The system involves two fundamental-frequency FFwaves with orthogonal polarizations and a single second-harmonic SHone. The model makes it possible to intro- duce a 2D soliton, with hidden vorticity HV. Its vorticities in the two FF components are S 1,2 = ± 1, whereas the SH carries no vorticity, S 3 =0. We also consider an ordinary compound vortex, with 2S 1 =2S 2 = S 3 =2. Without the - 3 terms, the HV soliton and the ordinary vortex are moderately unstable. Within the propagation distance z 15 diffraction lengths, Z diffr , the former one turns itself into a usual zero-vorticity ZVsoliton, while the latter splits into three ZV solitons the splinters form a necklace pattern, with its own intrinsic dynamics. To gain analytical insight into the azimuthal instability of the HV solitons, we also consider its one-dimensional counterpart, viz., the modulational instability MIof a one-dimensional CW continuous- wavestate with “hidden momentum,” i.e., opposite wave numbers in its two components, concluding that such wave numbers may partly suppress the MI. As concerns analytical results, we also find exact solutions for spreading localized vortices in the 2D linear model; in terms of quantum mechanics, these are coherent states with angular momentum we need these solutions to accurately define the diffraction length of the true solitons. The addition of the - 3 interaction strongly stabilizes both the HV solitons and the ordinary vortices, helping them to persist over z up to 50 Z diffr . In terms of the possible experiment, they are completely stable objects. After very long propagation, the HV soliton splits into two ZV solitons, while the vortex with S 3 =2S 1,2 =2 splits into a set of three or four ZV solitons. DOI: 10.1103/PhysRevE.71.036608 PACS numbers: 42.65.Tg I. INTRODUCTION Two-dimensional 2Dspatial solitons with embedded vorticity S constitute a class of topologically charged local- ized nonlinear modes in optical media, which have recently drawn a lot of interest, starting with the work that predicted by means of direct simulationsstable vortices with S =1 in a model based on the 2D nonlinear Schrödinger equation with competing self-focusing cubic and self-defocusing quintic nonlinearities 1. Further analysis had supported this result by showing that the spectrum of eigenmodes of small perturbations around the vortex solitons does not contain un- stable eigenvalues, provided that the soliton’s integral power quadratic normexceeds a certain minimum value in other words, the external size of the corresponding annulus-shaped soliton must be large enough, see details in Ref. 2. It had also been found that higher-order vortex solitons in the 2D cubic-quintic CQmodel have their stability regions for S =2 2a short review of the topic was given in Ref. 3, and also for S 2 at least, up to S =5, although for S 3 the stability region is very narrow, corresponding to the vortex solitons with an extremely large size 4,5. A problem hampering experimental observation of the stable vortex solitons is that optical media, which may be approximated by the CQ nonlinear Schrödinger NLSequa- tion these are chalcogenide glasses 6and some organic substances 7, feature quite strong nonlinear loss two- photon absorptionalongside the CQ nonlinearity 8, which can easily kill any soliton 9. For this reason, and also be- cause it was necessary to understand how general the con- clusion about the existence of stable spatial vortex solitons was, this issue was further investigated in another model, which is also based on competing nonlinearities, viz., qua- dratic 2 alias second-harmonic-generatingand self- defocusing cubic - 3 terms without the latter one, all the vortex solitons in 2 media are subject to strong instability against azimuthal perturbations, which splits them into a set of separating zero-vorticity ZVstable solitons, as was shown both theoretically 10and experimentally 11. For the first time, stable vortices with S = 1 and S =2 which were called vortex rings, because of their annular shapein the combined 2 : - 3 model were found in Ref. 12. The sta- bility was demonstrated through the calculation of stability eigenvalues and verified by direct simulations see a brief review in Ref. 13. Recently, this analysis was extended to higher-order solitons, with a conclusion that, as well as in the CQ model, the vortex rings with S 2 have their narrow stability regions, corresponding to the rings with a very large outer radius 14. The 2 models involve, at least, two waves—the fundamental-frequency FFand second-harmonic SH ones. The soliton’s vorticity S is carried by the FF wave, whereas the SH vorticity is 2S. In the real experiment, the FF-SH phase matching, which is a necessary condition for PHYSICAL REVIEW E 71, 036608 2005 1539-3755/2005/713/03660812/$23.00 ©2005 The American Physical Society 036608-1