Generation of higher-order optical „ 2 ¿ 1 … -dimensional spatial vector solitons in a nonlinear anisotropic medium Carsten Weilnau* and Cornelia Denz Institute of Applied Physics, Westfa ¨lische Wilhelms-Universita ¨t Mu ¨nster, Corrensstrasse 2-4, D-48149 Mu ¨nster, Germany Marcus Ahles, Andreas Stepken, Kristian Motzek, and Friedemann Kaiser Institute of Applied Physics, Darmstadt University of Technology, Hochschulstrasse 4-6, D-64289 Darmstadt, Germany ~Received 23 May 2001; published 9 October 2001! We investigate the generation of higher-order optical vector solitons in two transverse dimensions in aniso- tropic nonlinear media consisting of an incoherent superposition of a Gaussian beam and a higher-order laser mode with a complex internal modal structure. We demonstrate both numerically and experimentally various examples of these stable self-trapped light structures and show that vortex modes carrying topological charge always decay into multiple-humped structures that remain self trapped during propagation. Furthermore, we demonstrate the mutual stabilization of a triple- and a double-humped transverse light structure leading to the formation of a two-dimensional vector soliton without a stabilizing fundamental Gaussian mode. DOI: 10.1103/PhysRevE.64.056601 PACS number~s!: 42.65.Tg, 05.45.Yv, 42.65.Hw I. INTRODUCTION Stable self focusing of light in a medium with a saturable Kerr-type nonlinearity has attracted much research interest within the last decade @1#. A monochromatic and highly co- herent light beam propagating in a saturable nonlinear mate- rial induces a refractive index modulation that counterbal- ances the natural diffraction. Therefore, the beam remains self trapped and propagates as the fundamental mode of its self-induced waveguide. These self-focused light structures that consist of only one optical field are denoted as scalar solitons. In contrast, vector solitons are self-trapped optical beams that consist of more than one optical field. They were first suggested by Manakov @2# for the case of a Kerr non- linearity and two beams of different polarization states @3#. Here, at least two copropagating beams interact via the non- linear response of the material and jointly induce a multi- mode waveguide in which they propagate as eigenmodes @4#. It is essential for the formation of all kinds of vector solitons that the interference between the individual components must not contribute to the induced refractive index change D n , and therefore, it has to be destroyed. For this purpose it is convenient to use mutually incoherent components. Optical spatial vector solitons have been extensively ana- lyzed in the planar (1 11)-dimensional ~D! geometry in me- dia with a Kerr-like saturable optical nonlinearity. Various combinations of a fundamental single-humped and a double- humped beam were observed experimentally @5# and studied theoretically @6#. Further on, collision-induced shape trans- formation @7# as well as energy exchange upon collision @8# of these soliton pairs have been reported. Recently, the exis- tence of multicomponent solitary waves in two transverse dimensions has been predicted on the basis of a saturable and isotropic model @9,10# and subsequently has been observed in experiments @11,12#. They consist of one bell-shaped Gaussian beam and a second beam bearing a higher-order laser mode of Hermite-Gaussian ~HG! or Laguerre-Gaussian ~LG! type. Among various possible configurations it is par- ticularly the HG 01 -like dipole mode that allows the genera- tion of a very robust type of optical spatial vector soliton; the dipole-mode vector soliton. In contrast, all combinations of a fundamental Gaussian mode and a LG 01 -modelike vortex carrying topological charge ( m 51) are linearly unstable and decay into a stable dipole-mode structure carrying angular momentum @10,11#. The robustness of the dipole-mode vec- tor soliton motivates the investigations on higher-order mul- tihumped solitary waves. Multihumped self-trapped optical beams have only been realized in the planar (1 11)D geom- etry so far @5# and a numerical stability analysis revealed that combinations consisting of higher-order modes such as triple-humped transverse light structures are linearly unstable @6#. The instability that leads to a breakup of the combined structures becomes dominant at large propagation distances and was therefore not observed experimentally. Previous the- oretical investigations that describe the formation of vector solitons in photorefractive crystals are based on saturable and isotropic nonlinear models @9,10#. However, the photorefrac- tive nonlinearity is of anisotropic nature @13# and therefore, the experimental results deviate from numerical simulations. Here, we present a numerical analysis of vector solitons in bulk anisotropic medium. It is particularly the nonlocal na- ture of the anisotropic refractive index change in a DC- electric field biased strontium barium niobate ~SBN! photo- refractive crystal that supports the formation of these multicomponent solitary waves with an elaborate geometry. Moreover, we demonstrate experimentally and numerically the existence of composite solitons consisting of higher- order modes. Our contribution is divided into three main parts. First, we demonstrate that the incoherent combination of a Gaussian and a vortex beam with a topological charge of m 52 does not form a stable self-trapped structure but decays via an intermediate state consisting of two single-charged vortices into a triple-humped structure ~THS!. Second, we show that the structure, consisting of a triple-humped higher-order mode and a fundamental Gaussian mode, propagates self consistently in the nonlinear material and forms a triple- *Electronic mail: weilnau@uni-muenster.de PHYSICAL REVIEW E, VOLUME 64, 056601 1063-651X/2001/64~5!/056601~7!/$20.00 ©2001 The American Physical Society 64 056601-1