15 August 1999 Ž . Optics Communications 167 1999 125–128 www.elsevier.comrlocateroptcom Observation of modulation instability in a fiber soliton ring laser D.Y. Tang a, ) , P.D. Drummond a , W.S. Man b , H.Y. Tam b a Centre for Laser Science, Department of Physics, The UniÕersity of Queensland, Brisbane, Qld 4072, Australia b Department of Electrical Engineering, The Hong Kong Polytechnic UniÕersity, Hung Hom, Kowloon, Hong Kong Received 22 February 1999; received in revised form 24 May 1999; accepted 26 May 1999 Abstract We report on an experimental observation of modulation instability in a passively mode-locked fiber soliton ring laser. We show that although in fiber soliton lasers soliton pulses are inherently stable against modulation instability, dispersive waves caused by the periodic perturbation of solitons propagating in the cavity are not. Due to modulation instability the dispersive waves can become unstable and consequently result in the generation of new spectral components in the soliton spectrum. q 1999 Elsevier Science B.V. All rights reserved. PACS: 42.55Wd; 42.65Sf; 42.65Tg Keywords: Fiber laser; Modulation instability; Soliton; Mode-locking Erbium doped fiber soliton lasers as a compact, stable source of ultrashort pulses have attracted con- w x siderable attention recently 1–11 . In an erbium doped fiber soliton laser, soliton pulses propagating in the cavity experience periodical perturbations caused by the cavity gain and output loss. It is a well-known fact that when the condition Z <Z is a 0 not met, where Z and Z are the amplifier and a 0 soliton period, respectively, a sideband instability characterized by the appearance of discrete spectral w x components in the soliton spectrum occurs 4,10,11 . Sideband instability arises as a result of the reso- nance between the periodic gain and loss perturba- tions experienced by a soliton and the characteristic w x frequency of the soliton 12–14 . It is the primary ) Corresponding author. E-mail: tang@physics.uq.edu.au mechanism limiting the minimum pulse duration ob- tainable in a fiber soliton laser. In this paper we report on an experimental obser- vation of another instability in a fiber soliton ring laser. We show experimentally that under certain conditions, new spectral components other than those caused by the sideband instability can appear in the soliton spectrum of the laser. Analyzing character- istics of these new spectra suggests that they are the result of modulation instability in the laser. In partic- ular, we point out that due to modulation instability, dispersive waves in a fiber soliton laser can become unstable, and consequently generate new spectral components in the soliton spectrum. The fiber soliton laser used in our experiment is schematically shown in Fig. 1. It is a unidirectional fiber soliton ring laser. Unidirectional operation is forced by the inclusion of a polarization dependent isolator in the cavity. The laser cavity is about 10 m 0030-4018r99r$ - see front matter q 1999 Elsevier Science B.V. All rights reserved. Ž . PII: S0030-4018 99 00277-1