Physica D 195 (2004) 1–28 Non-Gaussian error probability in optical soliton transmission G. Falkovich a , I. Kolokolov b , V. Lebedev c , V. Mezentsev d,∗ , S. Turitsyn d a Physics of Complex Systems, Weizmann Institute of Science, Rehovot 76100, Israel b Budker Institute of Nuclear Physics, Novosibirsk 630090, Russia c Landau Institute for Theoretical Physics, Kosygina 2, Moscow 117940, Russia d Aston University, Birmingham B4 7ET, UK Received 10 September 2001; received in revised form 14 October 2003; accepted 6 January 2004 Communicated by C.K.R.T. Jones Abstract We find the probability distribution of the fluctuating parameters of a soliton propagating through a medium with additive noise. Our method is a modification of the instanton formalism (method of optimal fluctuation) based on a saddle-point approximation in the path integral. We first solve consistently a fundamental problem of soliton propagation within the framework of noisy nonlinear Schrödinger equation. We then consider model modifications due to in-line (filtering, amplitude and phase modulation) control. It is examined how control elements change the error probability in optical soliton transmission. Even though a weak noise is considered, we are interested here in probabilities of error-causing large fluctuations which are beyond perturbation theory. We describe in detail a new phenomenon of soliton collapse that occurs under the combined action of noise, filtering and amplitude modulation. © 2004 Elsevier B.V. All rights reserved. Keywords: Soliton; Non-Gaussian statistics; Error probability; Optical communication 1. Introduction Solitons play an important role in the dynamics and statistics of nonlinear systems in fields as diverse as hydrody- namics, plasmas, nonlinear optics, molecular biology, solid state physics, field theory, and astrophysics. Presumably the most impressive practical implementation of the fundamental soliton concept has been achieved in fiber optics, where soliton pulses are used as the information carriers (elementary “bits”) to transmit digital signal at high bit rates over long distances. Fiber optic applications of the soliton theory are governed by the integrable nonlinear Schrödinger equation (NLSE) and its modifications related to different control elements introduced into the optical line. The limitations on the error-free transmission distance are set mainly by the spontaneous emission noise added by in-line optical amplifiers. Even though the noise is weak one cannot generally use a perturbation approach to obtain the error probability because errors occur when signal changes substantially [2,3]. A priori it is not even clear whether one may still consider signal as a soliton-like or fluctuations with a substantial change of the waveform ∗ Corresponding author. E-mail address: v.mezentsev@aston.ac.uk (V. Mezentsev). 0167-2789/$ – see front matter © 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.physd.2004.01.044