Self-healing in transparent optical packet switching mesh networks: A reinforcement learning perspective Iván S. Razo-Zapata ⇑ , Gerardo Castañón, Carlos Mex-Perera Department of Electrical and Computer Engineering, Tecnológico de Monterrey, Campus Monterrey, Av. Eugenio Garza Sada 2501 Sur, Col. Tecnológico, Monterrey, NL CP 64849, Mexico article info Article history: Received 22 June 2013 Received in revised form 24 September 2013 Accepted 4 November 2013 Available online 11 November 2013 Keywords: Optical packet switching networks Dimensioning Reinforcement learning Self-healing Attacks Monte Carlo simulation abstract While transparent optical networks become more and more popular as the basis of the Next Generation Internet (NGI) infrastructure, such networks raise many security issues because they lack the massive use of optoelectronic monitoring. To increase these net- works’ security, they will need to be equipped with proactive and reactive mechanisms to protect themselves not only from failures and attacks but also from ordinary reliability problems. This work presents a novel self-healing framework to deal with attacks on Trans- parent Optical Packet Switching (TOPS) mesh networks. Contrary to traditional approaches which deal with attacks at the fiber level, our framework allows to overcome attacks at the wavelength level as well as to understand how they impact the network’s performance. The framework has two phases: the dimensioning phase (DP) dynamically determines the optical resources for a given mesh network topology whereas the learning phase (LP) generates an intelligent policy to gracefully overcome attacks in the network. DP uses heu- ristic reasoning to engineer the network while LP relies on a reinforcement learning algo- rithm that yields a self-healing policy within the network. We use a Monte Carlo simulation to analyze the performance of the aforementioned framework not only under different types of attacks but also using three realistically sized mesh topologies with up to 40 nodes. We compare our framework against shortest path (SP) and multiple path rout- ing (MPR) showing that the self-organized routing outperforms both, leading to a reduction in packet loss of up to 88% with average packet loss rates of 1  10 3 . Finally, some con- clusions are presented as well as future research lines. Ó 2013 Elsevier B.V. All rights reserved. 1. Introduction Transparent Optical Packet Switching (TOPS) networks are a relatively new technology for very high data commu- nication rates, flexible switching and broadband applica- tion support. More specifically, they provide transparency features allowing data routing and switching without interpretation or regression of signals within the network, i.e. without opto-electronic-opto conversions [1]. Since TOPS networks only contain transparent optical components, they not only differ from the traditional opti- cal networks but also bring about a new set of problems for network security such as power drop (PD), wavelength misalignment (WM), In-Band Jamming (IBJ) and Out-Band Jamming (OBJ) [2]. Because of these problems, the wavelengths channels are seriously affected, i.e. they may remain inoperable for a given period of time. Moreover, these problems occur mainly due to failures and attacks. According to Rejeb et al., failures occur due to the physical natural fatigue and aging of optical devices [3]. They occur once and re- main within the devices until they are repaired. Contrary, the attacks appear and disappear often sporadically any- where in the network, causing additional failures and 1389-1286/$ - see front matter Ó 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.bjp.2013.11.002 ⇑ Corresponding author. Tel./fax: +52 8181582293. E-mail addresses: A00804384@itesm.mx (I.S. Razo-Zapata), gerardo. castanon@itesm.mx (G. Castañón), carlosmex@itesm.mx (C. Mex-Perera). Computer Networks 60 (2014) 129–146 Contents lists available at ScienceDirect Computer Networks journal homepage: www.elsevier.com/locate/comnet