ACM Journal of Autonomous and Adaptive Systems, Vol. XX, No. XX, Article XX. Publication date: Month YYYY. SDN Flow Entry Management Using Reinforcement Learning TING-YU MU, Western Michigan University, Kalamazoo MI 49008 U.S.A ALA AL-FUQAHA, Western Michigan University, Kalamazoo MI 49008 U.S.A KHALED SHUAIB, United Arab Emirates University FARAG M. SALLABI, United Arab Emirates University JUNAID QADIR, Information Technology University, Lahore Pakistan Modern information technology services largely depend on cloud infrastructures to provide their services. These cloud infrastructures are built on top of datacenter networks (DCNs) constructed with high-speed links, fast switching gear, and redundancy to offer better flexibility and resiliency. In this environment, network traffic includes long-lived (elephant) and short-lived (mice) flows with partitioned/aggregated traffic patterns. Although SDN-based approaches can efficiently allocate networking resources for such flows, the overhead due to network reconfiguration can be significant. With limited capacity of Ternary Content-Addressable Memory (TCAM) deployed in an OpenFlow enabled switch, it is crucial to determine which forwarding rules should remain in the flow table, and which rules should be processed by the SDN controller in case of a table-miss on the SDN switch. This is needed in order to obtain the flow entries that satisfy the goal of reducing the long-term control plane overhead introduced between the controller and the switches. To achieve this goal, we propose a machine learning technique that utilizes two variations of reinforcement learning (RL) algorithms —the first of which is traditional reinforcement learning algorithm based while the other is deep reinforcement learning based. Emulation results using the RL algorithm show around 60% improvement in reducing the long-term control plane overhead, and around 14% improvement in the table-hit ratio compared to the Multiple Bloom Filters (MBF) method given a fixed size flow table of 4KB. Categories and Subject Descriptors: C.4 [Computer System Organization]: Performance of Systems; I.6.5 [Simulation and Modeling]: Applications General Terms: Design, Algorithms, Experimentation, Performance Additional Key Words and Phrases: OpenFlow, Software Defined Networking (SDN), Big Data; Flow Entry, Elephant and Mice flows, Reinforcement learning, Machine learning, Ternary Content Addressable Memory, Mininet ACM Reference format: Ting-Yu Mu, Ala Al-Fuqaha, Khaled Shuaib, Farag M. Sallabi, and Junaid Qadir. 2018. SDN Flow Entry Management Using Reinforcement Learning. ACM Trans. Auton. Adapt. Syst. 1 INTRODUCTION The rapid growth of big data processing and the demand for massive-scale datacenters has increased the need for more efficient and intelligent network management systems, which has motivated an evolution of networking architecture towards Software-Defined Networking (SDN) [1], [3]. SDN is a popular networking paradigm of programmable networks that decouples the packet forwarding mechanism (data plane) from the control decisions (control plane). The goal of SDN is to provide an open programmable interface allowing the development of applications that dynamically control and manage connectivity among network elements. This enables the network to be more “application-aware,” and “network-aware.” Currently, OpenFlow [2], [3] is the dominating protocol used by controllers and switches in an SDN to exchange information. OpenFlow allows a switch to notify the controller of an incoming packet for which no forwarding rules can be found in the flow table. Similarly, a controller can send control messages to a switch requesting it to add a new or modify an existing forwarding rule in the flow table to serve the incoming packets. This information exchange process is also referred to as the network control overhead in this paper. This process consumes networking resources and introduces latency to the packet forwarding process.