This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSM.2015.2413755, IEEE Transactions on Network and Science Management IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, TO APPEAR 1 Striking a Balance Between Traffic Engineering and Energy Efficiency in Virtual Machine Placement Dallal Belabed, Graduate Student Member, IEEE, Stefano Secci, Member, IEEE, Guy Pujolle, Senior Member, IEEE, Deep Medhi, Senior Member, IEEE Abstract—The increasing adoption of server virtualization has recently favored three key technology advances in data-center networking: the emergence at the hypervisor software level of virtual bridging functions, between virtual machines and the physical network; the possibility to dynamically migrate virtual machines across virtualization servers in the data-center network (DCN); a more efficient exploitation of the large path diversity by means of multipath forwarding protocols. In this paper, we investigate the impact of these novel features in DCN optimization by providing a comprehensive mathematical formulation and a repeated matching heuristic for its resolution. We show, in particular, how virtual bridging and multipath forwarding impact common DCN optimization goals, traffic engineering (TE) and energy efficiency (EE), and assess their utility in the various cases of four different DCN topologies. We show that virtual bridging brings a high performance gain when TE is the primary goal and should be deactivated when EE becomes important. Moreover, we show that multipath forwarding can bring relevant gains only when EE is the primary goal and virtual bridging is not enabled. Index Terms—Virtual Bridging, Multipath Forwarding, Data Center Networking, VM Placement, Traffic Engineering. I. I NTRODUCTION T He advent of efficient software virtualization tech- niques allows running server virtualization at competitive performance-cost trade-offs with respect to legacy solutions. The increasing adoption of server virtualization has recently favored three key technology advances in data-center network- ing: the emergence of virtual bridging functions at the hypervi- sor software level (between virtual machines and the physical network); a more efficient exploitation of path diversity by means of multipath forwarding protocols; the possibility to dynamically migrate virtual machines across virtualization servers at different places in the data-center network (DCN). In the context of DCN optimization, virtual bridging is useful for the management of Virtual Machines (collocated in the same virtualization server), by offloading inter-Virtual Ma- chine (VM) traffic from access and aggregation switches at the expense of an additional computing load on the virtualization server. Moreover, with the emergence of flat DCN topologies, D. Belabed, S. Secci, and G. Pujolle are with Sorbonne Universit´ es, UPMC Univ Paris 06, UMR 7606, LIP6, F-75005, Paris, France (e-mails: dallal.belabed@upmc.fr, stefano.secci@upmc.fr, guy.pujolle@upmc.fr). D. Medhi is with the University of Missouri-Kansas City, 5100 Rockhill Road Kansas City, MO 64110-2499, USA (e-mail: dmedhi@umkc.edu). A preliminary version of the paper appears in the proc. of ITC 2014 [1], and some preliminary results also in the proc. of DCPERF’14 workshop [2]. This work was partially supported by the Systematic FUI 15 project RAVIR, the ANR Reflexion project (contract nb: ANR-14-CE28-0019), and the National Science Foundation grant CNS-0916505. such as Fat-Tree [3], DCell [4], and BCube [5] multipath forwarding can become useful to fully utilize the available paths and capacity and therefore offer higher throughput and resiliency to the servers. The ability to synchronize VM copies and migrate across virtualization servers (referred in the following also as ‘containers’ or ‘VM containers’) further adds elasticity to the cloud fabric by allowing fault-restoration and resource consolidation. Virtual machine placement algorithms typically address traf- fic engineering (TE) [6] [7] or energy efficiency (EE) [8] [9] objectives, such as to minimize the maximum link utilization when balancing the traffic load on DCN links or to maximize server utilization to turn off or hibernate some servers to save energy. Addressing TE and EE goals eventually leads to savings in DCN maintenance and planning costs while increas- ing the performance. The relationship between the presented three recent trends, virtual bridging 1 , multipath forwarding 2 , and VM placement, is a rather unexplored subject that we investigate in this paper. The contribution of the paper is two-fold: • Given that, to the best of our knowledge, no work at the state of the art offers a DCN optimization framework supporting virtual bridging and multipath forwarding, we formally formulate the virtual machine placement optimization problem with these features in a novel, compact, and versatile formulation. For its resolution with dense, flat, and large DCN topologies, we propose a repeated matching heuristic. • We analyze the impact of virtual bridging and multipath forwarding in DCN optimization with TE and EE objec- tives, and with a detailed sensibility analysis including four different DCN topologies (3-layer, FatTree, BCube, DCell) that cover all possible cases. We draw observa- tions on the case-by-case suitability of virtual bridging and multipath forwarding features with respect to DCN VM placement optimization. Section II presents the background of our work. The DCN optimization model is formulated in Section III, the proposed heuristic in Section IV, and simulation results are in Section V. Section VI concludes the paper. 1 The term ‘virtual bridging’ is used in the following to refer to the traffic switching operation at the software hypervisor level of the virtualization servers (VM containers). With virtual bridging enabled, the virtual bridge switches traffic between VMs in the same container, as well as traffic coming from outside and going outside the container. 2 The term ‘multipath forwarding’ is used in the following to apply to the case when the traffic load at a given switching node can be balanced over multiple network paths.