Network Topology Optimization For Data Aggregation Soham Das and Sartaj Sahni Department of Computer and Information Science and Engineering University of Florida Gainesville, USA Email: {sdas, sahni}@cise.uﬂ.edu Abstract—In this paper, we show that the problem of conﬁguring the topology of a data center network to opti- mize data aggregation is NP-hard even when the number of aggregators is 1. Further, the approximation ratio of the algorithm proposed by Wang, Ng, and Shaikh [3] for the case of a single aggregator is (k+1)/2, where k is the degree of ToR (top-of-rack) switches and this algorithm also exhibits an anomalous behavior- increase in the switch degree may result in an increase in the aggregation time. By comparison, if topology conﬁguration is done using the longest processing time (LPT) scheduling rule, the approximation ratio is (4/3-1/(3k)). We show that for every instance of the single aggregator topology conﬁguration problem, the time required to aggregate using the LPT conﬁguration is no more than that using the Wang et al. rule. By coupling the LPT rule with the rule of Wang et al., we achieve a better throughput as promised by LPT and at the same time reduce the total network trafﬁc. Experimental results show that the LPT rule reduces aggregation time by up to 90% compared to the Wang et al. rule. The reduction in aggregation time afforded by a known improvement, COMBINE, of LPT relative to Wang et al. is up to 90.5%. More interestingly, when either of the LPT rule or COMBINE is augmented with the Wang et al. rule, total network trafﬁc is reduced by up to 90% relative to using LPT and COMBINE with chains. Keywords-Data Center Networks; Software Deﬁned network- ing; Big Data applications; Map-Reduce tasks I. I NTRODUCTION A large data center may be comprised of thousands of racks with each rack housing tens of computers [3]. Each rack has a top-of-rack (ToR) switch with some number (k) of optical links. Current commercial ToR switches have k ≤ 10 [3]. Software deﬁned networking (SDN) enables one to dynamically (re)conﬁgure the topology of the network comprised of ToR switches and their optical links. In the case of OpenFlow enabled ToR switches, this reconﬁgura- tion can be done by changing the OpenFlow rules stored in each ToR switch. In big data applications [1], [2], the time required to conﬁgure a desired network topology in this way is small compared to the time required to run the application using the conﬁgured topology. For example, in a big data MapReduce application, the time required to aggregate data from the mappers to the reducers is orders of magnitude greater than the reconﬁguration time. Hence, the performance of these applications can be enhanced by reconﬁguring the network topology to one that is optimized for the application rather than using a generic topology or A B(2) C(5) D(3) F(1) E(3) A F(1) C(5) D(3) B(2) E(3) a. b. Figure 1. Aggregation Tree prototype the one used by an earlier application. In this paper, we focus on network topology optimization for the data aggregation task common to many big data applications. In the data aggregation problem, data residing in several racks of a data center are to be aggregated into a single speciﬁed rack called the aggregator. In general, we may have several aggregators with each aggregating data from some subset of the racks. However, in this paper, we limit ourselves to the case of a single aggregator. Figure 1(a) shows an example aggregator tree. In this ﬁgure, the root (A) denotes the aggregator rack, the remaining nodes (B, C , D, E, and F ) denote racks from which data is to be aggregated into the root, and the edges denote optical links. The shown aggregator tree requires k ≥ 3 as node B uses three optical links to connect to nodes A, D, and E. To accomplish the required data aggregation, each node sends its data to its parent who then sends the received data to its parent and so on until all data reach the root. Suppose that node B has 2Gb of data to send to the root and that nodes D and E each have 3Gb of data to send to the root as shown in brackets in the ﬁgure. We assume that the data is transmitted in packets, a node can receive incoming data from all children in parallel, and that while a node is receiving packets from its children, it can transmit previously received packets to its parent. If each optical link has a bandwidth of 10 Gbps, the data from B, D, and E can be aggregated into A in (2 + 3 + 3)/10 = 0.8s. If C and F have 5Gb and 1Gb, respectively, of data that is to be aggregated into A, the time for this is 0.6s. Since A receives data from B and C in parallel, the total aggregation time is max{0.8, 0.6} = 0.8s. When the aggregation tree of Figure 1(b) is used, the aggregation time is max{0.1+0.3+0.3, 0.2+0.5} =0.7s. We deﬁne the total network trafﬁc to be the sum of the amounts of data moved through each optical link. For example, the total network trafﬁc using the topology of Figure 1(a) is 8 (link AB) + 6 (link AC ) + 3 (link BD)