Evaluating High Performance Computing on the Windows Azure Platform Eduardo Roloff, Francis Birck, Matthias Diener, Alexandre Carissimi, Philippe O. A. Navaux Informatics Institute Federal University of Rio Grande do Sul Porto Alegre, Brazil {eroloff, fbmoreira, mdiener, asc, navaux}@inf.ufrgs.br Abstract—Using the Cloud Computing paradigm for High- Performance Computing (HPC) is currently a hot topic in the research community and the industry. The attractiveness of Cloud Computing for HPC is the capability to run large applications on powerful, scalable hardware without needing to actually own or maintain this hardware. Most current research focuses on running HPC applications on the Amazon Cloud Computing platform, which is relatively easy because it supports environments that are similar to existing HPC solutions, such as clusters and supercomputers. In this paper, we evaluate the possibility of using Microsoft Windows Azure as a platform for HPC applications. Since most HPC applications are based on the Unix programming model, their source code has to be ported to the Windows programming model in addition to porting it to the Azure platform. We outline the challenges we encountered during porting applications and their resolutions. Furthermore, we introduce a metric to measure the efficiency of Cloud Comput- ing platforms in terms of performance and price. We compared the performance and efficiency of running these benchmarks on a real machine, an Amazon EC2 instance and a Windows Azure instance. Results show that the performance of Azure is close to the performance of running on real machines, and that it is a viable alternative for running HPC applications when compared to other Cloud Computing solutions. Keywords-High-performance computing (HPC); Perfor- mance Evaluation; Benchmark; Efficiency; NAS; I. I NTRODUCTION In the cloud computing field, the development of cloud services by different companies has impacted the scientific community. Several companies, such as Amazon, Google, IBM and Microsoft, are providing scalable cloud computing services that can potentially replace private cluster and grid systems [1]. Among these, the Amazon cloud has been extensively evaluated in the scientific community [2][3]. The reason for this popularity is that it uses the Infrastructure as a Service (IaaS) model [4]. In this model, developers have more flexibility to adapt the programming and execution environment to the needs of their applications. Deployment on IaaS is made using virtual machines, where the developer can choose the operating system and support libraries, among others. On the other hand, the Windows Azure platform is based on the Platform as a Service (PaaS) model, and evaluating its performance using standard HPC applications and bench- marks presents some challenges. In the PaaS model, the developer does not have control over the cloud infrastructure and the operating system, and has to use the tools and libraries supplied by the service provider. More specifically, the PaaS model leads to two challenges for the execution of HPC applications on Azure. First, since the source code of HPC applications is usually aimed at operating systems compatible with Unix, it needs to be converted to the Windows programming model supported by Azure. This is difficult in many cases, especially if support libraries (such as parallelization APIs or I/O libraries) are needed or if the build system or compiler are not compatible with Windows. Second, there are additional modifications necessary to adapt the application to the requirements of the Azure PaaS, such as the transformation of the source code into a library executable on Azure. For these reasons, there are few studies evaluating the performance of HPC applications on Windows Azure. Previous research in HPC on Azure focuses on writing new applications specifically for Azure [5] [6], which leads to duplication of work and lack of comparability between different solutions. The goal of this paper is to port a set of well-known HPC benchmarks, the NAS Parallel Benchmarks (NPB) [7], to the Azure PaaS and evaluate their performance. Our work has three main contributions. First, we analyze the complexity of porting existing HPC applications to the Azure platform, detailing the challenges and their resolutions. Second, we introduce a metric which compares the price and perfor- mance of different cloud computing solutions, which leads to the notion of efficiency of a cloud computing solution. Finally, we evaluate the performance and efficiency of Azure by comparing it to a real machine and to an Amazon IaaS cloud. This paper is organized as follows. In Section II we give an overview about related work in this area and compare them to our work. Section III describes and explains the porting process of HPC applications to the Azure platform. Moreover, we detail the challenges we encountered during the process and their solutions. We introduce an efficiency metric for cloud systems and present the evaluation method- ology of the performance experiments in Section IV. The results of our experiments are shown and discussed in Section V. In Section VI we conclude our work with discussions on the future of cloud computing on Azure based