Action: A New Metric for Evaluating the Energy Efficiency on High Performance Computing Platforms (ranked on Green500 List) E. M. KARANIKOLAOU, M. P. BEKAKOS Dept. of Electrical & Computer Engineering Democritus University of Thrace 67100 Xanthi GREECE Abstract: - The need for new and more reliable metrics is always in demand. In this paper, a new metric is proposed for the evaluation of high performance computing platforms in conjunction with their energy consumption. The aim of the new metric is to reliably compare different HPC systems concerning their energy efficiency. The metric provides a mean to rank supercomputers of similar capabilities, avoiding the misleading results of metrics like performance-per-watt, currently used for ranking systems, as in the Green500 list, where systems with totally different sizes and capabilities are ranked consecutively. An example of this misuse for two adjacent systems in the Green500 list, is discussed. A comparative study for the energy efficiency of three high performance computing platforms, with different architectures, using the proposed metric is presented. This paper highlights the cases where a metric, like the one that is used in the Green500 list, may produce erroneous results in the ranking of the most energy efficient supercomputers. Key-Words: Energy efficiency, metrics, supercomputer ranking, HPC, distributed/shared memory platforms, manycore platforms, Green500. Received: July 5, 2021. Revised: November 8, 2021. Accepted: November 22, 2021. Published: January 5, 2022. 1 Introduction The evaluation of high performance computing platforms is a complicated issue and is a function of multiple correlated factors. These correlated factors include the application itself, the algorithm, the problem size, the programming language, the implementation, the amount of human effort for optimization, the compiler’s version as well as its capability for optimization, the operating system used, the system’s architecture, the load from other users or processes, the hardware specifications (CPU, size of cache, memory bandwidth, GPU), as well as the specifications of the interconnection network. In order to find the best performance of an algorithm on a specific High Performance Computing (HPC) system, the algorithm has to be tested for the maximum size of a problem that could fit into the system’s memory. Normally, an amount of memory has to be taken into account for the operating system processes. A good practice for estimating the maximum size of a problem that can fit into the memory is to calculate the problem size for a percentage of 80% of the total system’s memory [6]. Beyond performance, computer architects face a new significant challenge, the need for energy efficiency. Energy becomes an apparent obstacle to realize performance scaling; thus, low power techniques and algorithms for multicore systems, such as Dynamic Voltage Scaling (DVS) [18], have been a major design trend over the last years [3, 4]. Moreover, in an era where energy is a very valuable resource, there is a crucial need to define the tradeoffs between performance and energy consumption on HPC platforms. Energy is as important as performance and the list for ranking the world’s most energy efficient supercomputers, the Green500 list [5], is an effort to encourage supercomputing stakeholders to ensure that supercomputers are only simulating climate changes and not creating them [19]. However, the current evaluation metrics used for energy efficiency, as well as for performance measurements, are not optimal. Hence, new evaluation metrics are introduced -over time- measuring overall performance, while taking into account power and energy as well, in order to provide a better framework for ranking high performance computing systems [1, 2]. WSEAS TRANSACTIONS on COMPUTERS DOI: 10.37394/23205.2022.21.4 E. M. Karanikolaou, M. P. Bekakos E-ISSN: 2224-2872 23 Volume 21, 2022