Representing the Scaling Behavior of Parallel Algorithm-Machine Combinations zy Diane T. Rover zyxwv and Xian-He Sun Department of Electrical Engineering Michigan zyxwvuts State University East Lansing, MI 48824 Abstract zyxwvutsr The scaling of algorithms zyxwvutsrq and machines is essential to achieve the goals of high-perfarmance computing. Thus, scalability has become an important aspect of parallel algorithm and machine design. It is a desirable property that has been used to describe the demand for proportionate changes zyxwvutsr in performance with adjustments in system size. It should provide guidance toward an optimal choice of an architecture, algorithm, machine size, and problem size combination. However, as a performance metric, it is not yet well defined or understood. This paper summarizes several scalability metrics, including one that highlighss the behavior of algorithm-machine combinations as sizes are varied under an isospeed zyxwvutsrq condition. A scaling relation is presented to facilitate general mathematical and visual techniques for characterizing and comparing the scalability information of these metrics. 1 Introduction For massively parallel computing, we ideally seek performance that increases linearly with system size. In general, because of design restrictions and physical limitations, the efficiency provided by a given computer for a given algorithm will decrease when the system size is increased. Problem size in general has to be increased to achieve a linear increase of performance. The ratio of system size and problem size increases is closely related to the computation and communication pattems of the application and to the hardware support of the underlying architecture. The word scalable, or scalability, has been widely used in practice to describe how the system and problem size will influence the performance of parallel machines and algorithms. Scalability is a desirable property that has been used to describe the demand for proportionate changes in performance with adjustments in system size. More specifically, scalability is an indicator of the ability of a parallel architecture to support parallel processing at different machine sizes and of the inherent parallelism of a parallel algorithm. What are its capabilities as a performance metric? Can it predict the performance of Mathematical Sciences Department Clemson University Clemson, SC 29634 large system and problem sizes based on the performance of small system and problem sizes? Can it suggest which computer systems could be built with more processors and which algorithms might be more suitable for larger computer systems? As a useful metric, it should provide guidance toward an optimal choice of an architecture, algorithm, machine size, and problem size combination to deliver high performance. The performance metric and its measurement are open to debate. Salabilities of computer systems and programs are difficult to quantify, evaluate, and compare. Intensive research has been conducted in this area during recent years [I]. Traditional metrics, such as speedup, have been applied in analyses, however new metrics have been proposed to measure the scalability of algorithms and computer systems [2, 31, and a formal definition for the scalability of parallel machines also has been proposed [4]. Each of these proposed metrics provides ways to measure certain special properties of algorithms andor architectures. Motivated by the particular deficiencies of existing metrics, the authors have proposed yet another scalability metric [5]. It is based on an isospeed condition. Analytical and experimental results show that this metric definition provides a unique quantitative measurement to describe the behavior of a parallel algorithm-machine combination as sizes are varied. To interpret the scalability information provided by this and other metria-in a simple, uniform, and extensible context, we have adopted a general definition and visual representation based on mathematical relations. These representations should be useful for comparison and prediction. The isospeed scalability metric and general scaling relation are introduced in this paper. 2 Scalability Metria In general, scalability has been defined as: 1. A function of the variation in system size. 2. A property of the pair of parallel algorithm and machine (i.e., the parallel algorithm-machine combi- nation). 3. A quantity that can be calculated based on measure- ment, evaluated, compared, and used for prediction. 569 0-81862772-7192 $3.00 Q 1992 IBBB