Model-Guided Performance Tuning of Parameter Values: A Case Study with Molecular Dynamics Visualization Yoonju Lee Nelson * , Bhupesh Bansal † , Mary Hall * , Aiichiro Nakano † , and Kristina Lerman * * USC/Information Sciences Institute; Marina del Rey, CA 90292 † University of Southern California; Los Angeles, CA 90089 Abstract In this paper, we consider the interaction between appli- cation programmers and tools that automatically search a space of application-level parameters that are believed to impact the performance of an application significantly. We study performance tuning of a large scientific application, the visualization component of a molecular dynamics sim- ulation. The key contribution of the approach is the use of high-level programmer-specified models of the expected performance behavior of individual parameters. We use these models to reduce the search space associated with the range of parameter values and achieve results that perform close to that of a more exhaustive search of the parameter space. With this case study, we show the importance of ap- propriate parameter selection, with the difference between best-case and worst-case performance with a particular in- put data set and processor configuration of up to a factor of 17. We show that through the use of models, we can drasti- cally reduce search time, examining only 0.3% to 5% of the search space, and usually select an implementation that is close to the best performance, within 0.84% to 15%, even though the models are not completely accurate. 1 Introduction The complexity of today’s architectures is growing, with many-core processors and petascale systems projected in the near future, increasing use of high-performance accel- erators such as graphics processors and FPGAs, and in- corporation of a diversity of architectural features includ- ing SIMD compute engines, deep memory hierarchies, and software-managed storage. Achieving high performance on a new architecture will demand that codes are carefully tuned to the unique features of that system. Historically, the burden of achieving high performance on new platforms has largely fallen on the application pro- grammers. For key portions of their computation, the pro- grammer derives a sequence of different but equivalent im- plementations. Each implementation variant is first de- bugged, and then its performance characteristics are eval- uated. This lengthy process continues until the programmer either arrives at a suitable implementation, or, as is often the case, decides to give up on further performance tuning. Dif- ferent implementations are often needed for different input data sets, or as the application is ported to other platforms. In this paper, we propose tool support for this perfor- mance tuning process, with the goal of dramatically increas- ing the productivity of application scientists in tuning and porting their codes. We focus on empirical techniques to find the best integer value of application-level parameters that the programmer has identified as being critical to per- formance. For example, in our case study, the computa- tion partitioning across X, Y and Z dimensions represents a set of dependent application-level parameters. Our work is closely related to Active Harmony, an automated tuning system that performs on-line selection of application-level parameter values [5, 15]. As compared with Active Har- mony, which assumes no a priori knowledge of the behavior of application parameters, we propose an interaction with the programmer whereby they provide high-level models of the impact of parameter values, which are then used by the system to guide and to prune the search for optimization parameters. We call this approach model-guided empirical optimization, in that models and empirical techniques are used in a hybrid approach. To evaluate this concept, we perform extensive experi- ments with a case study application, the visualization com- ponent of a molecular dynamics simulation. The applica- tion programmer has identified a set of five application-level parameters believed to have significant impact on perfor- mance. Through the programmer’s insights into perceived performance impact of the parameters, we have developed a set of general parameter models that describe common behavior of an optimization parameter, when considered in isolation. The programmer has also provided guidance on appropriate default parameter values and ranges. We use the case study to demonstrate the effectiveness of this approach. We ran a series of experiments on 1, 2, 4 and 8 processors for three time steps of an MD simulation to evaluate how well the models predict application performance and limit the number of points to be tested. 1