Extending the LLVM/Clang Framework for OpenMP Metadirective Support Alok Mishra Stony Brook University Stony Brook, NY, USA alok.mishra@stonybrook.edu Abid M. Malik Brookhaven National Laboratory Brookhaven, USA amalik@bnl.gov Barbara Chapman Stony Brook University Stony Brook, NY, USA barbara.chapman@stonybrook.edu Brookhaven National Laboratory Brookhaven, USA Abstract—OpenMP 5.0 introduces many new directives to meet the demand of emerging high performance computing systems. Among these new directives, the metadirective and declare variant directives are important to control the execution behavior of a given application by compile-time adaptation based on the OpenMP context. The metadirective directive allows the selection of OpenMP directives based on the enclosing OpenMP context as well as on user-defined conditions. The declare variant directive declares a specialized variant of a base function and specifies the context in which that specialized variant is used. Support for these directives are available in few compilers with some limitations. Support for metadirective is not available in Clang. This paper presents our implementation of the metadirective directive in Clang. In this paper, we present an implementation which supports the OpenMP 5.0 metadirective specification. However, in addition, this work also implements a dynamic extension to the user-specified conditions. A dynamic evaluation of user-defined conditions provides programmers more freedom to express a range of adaptive algorithms that improve overall performance of a given application. For example, a program can estimate the cost of execution, with respect to time taken or energy consumption, of a kernel based on some dynamic or static variables and decide whether or not to offload the kernel to GPU using the metadirective. Since there is a significant lack of knowledge about the usage and performance analysis of metadirective, the work also studies its impact on application characteristics. To achieve this, we have modified several benchmark codes in the Rodinia benchmark suite. The Rodinia benchmark includes applications and kernels which target multi-core CPU and GPU platforms which helps programmers study the emerging com- puting platforms. Our modification to the Rodinia benchmarks enables the application developer to study the behavior of metadirective. Our analysis reveal that the main advantage of the dynamic implementation of metadirective is that it adds minimal to no overhead to the user application, in addition to allowing flexibility to the programmers to introduce portability and adaptability to their code. Our modification of the Rodinia benchmark suite provides several guidelines for programmers to achieve better performance with metadirective. Index Terms—OpenMP 5.0, Metadirective, LLVM, Clang, Dynamic context. I. I NTRODUCTION To meet the demand of productive parallel computing on existing and emerging high performance computing systems, the OpenMP standard has evolved significantly in recent years [1]. Since the creation of the standard in 1997, that specified a handful of directives, substantial amount of new constructs have been introduced and most existing APIs have been enhanced in each revision. The latest version of OpenMP 5.0, released in 2018, has more than 60 directives [2]. Variant directives is one of the major features introduced in OpenMP 5.0 specification to facilitate programmers to improve perfor- mance portability. These directives can enable adaptation of OpenMP pragmas and user code at compile and runtime. The OpenMP specification defines traits to describe active OpenMP constructs, execution devices, functionality provided by an implementation, and context selectors based on the traits and user-defined conditions. It also defines variant directives like metadirective and declare variant which uses context selectors to choose various directives or functions. • The metadirective directive is an executable directive that conditionally resolves to another directive at compile time by selecting from multiple directive variants based on traits that define an OpenMP condition or context. • The declare variant directive has similar functionality as metadirective but selects a function variant at the call-site based on context or user-defined conditions. The mechanism provided by these directives for selecting vari- ants is more convenient to use than the C/C++ pre-processing since it directly supports variant selection in OpenMP and allows an OpenMP compiler to analyze and determine the final directive from variants and context as shown in Figure 1. However, OpenMP 5.0 restricts context traits to be com- pletely resolvable at compile time. This constrains the potential to optimize an OpenMP application based on runtime behavior or input data. An extension to allow runtime adaptation, based on properties like system architecture or input characteristics, may increase the efficiency of an application. Applications that would benefit from this feature include those that use traits based on problem size, loop count, and the number of threads. For example, most libraries parallelize and optimize matrix multiplications depending on the sizes of the input matrix. In this work we explore the possibility of extending metadirective user-condition to dynamically resolve the context selector based on runtime attributes. The high performance computing community has been exploring novel platforms to push performance forward [3]. 33 2020 IEEE/ACM 6th Workshop on the LLVM Compiler Infrastructure in HPC (LLVM-HPC) and Workshop on Hierarchical Parallelism for Exascale Computing (HiPar) 978-0-7381-1042-4/20/$31.00 ©2020 IEEE DOI 10.1109/LLVMHPCHiPar51896.2020.00009