Speculation Without Regret: Reducing Deoptimization Meta-data in the Graal compiler Gilles Duboscq 1 Thomas Würthinger 2 Hanspeter Mössenböck 1 1 Institute for System Software, Johannes Kepler University Linz, Austria 2 Oracle Labs, Linz, Austria {duboscq, moessenboeck}@ssw.jku.at thomas.wuerthinger@oracle.com Abstract Speculative optimizations are used in most Just In Time (JIT) com- pilers in order to take advantage of dynamic runtime feedback. These speculative optimizations usually require the compiler to produce meta-data that the Virtual Machine (VM) can use as fall- back when a speculation fails. This meta-data can be large and incurs a significant memory overhead since it needs to be stored alongside the machine code for as long as the machine code lives. The design of the Graal compiler leads to many speculations falling back to a similar state and location. In this paper we present deopti- mization grouping, an optimization using this property of the Graal compiler to reduce the amount of meta-data that must be stored by the VM without having to modify the VM. We compare our technique with existing meta-data compression techniques from the HotSpot Virtual Machine and study how well they combine. In or- der to make informed decisions about speculation meta-data, we present an empirical analysis of the origin, impact and usages of this meta-data. Categories and Subject Descriptors D.3.4 [Programming Lan- guages]: Processors—Compilers, Optimization, Run-time environ- ments General Terms Algorithms, Languages, Performance Keywords java virtual machine; just-in-time compilation; specu- lative optimization; metadata 1. Introduction Deoptimization [6, 8] is a concept used in modern VMs that allows program execution to fall back from an optimized state to a less optimized state, e.g., from compiled code to interpreted code. The ability to deoptimize the running program at almost any position allows the JIT compiler of a VM to perform aggressive optimiza- tions based on speculative assumptions. If those assumptions turn [Copyright notice will appear here once ’preprint’ option is removed.] out to be wrong later, the optimized compiled program falls back to an unoptimized interpreted execution or a baseline compiler 1 . However deoptimization usually imposes memory costs be- cause it require meta-data to be stored alongside the optimized machine code. While virtual machines may encode this data in a way that minimizes these costs, we believe that compilers can also improve on the amount of meta-data (i.e., deoptimization informa- tion) they handout to the VM. In this paper, we present deoptimiza- tion grouping, a VM-independent compiler optimization that helps reducing the amount of generated deoptimization information with- out compromising any speculative optimizations performed by the compiler. We also want to gather empirical data on reduction of meta- data overhead using our technique and using existing compression techniques from the HotSpot VM. In particular, we want to see if both techniques can be combined constructively. We also collect empirical data about deoptimization information throughout the VM: its production by the compiler, its memory overhead while stored and finally its usage in the VM. To summarize, the contributions of this paper are: • An analysis of the origin of deoptimization, its impact on as- sociated deoptimization information and its usage in the Java HotSpot VM. • A VM-independent compiler optimization that reduces the amount of deoptimization information that needs to be stored by the VM. 2. Background bytecode profiling info JIT compiler Interpreter Runtime Native methods machine code deoptimization info compiles interprets uses generates generates deoptimizes calls Figure 1: System overview We implement deoptimization grouping as part of the Graal OpenJDK project [10] and its Graal compiler. The Graal compiler is written in Java and is designed to produce highly optimized code. It can be used to replace the standard compilers of the Java HotSpot VM. 1 Deoptimizing to a baseline compiler is usually a special case of On Stack Replacement (OSR). 1 2014/8/18