Operating System Support for Shared-ISA Asymmetric Multi-core Architectures Tong Li † , Paul Brett † , Barbara Hohlt † , Rob Knauerhase † , Sean D. McElderry ‡ , and Scott Hahn † † Intel Labs ‡ Digital Enterprise Group Intel Corporation {tong.n.li,paul.brett,barbara.a.hohlt,rob.knauerhase,sean.d.mcelderry,scott.hahn}@intel.com Abstract Current trends in multi-core processor implementa- tion scale by duplicating a single core design many times in a package; however, this approach can cause inefficient utilization of resources, such as die space and power. Recent research has proposed asymmetric cores as an alternative solution. This paper explores the design space for asymmetric multi-core architec- tures, and presents a case study and prototype of one design in which cores implement overlapping, but non- identical instruction sets. We propose fault-and-migrate, which enables the OS to manage hardware asymmetries transparently to applications. Our mechanism traps the fault when a core executes an unsupported instruction, migrates the faulting thread to a core that supports the instruction, and allows the OS to migrate it back when load bal- ancing is necessary. We have also developed three ap- proaches to emulate future asymmetric processors us- ing current hardware. Preliminary evaluation shows that fault-and-migrate enables applications to execute transparently and incurs less than 4% overhead for a SPEC CPU2006 * benchmark. 1. Introduction Advances in semiconductor technology have driven the hardware industry to integrate an increasing num- ber of cores on-chip. Most existing multi-core proces- sors consist of identical cores. However, as the num- ber of cores continues to scale, this design can lead to inefficient utilization of chip real estate. Recent re- search [2, 3, 7, 8, 9, 15, 16, 22] proposes asymmetric (or heterogeneous) architectures as an alternative solu- tion. For example, Kumar et al. [15] showed that, for the same die area, integrating out-of-order cores with a few in-order ones achieves comparable performance to traditional designs, but much higher energy efficiency. There is a large design space for asymmetric archi- tectures, ranging from cores differing only in clock speed to those differing in microarchitecture and ISA. The choices of hardware-software interface also vary. For example, hardware can hide asymmetry and ex- pose the traditional view of identical cores. Alterna- tively, it can expose some cores as CPUs while oth- ers as coprocessors or peripherals. It can also expose all cores as CPUs and allow software to manage them completely. This paper explores the design space for asym- metric multi-core architectures and presents a case study of one design, instruction-based asymmetry, in which cores have overlapping, but non-identical in- struction sets. To enable transparent execution of ap- plications, we propose an OS mechanism, fault-and- migrate, which traps the fault when a core executes an unsupported instruction and migrates the faulting thread to one that supports the instruction. Different policies exist as to when the thread can migrate back. For example, threads that execute unsupported instruc- tions infrequently can migrate back quickly for load balance, whereas threads that frequently execute those instructions can stay longer on the new core to avoid the overhead of fault handling and migration. We have implemented fault-and-migrate in the Linux kernel 2.6.20 and developed three approaches to emulate future asymmetric processors using current hardware. Our first approach emulates the removal of 1