Benchmark Applications Used in Mobile Cloud Computing: A Systematic Mapping Study Francisco Airton Silva, Paulo Maciel, Eder Quesado, Germano Zaicaner, Matheus Dornelas, Bruno Silva Federal University of Pernambuco (UFPE) – Recife, Pernambuco, Brazil {faps,prmm,eraqs,gz,mdr,bs}@cin.ufpe.br Abstract—Mobile Cloud Computing (MCC) integrates mo- bile computing and cloud computing aiming to extend mobile devices capabilities through offloading techniques. In MCC, many controlled experiments have been performed using mobile applications as benchmarks. Usually, these applications are used to validate proposed algorithms, architectures or frameworks. The task of choosing a specific benchmark to evaluate MCC proposals is difficult because there is no standard applications list. This paper presents a systematic mapping study for benchmarks used in MCC research. Taking five months of work, we have read 763 papers from MCC field. We catalogued the applications and characterizes them considering three facets: category (e.g.: games, imaging tools); evaluated resource (e.g.: time, energy); and platform (e.g.: Android, iPhone). The mapping study evidences research gaps and research trends. Providing a list of download- able standardized benchmarks, this work can aid better choices to guide more reliable research studies since the same application could be used for different scientific purposes. Index Terms—Mobile Cloud Computing; Offloading; Partition- ing; Performance Evaluation I. I NTRODUCTION Cloud computing can be defined as the aggregation of computing as a utility and software as a service where applications are delivered as services over the Internet and data centers provide those services [1]. In another side there are smartphone applications increase in complexity and re- quired resources. Unfortunately, the advances in smartphone hardware and battery life have been slow to respond to the computational demands of applications evolved over the years. Hence, many applications are still unsuitable for smartphones due to constraints, such as low processing power, limited mem- ory, unpredictable network connectivity, and limited battery life [52]. To tackle this problem a strategy called offloading is applied. Computation offloading is a process that migrates resource- intensive computations from a mobile device to the resource- rich cloud (called cloudlet, in case of nearby infrastructure). Cloud based computation offloading enhances the applications performance, reduces battery power consumption, and execute applications that are unable to execute due to insufficient smartphone resources. Mobile cloud computing (MCC) is an integration of cloud computing technology with mobile computing in order to make mobile devices resource-full in terms of computational power, memory, storage, energy, and context awareness. A sig- nificant amount of research has been performed on computa- tion offloading in such a field [18], [9], [33], [53]. These stud- ies usually focus on: why to offload (improve performance or save energy); when to decide offloading; what mobile systems use offloading; and which are the infrastructures for offloading. Aiming to conduct these studies, most of the researchers adopt mobile applications to prove their hypotheses, when proposing new theories. However, there is no place with a list of possible applications that could be used in experiments in mobile cloud computing. Consequently, researchers might not know the level of adoption of a specific application in the field. They may also be unaware of which platform (e.g.: iPhone) is more used combined with a specific application category (e.g.: Mathematical app). These pieces of information are useful to guide new research studies and standardize the characteristics of controlled experiments with offloading techniques. This paper provides such information by addressing and answering the question “What are the benchmark offloaded applications used in MCC and which characteristics define them?”. This paper presents a systematic mapping study, performed in order to map out the applications used in MCC field. By means of analyzing three facets (category, platform, and evaluated resource), we synthesize implications for practicing, identifying research trends, open issues, and areas for improve- ment. A mapping study is an evidence-based approach, applied in order to provide an overview of a research area, and identify the quantity and type of research. The remainder of this paper is organized as follows: In Section II, the systematic mapping study method is better described; Section III reports the findings based on the fre- quency of applications use; Section VI presents the related work; Section VII draws some conclusions and provides recommendations for further research on this topic. II. SYSTEMATIC MAPPING STUDY PROCESS A mapping study is a systematic process that provides an overview and summarizes published paper results of a partic- ular research area, by answering questions and categorizing studies. As main benefit, it can be used to identify gaps in the existing research that will lead to topics for further investigation. Thus, a systematic mapping study was used in this research to “map out” the benchmarks used in mobile cloud computing. The study follows the systematic mapping process proposed by Petersen et al. [43]. The essential process, is composed of five steps with specific outcomes and each phase is discussed in the following sections: (Definition of