An Investigation into Android Run-time Permissions from the End Users’ Perspective Gian Luca Scoccia Gran Sasso Science Institute L’Aquila, Italy gianluca.scoccia@gssi.it Stefano Ruberto Gran Sasso Science Institute L’Aquila, Italy stefano.ruberto@gssi.it Ivano Malavolta Vrije Universiteit Amsterdam Amsterdam, The Netherlands i.malavolta@vu.nl Marco Autili University of L’Aquila L’Aquila, Italy marco.autili@univaq.it Paola Inverardi University of L’Aquila L’Aquila, Italy paola.inverardi@univaq.it ABSTRACT To protect the privacy of end users from intended or unintended malicious behaviour, the Android operating system provides a permissions-based security model that restricts access to privacy- relevant parts of the platform. Starting with Android 6, the per- mission system has been revamped, moving to a run-time model. Users are now prompted for confirmation when an app attempts to access a restricted part of the platform. We conducted a large-scale empirical study to investigate how end users perceive the new run-time permission system of Android, collecting and inspecting over 4.3 million user reviews about 5,572 apps published in the Google Play Store. Among them, we iden- tified, classified, and analyzed 3,574 permission-related reviews, employing machine learning and Natural Language Processing techniques. Out of the permission-related reviews, we determined recurring points made by users about the new permission system and classified them into a taxonomy. Results of our analysis sug- gest that, even with the new system, permission-related issues are widespread, with 8% of collected apps having user reviews with negative comments about permissions. We identify a number of points for improvement in the Android run-time permission system, and provide recommendations for future research. KEYWORDS Android, Permissions, Apps, Opinion Mining, Review Analysis, Privacy, Security ACM Reference Format: Gian Luca Scoccia, Stefano Ruberto, Ivano Malavolta, Marco Autili, and Paola Inverardi. 2018. An Investigation into Android Run-time Permissions from the End Users’ Perspective. In MOBILESoft ’18: MOBILESoft ’18: 5th IEEE/ACM International Conference on Mobile Software Engineering and Systems , May 27–28, 2018, Gothenburg, Sweden. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3197231.3197236 Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. MOBILESoft ’18, May 27–28, 2018, Gothenburg, Sweden © 2018 Association for Computing Machinery. ACM ISBN 978-1-4503-5712-8/18/05. . . $15.00 https://doi.org/10.1145/3197231.3197236 1 INTRODUCTION As mobile devices provide more advanced features, more sensitive data are manipulated and stored, including not only personal infor- mation but also data collected via sensors [1]. Aiming at protecting the privacy of end users from intended or unintended malicious be- havior, the Android operating system provides a permissions-based security model that restricts the access to security- and privacy- relevant parts of the platform. Past research in the field has evidenced the existence of usability issues within the permission system [2–4]. Only a minority of users are aware of the implications of their privacy decisions and warning dialogs are not easily understood. Towards addressing these problems, the permission system has been revamped and, starting with Android 6 (i.e., Android API level 23), access to privacy- and security-relevant parts of the platform is enforced by a new run-time permission system. Under the new permission system, users are prompted for confirmation when an app attempts to access a restricted part of the platform for the first time [5]. In this paper, we investigate how end users perceive the new run-time permission system of Android, with the ultimate goal of identifying possible points of improvement present in the per- mission system, despite the recent changes. For this purpose, we conducted a large-scale empirical study on over 4.3 million user reviews about 5,572 apps published on the Google Play Store that adopt the run-time permission system (identified within an ini- tial dataset of over 18 million user reviews belonging to 15,124 apps). By using a combination of an established keyword-based ap- proach [6], we identified among them potential permission-related reviews regarding the new Android permission system. We manu- ally analyzed a statistically representative sample of the reviews, and categorized the main concerns expressed by end users about the new system into a taxonomy. Then, by making use of machine learning and Natural Language Processing (NLP) techniques, we classified a complete set of 3,574 permission-related reviews ac- cording to the previously-built taxonomy. Finally, by analyzing the achieved classification, we identified a number of points for improvement in the new permission system related to, e.g., the lack of clarity of developers when requesting permissions, excessive number of requested permissions, and permission-related bugs. We