arXiv:1912.09859v1 [cs.LG] 20 Dec 2019 1 Lightweight and Unobtrusive Privacy Preservation for Remote Inference via Edge Data Obfuscation Dixing Xu, Mengyao Zheng, Linshan Jiang, Chaojie Gu, Rui Tan, and Peng Cheng Abstract—The growing momentum of instrumenting the In- ternet of Things (IoT) with advanced machine learning tech- niques such as deep neural networks (DNNs) faces two practical challenges of limited compute power of edge devices and the need of protecting the confidentiality of the DNNs. The remote inference scheme that executes the DNNs on the server-class or cloud backend can address the above two challenges. However, it brings the concern of leaking the privacy of the IoT devices’ users to the curious backend since the user-generated/related data is to be transmitted to the backend. This work develops a lightweight and unobtrusive approach to obfuscate the data before being transmitted to the backend for remote inference. In this approach, the edge device only needs to execute a small-scale neural network, incurring light compute overhead. Moreover, the edge device does not need to inform the backend on whether the data is obfuscated, making the protection unobtrusive. We apply the approach to three case studies of free spoken digit recognition, handwritten digit recognition, and American sign language recognition. The evaluation results obtained from the case studies show that our approach prevents the backend from obtaining the raw forms of the inference data while maintaining the DNN’s inference accuracy at the backend. Index Terms—Internet of Things, edge computing, deep neural networks, privacy, data obfuscation I. I NTRODUCTION T HE fast development of sensing and communication tech- nologies and the wide deployment of Internet-enabled smart objects in the physical environments foster the forming of the Internet of Things (IoT) as a main data generation infrastructure in the world. The tremendous amount of IoT data provides great opportunities for various applications powered by advanced machine learning (ML) technologies. IoT in nature is a distributed system consisting of nodes equipped with sensing, computing, and communication capa- bilities. In order to build scalable and efficient applications on top of IoT, edge computing is a promising hierarchical system paradigm. In edge computing, the widespread net- work edge devices (e.g., home gateways, set-top boxes, and personal smartphones) collect and process the data from the end devices that are normally smart objects deeply embedded in the physical environments (e.g., smart toothbrushes, smart body scales, smart wearables, and various embedded sensors). Then, the edge devices interact with the cloud backends of Dixing Xu and Mengyao Zheng contributed equally to this research. They are with Xi’an Jiaotong-Liverpool University. This work was completed when Dixing Xu was visiting Nanyang Technological University (NTU) and then Zhejiang University (ZJU), and when Mengyao Zheng was visiting NTU. (e-mail: {dixing.xu15, mengyao.zheng16}@student.xjtlu.edu.cn) Linshan Jiang, Chaojie Gu and Rui Tan are with NTU. (e-mail: {linshan001, gucj, tanrui}@ntu.edu.sg) Peng Cheng is with ZJU. (e-mail: pcheng@iipc.zju.edu.cn) the applications to exchange processed data summaries and/or commands. Thus, by deploying certain data processing tasks on the Internet edge, the communication bandwidth usage can be reduced and the scalability of the IoT applications can be improved. However, the implementation of the IoT edge that can leverage the latest ML technologies faces two challenges: • Separation of data sources and ML compute power: With the advances of deep learning, the depth of inference models and the needed compute power to support these deep inference models increase drastically. Thus, the execution of these deep inference models on the IoT end or edge devices that have limited compute resources may be infeasible or cause too long inference time. Moreover, the execution of deep inference models on battery-based edge devices (e.g., smartphones) may not be desirable due to high power consumption. A remote server-class or cloud backend with abundant ML compute power including powerful hardware acceleration is still desired for deep inference model execution. • Confidentiality of inference models: A deployable infer- ence model often requires significant efforts in model training and manual tuning. Thus, an inference model in general contains intellectual properties under the enter- prise settings. Even when the edge devices can execute the model and meet timing/energy constraints, deploying the inference model to the edge devices in the wild may lead to the risk of intellectual property infringement (e.g., extraction of the model from the edge device memory). Moreover, the leak of the inference model can aggravate the cybersecurity concern of adversarial examples [1]. Therefore, it is desirable to protect the confidentiality of the deep inference models. To address the above two issues, remote inference is a natural solution, in which an edge device sends the inference data to the backend, then the backend executes the inference model and sends back the result. There are existing appli- cations adopting remote inference. PictureThis [2], a mobile App, captures a picture of plant using the smartphone’s camera and then sends the picture to the cloud backend that runs an inference model to identify the plant. Amazon Alexa, a voice assistant, processes captured voices locally and also transmits the voice recordings to the cloud backend for further analysis and storage [3], [4]. However, remote inference inevitably incurs privacy concerns, especially when the inference data is collected in the user’s private space and time, such as voice recordings in households [4]. The pictures for plant recognition