Rendering Multi-party Mobile Augmented Reality From Edge Lei Zhang *† , Andy Sun † , Ryan Shea † , Jiangchuan Liu † , Miao Zhang † {lza70,hpsun,rws1,jcliu,mza94}@sfu.ca * College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China † School of Computing Science, Simon Fraser University, Burnaby, Canada ABSTRACT Mobile augmented reality (MAR) augments a real-world environ- ment (probably surrounding or close to the mobile user) by computer- generated perceptual information. Utilizing the emerging edge com- puting paradigm in MAR systems can reduce the power consump- tion and computation load for the mobile devices and improve responsiveness of the MAR service. Dierent from existing studies that mainly explored how to better enable the MAR services utiliz- ing edge computing resources, our focus is to optimize the video generation stage of the edge-based MAR services–eciently using the available edge computing resources to render and encode the augmented reality as video streams to the mobile clients. Speci- cally, for multi-party AR applications, we identify the advantages and disadvantages of two encoding schemes, namely colocated encoding and spilt encoding, and examine the trade-o between performance and scalability when the rendering and encoding tasks are colocated or split. Towards optimally placing AR video render- ing and encoding in the edge, we formulate and solve the rendering and encoding task assignment problem for multi-party edge-based MAR services to maximize the QoS for the users and the edge computing eciency. The proposed task assignment scheme is proved to be superior through extensive trace-driven simulations and experiments on our prototype system. CCS CONCEPTS • Information systems → Multimedia content creation; • Com- puting methodologies → Mixed / augmented reality; • Net- works → Cloud computing; • Human-centered computing → Ubiquitous and mobile computing systems and tools. KEYWORDS Augmented Reality, Mobile, Edge Computing ACM Reference Format: Lei Zhang *† , Andy Sun † , Ryan Shea † , Jiangchuan Liu † , Miao Zhang † . 2019. Rendering Multi-party Mobile Augmented Reality From Edge. In 29th ACM SIGMM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV ’19), June 21, 2019, Amherst, MA, USA. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/3304112.3325612 Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and the full citation on the rst page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specic permission and/or a fee. Request permissions from permissions@acm.org. NOSSDAV ’19, June 21, 2019, Amherst, MA, USA © 2019 Association for Computing Machinery. ACM ISBN 978-1-4503-6298-6/19/06. . . $15.00 https://doi.org/10.1145/3304112.3325612 Encoded Video User Interactions Video Stream Rendering & Encoding Instances Sensor Data/ User Inputs Rendering Instance Encoding Instances MAR Clients (b) Split Encoding (a) Collocated Encoding MAR Clients Figure 1: MAR rendering and encoding 1 INTRODUCTION Integrating powerful sensing capability and unparalleled mobile communication abilities smartphones have led to a plethora of new and exciting applications, e.g., cognitive assistance, virtual/augmented reality (VR/AR). Such mobile augmented reality (MAR) is extremely promising for a wide range of applications such as gaming, tourism, entertainment, advertisement, education, and manufacture [2]. It has been reported that MAR will be the primary drive of a $108 billion virtual/augmented reality market by 2021 1 . In response to such great popularity and huge market increase, major industry players have released their AR develop platforms, such as Apple’s ARKit 2 and Facebook’s AR Studio 3 , to incubate various novel MAR apps. Other than the single-device AR applications, multi-party AR applications are emerging and becoming more and more popular, which allow multiple users to share and interact with the same aug- mented reality and thus signicantly enhance the user experience. Examples of multi-party AR applications can be the AR games 4 that allow multiple players to participate and compete with other. Another timely example is augmented vehicular reality[10], which utilizes visual information from nearby vehicles to broaden the vehicle’s visual horizon through AR. Rather than being generated by each client individually, such multi-party AR is contributed and aected by all the users. 1 https://www.digi-capital.com/news/2017/01/after-mixed-year-mobile-ar-to-drive- 108-billion-vrar-market-by-2021/ 2 https://developer.apple.com/arkit/ 3 https://developers.facebook.com/products/ar-studio 4 https://developer.apple.com/documentation/arkit/swiftshot_creating_a_game_for_ augmented_reality 67