A Semantic Geo-Tagged Multimedia-Based Routing in a Crowdsourced Big Data Environment Faizan Ur Rehman 1, 2 , Ahmed Lbath 1 , Abdullah Murad 2, 3 , Md. Abdur Rahman 2, 4 , Bilal Sadiq 2 , Akhlaq Ahmad 2, 5 , Ahmad Qamar 2, 6 , Saleh Basalamah 2, 4 1 Department of Computer Science, LIG, University of Grenoble Alpes, France, { 2 KACST GIS Technology Innovation Center, 3 Innovation and Entrepreneurship Institute, 4 College of Computer and Information Systems}, Umm Al-Qura University, Saudi Arabia, 5 KICT, International Islamic University Malaysia, 6 University Sains Malaysia {faizan.ur-rehman, ahmed.lbath}@imag.fr, {aamurad, marahman, rmsadiq, aajee, amqamar, smbasalamah}@uqu.edu.sa ABSTRACT Traditional routing algorithms for calculating the fastest or shortest path become ineffective or difficult to use when both source and destination are dynamic or unknown. To solve the problem, we propose a novel semantic routing system that leverages geo-tagged rich crowdsourced multimedia information such as images, audio, video and text to add semantics to the conventional routing. Our proposed system includes a Semantic Multimedia Routing Algorithm (SMRA) that uses an indexed spatial big data environment to answer multimedia spatio- temporal queries in real-time. The results are customized to the users’ smartphone bandwidth and resolution requirements. The system has been designed to be able to handle a very large number of multimedia spatio-temporal requests at any given moment. A proof of concept of the system will be demonstrated through two scenarios. These are 1) multimedia enhanced routing and 2) finding lost individuals in a large crowd using multimedia. We plan to test the system’s performance and usability during Hajj 2015, where over four million pilgrims from all over the world gather to perform their rituals. Categories and Subject Descriptors H.2.8 [Database Applications]: Spatial databases and GIS H.2.4 [Information Systems]: Query Processing General Terms Design, Algorithms. Keywords Crowdsourcing, Geo-Tagged Multimedia, Semantic Multimedia Routing, Spatio-temporal Multimedia Queries. 1. INTRODUCTION Hajj poses a unique challenge to existing routing applications due to the fact that more than 4 million pilgrims with diversity in language, level of education, and culture come to Makkah city mostly for the first time with limited or no prior knowledge of using routing through smartphone [1]. Hence, finding lost individuals or POIs in such a large crowd poses a great difficulty for pilgrims, event organizers, ministries, governments, health industries, emergency departments, and family members. The recent advancements in technologies have contributed in making the high-speed network more available and the location and multimedia enabled smartphones more accessible. As a result, the number of pilgrims sharing geo-tagged multimedia data, which carries semantics, have increased manifold, making crowdsourcing a reality. We argue that adding semantics in routing with the help of multimedia could make it easier for pilgrims with limited or no prior experience of map-based routing techniques to find each other and locate POIs. In this context, we present a novel semantic multimedia enabled routing mechanism to display routes that is easy to understand and can be used by most users. The system also provides constraint and context-aware multimedia information for POI exploring service. This will further help users to know the semantics of route with respect to their current location in real- time. For example, in case a user submits a query for a hotel, the system will show live results from all the nearby hotels with dynamic information such as vacancies, charges, images of nearby landmarks, available public and private parking with images, customer reviews and traffic constraints. Although geo-tagged multimedia has been used in many scenarios in the past but none has used in the field of adding semantics to routing. For example, in [2], the authors have used geo-tagged tweets to identify traffic constraints such as accidents and roadblocks whereas in [3] the authors have used a semantic algorithm to recommend interested POIs based on data collected from Foursquare and Instagram. Our system stands out from [2, 3] in that it 1) collects the source and destination of the route from the geo-tagged multimedia information submitted in real-time for route discovery 2) shows the publicly available POIs with multimedia associated with it within a certain radius of the calculated route to semantically help the user(s) and 3) indexes the different types of multimedia data separately in a spatial big data repository for an efficient real-time retrieval. Moreover, we have enhanced the indexing method [4] to support different types of real-time multimedia data with high arrival rates. 2. HIGH-LEVEL ARCHITECTURE Figure 1 shows the high-level architecture of the system. The system augments multimedia element to conventional routing using SMRA. Apart from the aggregated geo-tagged multimedia data, the system uses road network represented as graph G = (N, E) where N and E are the nodes and edges respectively for calculating the route. We divide the spatial area into cells; each Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author. Copyright is held by the owner/author(s) MM'15, October 26-30, 2015, Brisbane, Australia ACM 978-1-4503-3459-4/15/10. http://dx.doi.org/10.1145/2733373.2807985 759