Exploiting Semantic Trajectories using HMMs and BIM for Worker Safety in Dynamic Environments Muhammad Arslan Univ. Bourgogne Franche-Comte, Batiment I3M rue Sully, Dijon, France muhammad.arslan@u-bourgogne.fr Christophe Cruz Univ. Bourgogne Franche-Comte, Batiment I3M rue Sully, Dijon, France christophe.cruz@ubfc.fr Dominique Ginhac Univ. Bourgogne Franche-Comte, Batiment I3M rue Sully, Dijon, France dominique.ginhac@ubfc.fr Abstract— Understanding dynamic behaviors of moving objects using positioning technologies for construction safety monitoring is still an open research issue. One task; that is a small subset in the widespread field of objects dynamics is the enrichment of the location data of users with the semantic information for studying their mobility patterns in the context of the environment. However, incorporating the semantics related to the environment gets complex in case of the dynamic construction sites where the site spaces are kept evolving with time. For instance, new walls and infrastructure supports are added often on sites, while others are detached. Similar situations open more challenges to keep track of the changes in the attributes of the locations which involve with time for integrating semantics into the location data. Eventually, such changes to the site` locations will result in different user mobility patterns. For capturing the semantics of a dynamic environment and then understanding the user mobility patterns, a system is proposed based on semantic trajectories and Hidden Markov Model (HMM). In the end, Building Information Modeling (BIM) approach is used for visualizing the categorized user movements to help safety managers in monitoring site activities remotely by preventing other workforce from accessing such hazardous locations that involve unsafe movements. Keywords—Semantic trajectories; Building Information Modeling (BIM), mobility; Health and Safety (H&S); Hidden Markov Model (HMM) I. INTRODUCTION The construction industry is one of the largest industries and a major contributor to the economy [1]. Unfortunately, this industry is known for the least safe sector as compared to the other work sectors because construction workers are frequently exposed to the harsh environments [2]. Such uncertain and dynamic working environments consequence in high occurrences of serious injuries and even deaths [3, 4]. According to the Bureau of Labor Statistics (BLS), in 2016 out of 5,190 fatal occupational injuries, 937 were recorded from the U.S. construction industry [5]. Regardless of numerous efforts and more attention being paid to safety management practices in recent years [7], the rate of accidents in the construction industry continues to be high. The above statistics show that existing site monitoring systems for worker safety are not adequate for reducing fatal accidents. A closer look to the recent research reveals that one of the main reasons for construction accidents is because of unsafe worker mobility behaviors resulting in serious collisions with site objects and machinery [5]. For example, limited spatial awareness of the operating equipment involving sharp movements and rotations within the workers` proximity due to blind spots and surrounding noise can lead to hazardous situations on sites [8]. The latest technological developments in the location tracking systems have made very convenient to monitor sites for detecting unsafe worker behaviors [9]. The spatio-temporal points collected from a typical location acquisition system contains location coordinates with timestamps [10]. These raw points are transformed into finite meaningful episodes called trajectories after performing pre- processing techniques [10]. To achieve semantically enriched trajectories for enabling the desired understandings of the movements specific to the application, related contextual data of the environment needs to be integrated with the trajectories [11]. There exist many approaches in the literature for the semantic enrichment of trajectories [10-13]. The majority of these approaches are primarily designed for outdoor trajectory application scenarios and don’t have the capability for tracking the evolution of the building environment for constructing semantic trajectories. In order to perform semantic enrichment of trajectories by keeping track the dynamic building environment where the building objects are moving and changing with time, we have used our STriDE (Semantic Trajectories in Dynamic Environments) model. The main research objective of this study is not only to construct semantic trajectories but also to recognize the unsafe worker movements that can lead to accidents. As for recognizing and categorizing the movements, many case studies are present in the literature based on machine learning algorithms [14, 15]. Among them, statistical HMMs along with the Viterbi algorithm have been applied widely in many works and proved to be the most appropriate choice for categorizing movements and extracting patterns [16]. After using the HMMs, the categorized user movements are visualized using the Building Information Modelling (BIM) approach. The basic idea of using BIM is to have an interactive smart building model [17] that contains building geometry and real-time information related to the building locations which are more susceptible to have unsafe movements of the users. BIM-based visualizations can be used by the H&S managers that can lead to improved safety management intervention strategies by visualizing high-risk workers movements on a building map in real-time for preventing accidents. The paper is organized as follows. Section 2 introduces the related background literature. Section 3 presents the proposed solution for categorizing worker movements using HMMs and visualizing them on BIM. Section 4 discusses the proposed system, its benefits, and a conclusion.