37 th International Symposium on Automation and Robotics in Construction (ISARC 2020) A Robust Framework for Identifying Automated Construction Operations Aparna Harichandran a,b , Benny Raphael a and Abhijit Mukherjee b a Department of Civil Engineering, Indian Institute of Technology Madras, India b School of Civil and Mechanical Engineering, Curtin University, Perth, WA 6102, Australia E-mail: aparnaharichandran@gmail.com, benny@iitm.ac.in, abhijit.mukherjee@curtin.edu.au Abstract – Machine learning techniques have been successfully implemented for the identification of various construction activities using sensor data. However, there are very few studies on activity recognition in the automated construction of low-rise residential buildings. Automated construction is faster than conventional construction, with minimal human involvement. This requires high accuracy of identification for monitoring its operations. This paper discusses the development and testing of machine learning classifiers to identify normal automated construction operations with high precision. The framework developed in this work involves decomposing the activity recognition problem into a hierarchy of learning tasks in which activities at the lower levels have more details. The top recognition level divides the equipment states into two classes: ‘Idle’ and ‘Operations’. The second recognition level divides the ‘operations’ into major classes depending on the top-level activities performed by the equipment. The third recognition level further divides the activities into subclasses and so on. Since the number of classes and the similarity between them increase with the recognition level, identification becomes extremely difficult. The identification framework developed in this study classifies operations belonging to the parent class at each level in the hierarchy. The efficacy of this framework is demonstrated with a case study of a top- down modular construction system. In this construction system, the modules of a structural frame are assembled and lifted starting with the top floor followed by the ones below. The accelerometer data collected during top-down construction is used to identify the construction operations. The proposed framework shows superior performance over conventional identification using a flat list of classes. Keywords – Automated Construction; Construction Monitoring; Machine Learning; Accelerometer 1 Introduction Construction operations are monitored for several purposes like the determination of cycle time, productivity, fuel consumption, quality of work and possible failure conditions [1]–[3]. Identifying the activity with reasonable accuracy is sufficient for these purposes. However, for the development of a monitoring system to ensure safety, high accuracy of identification is necessary. Automated construction is faster than conventional construction, with minimal human involvement. In a fast automated construction system, an undetected faulty operation might cause catastrophic accidents [4], [5]. Besides, the level of detail required in this activity recognition problem is also higher. If an operation is detected as faulty in ongoing automated construction, the details like which operation, the stage of construction in which it happens and its location, have to be identified to take appropriate corrective actions. Hence, the operation identification problem has to be carefully formulated to develop a monitoring system. Existing studies on equipment activity recognition aim to improve the identification results by exploring advanced machine learning techniques, training options, hyperparameters, features extracted and also by carefully selecting the data [6]–[9]. The current study examines the significance of problem formulation in activity recognition. The main objective of this study is to identify automated construction operations with high accuracy. A hierarchical operation recognition framework has been developed in this study which involves decomposing the activity recognition problem into a hierarchy of learning tasks. At the top level, equipment states, ‘idle’ and ‘operations’ are identified. The activities at lower levels have more details. The performance of this framework is compared with that of the conventional approach to operation recognition which involves a flat list of classes to be separated. The two approaches were evaluated using data from an automated construction system (ACS) 473