Swaying displacement measurement for structural monitoring using computer vision and an unmanned aerial vehicle Tung Khuc a,⇑ , Tuan Anh Nguyen b , Hieu Dao b , F. Necati Catbas c a Department of Bridges and Highways Engineering, National University of Civil Engineering, Hanoi, Viet Nam b Department of Information Technology, National University of Civil Engineering, Hanoi, Viet Nam c Department of Civil, Environmental and Construction Engineering, University of Central Florida, Orlando, FL, USA article info Article history: Received 16 May 2019 Received in revised form 29 January 2020 Accepted 17 March 2020 Available online 20 March 2020 Keywords: Vision-based displacement measurement UAV Structural heath monitoring Noncontact measurement abstract Data acquisition is the challenging and crucial step for any structural health monitoring (SHM) scheme, especially on numerous measurement locations that are typically at very high elevations or largely inac- cessible areas, which are often linked to time-consuming, costly, and to some extent, dangerous sensor implementation and cable wiring. Noncontact vision-based measurement techniques have been recog- nized recently as a primarily feasible approach, although it is still characterized by some limitations. To address these constraints, the proposed study introduced an enhanced noncontact displacement mea- surement method that employed an unmanned aerial vehicle (UAV) and computer vision algorithms. Since UAV can carry cameras to approach any difficult-to-reach regions, the proposed system can over- come several bottlenecks of the state-of-the-art vision-based methods with regard to finding a stationary place for the camcorder and for mitigating the inaccuracy induced by the long distance between the cam- corder and the measurement location. Guided by the schematic framework for the system, a camera was mounted on the UAV for filming of the measurement point, and then displacements on that point were determined by a key-point vision-based measurement method. Moreover, translations generated by the UAV were obtained by means of reference objects on the background. Additionally, an autonomous scheme based on Canny edge detection and Hough transform were introduced for calculation of scale fac- tors between the pixel and engineering unit for every image frame to address the issue of very fluctuant distances from the UAV to the measurement location. Subsequently, the actual displacements of the mea- surement location were measured following the elimination of the UAV motions from the displacement data. The proposed system was verified on an experiment with a small-sized steel tower where the out- comes provided an initial confirmation of the approach’s promising potential. Ó 2020 Elsevier Ltd. All rights reserved. 1. Introduction A built structure is ascertained to undergo the processes of aging and deterioration through its estimated lifespan. Although aging is inevitable, degradation can be abated with gradual inspec- tion, assessment, and retrofitting, among other maintenance prac- tices, which on implied setbacks are typically costly and sometimes ineffective to the extent of expected collapse of the structure and similar accidents. As structural safety is a palpable demand in today’s engineering scenarios, methods and techniques capable of monitoring actual structures have attracted great atten- tion. There have been a number of schemes recently developed under structural health monitoring (SHM), although such are still faced with some barriers for real-life implementations, especially when it comes to the process of data acquisition process [1]. Any SHM framework begins with data acquisition from sensors. In most cases, the reliability of a specific SHM method is dependent on not only on the quality of the collected data but also on the measurement locations on the structures. For example, displace- ments collected from the top of a tower may carry more valuable information than the ones acquired from its base. Unfortunately, high-rise buildings, cable bridge towers, transmission steel towers, wind turbine pylons, etc., are highly characterized by locations containing rich information for monitoring studies, but often are difficult or even impossible to access. For example, mounting sensors including accelerations, strain gauges, linear variable dif- ferential transformers (LVDTs), and wiring cable on these positions are costly, time-consuming, and, to some extent, dangerous. https://doi.org/10.1016/j.measurement.2020.107769 0263-2241/Ó 2020 Elsevier Ltd. All rights reserved. ⇑ Corresponding author at: Department of Bridges and Highways Engineering, National University of Civil Engineering, 55 Giai Phong Street, Hanoi, Viet Nam. E-mail address: tungkd@nuce.edu.vn (T. Khuc). Measurement 159 (2020) 107769 Contents lists available at ScienceDirect Measurement journal homepage: www.elsevier.com/locate/measurement