Pattern Recognition Models for Smarter Infrastructure Sensing* I. Brilakis § , S. German § ABSTRACT This paper outlines a framework that intends to automate several civil infrastructure monitoring, assessment and management tasks. The first step in this framework is the paper’s main focus and aims to create pattern recognition models that will automate the recognition of infrastructure-related elements based on visual features from pattern recognition. This is achieved by identifying the visual characteristics of infrastructure element types and numerically representing them using image analysis tools. The derived representations along with their relative topology are then used to form element recognition models. These models are used as quantitative descriptions of the visual appearance of infrastructure element types for the purpose of automating their recognition. 1 INTRODUCTION The National Academy of Engineering recently listed “Restoring and Improving Urban Infrastructure” as one of the Grand Challenges of Engineering in the 21 st The key element missing is automation. Devising ways to achieve such automation is the topic of this paper. The long term objective of the author is to build on existing knowledge in the areas of Remote Sensing and Pattern Recognition and devise methods that will eventually enable the automation of several civil infrastructure monitoring, assessment and management tasks, such as construction sequence analysis, as-built quantity take-offs, as-built/as- designed comparisons, productivity and project monitoring and control systems, activity sequence analysis, maintenance decision making, lean construction, vulnerability assessment, etc. To achieve this long term objective, the author proposes a plan of action (Fig. 1) under which the first step (presented in this paper) is the creation of pattern recognition models that will automate the recognition of civil infrastructure- related elements based on the knowledge of 3D surfaces from remote sensing, and visual features from pattern recognition. century [1]. Two of the greatest challenges noted by the report are the need for more automation in construction, through advances in computer science and robotics, and the lack of viable methods to map and label existing infrastructure. For instance, over two thirds of the effort needed to model even simple infrastructure is spent on manually converting surface data to a 3D model [2]. The result is that as-built models are not produced for the vast majority of new construction and retrofit projects, which leads to rework and design changes [1] that cost up to 10% of the installed costs [3]. Any efforts towards automating the modeling process will increase the percentage of infrastructure projects being modeled and, considering that construction is a $900 billion industry [4], each 1% of increase can lead up to $900 million in savings. In the future, these models can then be used to accurately determine the size of objects (i.e. insulation/paint surface, excavation volume, etc.), the amount of identical or similar objects (bricks, doors, workers, trucks, etc.), the path, direction, velocity and position of materials, equipment and personnel at a site (tracking), and the size and shape of element *Supported by NSF grants 0933931 and 0904109. § Georgia Institute of Technology. Figure 1. Proposed framework Pattern Recognition Remote Sensing Visual features 3D Surfaces Measure Element Size Count Elements Track Elements Detect element defects State of Knowledge This Project Long term Contributions – Application Examples Insulation/Paint Surface, Excavation Volume Bricks, Doors, Workers, Trucks Materials, Personnel, Equipment Cracks, Air pockets, Spalling, Corrosion Application Areas As-built Quantity Take-Offs Project Control Systems Project Monitoring Systems Productivity Monitoring As-designed / As-built comparisons Maintenance Decision Making Activity Sequence Analysis Construction Sequence Analysis Infrastructure Elements Recognition