Proceedings of the ASME 2021 16th International Manufacturing Science and Engineering Conference MSEC2021 June 21-25, 2021, Virtual, Online MSEC2021-62468 AN AUGMENTED-REALITY BASED HUMAN-ROBOT INTERFACE FOR ROBOTICS PROGRAMMING IN THE COMPLEX ENVIRONMENT Wenhao Yang Department of Industrial and Systems Engineering Rochester Institute of Technology Rochester, New York 14623, USA Email: wy7711@rit.edu Qinqin Xiao Warner School of Education University of Rochester Rochester, New York 14627, USA Email: qxiao5@u.rochester.edu Yunbo Zhang * Department of Industrial and Systems Engineering Rochester Institute of Technology Rochester, New York 14623, USA Email: ywzeie@rit.edu ABSTRACT To solve the problems of complex robot programming tasks, we propose an Augmented Reality (AR) based human-robot in- terface for planning a collision-free path in a complex environ- ment. Current robot programming methods usually require a high level of experience in robot programming (online program- ming), the time-consuming 3D modeling of the working environ- ment for collision detection (offline programming), and a tedious and inefficient re-planing to adapt environment or task changes (both online and offline programming). In order to address these problems, an end-to-end AR human-robot interface is proposed, which provides a new affordance to users by enabling them to plan the path in the AR environment. A set of user-interactive tools allow users to define and edit waypoints as the high-level guidance and the direct inputs for the toolpath planning package, Kinematics and Dynamics Library (KDL). With the fast sens- ing of the workspace and accurate rendering, an in-situ simu- lation module is utilized for collision check and verification by the users’ perception. Users will repeat the process of 1) way- points definition and editing, and 2) the collision checking and path feasibility verification, until a satisfactory path is obtained. A preliminary testing is conducted in a use case with complex obstacles to verified the effectiveness and the efficiency of the proposed interface. * Corresponding author. 1 Introduction Nowadays, the industrial landscape is being transformed into the fourth revolution due to the increasing mechanization, automation, digitalization, and miniaturization. This transforma- tion, also known as the Industrial 4.0 or Smart Manufacturing, is promoted by changing operative framework conditions, such as short development periods, individualization on demand and higher flexibility in production [1]. The trend of “Smart Fac- tory” is rapidly growing in manufacturing sectors in recent years, including equipped technologies related to holistically digitized models and applications of Ubiquitous computing [2]. Industrial robots are crucial components in manufacturing industries due to their properties of high reliability, precision, repeatability, and predictability [3]. Additionally, facing other problems like the shortage and high cost of skilled workers, many small and medium enterprises (SMEs) apply industrial robots in various manufacturing processes with the purpose of high effi- ciency and low-cost [4]. Therefore, industrial robots have been applied in a wide variety of applications, such as materials han- dling, spot welding, and surface manufacturing [5]. There are two main categories of robot programming meth- ods, namely, online programming methods and offline program- ming methods (OLP) [6]. Conventionally, the operator manually moves the end-effector of the robot for desired locations by us- ing a teach pendant, thus the relevant robot motions are recorded. Currently, the OLP method is becoming more popular both in in- dustries and research, which finds a path planning solution based 1 Copyright © 2021 by ASME