Chapter 15 Hand Gestures Recognition Model for Augmented Reality Robotic Applications Youshaa Murhij and Vladimir Serebrenny Abstract Augmented reality (AR) is a research promising field. Its main idea is to integrate and merge the virtual world with the real world. Augmented reality could improve or enhance our perception of the real world by integrating virtual objects. The existing hand gesture applications related to augmented reality can detect hand motion or track it in addition to the ability to construct a 3D model of tracked hand using markers and motion sensing devices like Kinect, LeapMotion, and AR/VR instruments. In this paper, a hand gesture recognition model based on deep convolutional neural network is proposed to be used in 3D virtual environments for robotic teleportation. This model is tested on HTC VIVE Pro AR/VR instruments using the VIVE eye and on a Kinect v2 to control an industrial manipulator in real time using only the hand movements in both online and offline control modes. 15.1 Introduction In the field of robotics, augmented and virtual reality applications focus on telepor- tation missions. So, the robot or the manipulator will follow the movements of the user in real time to achieve a remote-control task. Which means, the more accurate the tracking and recognizing system is, the more reliable teleportation system we get. Based on the researches in the last few years, we can say the neural networks and artificial intelligent took the lead from old image processing techniques which encouraged us to integrate them in our project to build recognition and tracking system to construct three-dimensional (3D) hand models to be used in 3D virtual environment. Y. Murhij (B ) · V. Serebrenny Robotic Systems and Mechatronics Department, Bauman Moscow State Technical University, Moscow, Russia e-mail: murkhizhyu@student.bmstu.ru V. Serebrenny e-mail: vsereb@bmstu.ru © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2021 A. Ronzhin and V. Shishlakov (eds.), Proceedings of 15th International Conference on Electromechanics and Robotics “Zavalishin’s Readings”, Smart Innovation, Systems and Technologies 187, https://doi.org/10.1007/978-981-15-5580-0_15 187