A support vector machine approach for real time vision based Human Robot Interaction Nishikanto Sarkar Simul Depertment of CSE, SUST Sylhet, Bangladesh nishikanto.abm@gmail.com Nusrat Mubin Ara Depertment of CSE, SUST Sylhet, Bangladesh mubin.ara.24@gmail.com Md. Saiful Islam Depertment of CSE, SUST Sylhet, Bangladesh saiful-cse@sust.edu Abstract—Today humanoid robots are being exhibited to redact various task as a personal assistant of a human. To be an assistant, a robot needs to interact with human as a human. For this reason robot needs to understand the human gender, facial expression, facial gesture in real time. Ribo - A humanoid robot build in RoboSUST lab which has the ability to communicate in Bangla with the people speaking in Bengali. In this article the authors show the implementation of theoretical knowledge of the recognition of real time facial expression, detection of human gender and yes / no from facial gesture in Ribo. Real time facial expression and gender detection can be performed using Support Vector Machine (SVM). A prepared dataset containing the facial landmarks leveled as five different expression: sad, angry, smile, surprise and normal, is given to SVM to construct a classifier. For the prediction of any expression, facial images are taken in real time and provided the facial landmarks data to SVM. Local Binary Pattern(LBP) algorithm is used for extracting features from face images. These features leveled as male and female are responsible to build the classifier. The face gesture for detecting ’yes/no’ is performed by tracking the movement of face in a certain time. After those implementations the principal results will make a framework that will be used in Ribo to recognize human facial expression, facial gesture movement and detect human gender. Index Terms— Human robot interaction; Ribo; Real time fa- cial expression; Facial gesture; Machine Learning; Landmarks; LBP; SVM. I. INTRODUCTION The natural process of interaction between human and robot is the most highlighted issue in autonomous robot world. With the advances of artificial intelligence, Humanoid Robot grows more expectation in human mind to guide robot in natural ways. To fulfill this purpose, Ribo is prepared to detect human gender, real time facial expression and facial gesture. Different types of machine learning algorithms named LBP, SVM etc. are used to build this framework. Various types of input data are processed by these algorithms to classify the tasks. Local Binary Pattern (LBP) [1] and Sup- port Vector Machine (SVM) [2] are used to classify human gender. Facial Landmark Detection [3] and SVM are also used to detect facial expression. Facial gesture is detected using OpenCV tools and some threshold values. Now Ribo has the ability to communicate with human by understanding facial gesture, facial expression and human gender. During conversation sometimes people want to an- swer a question using gesture. Its quite easy to understand this for a human but not so easy for a robot. So, authors’ concern is to make it easy to understand for any autonomous robot which is implemented in Ribo robot. Ribo also can get human facial expression and gender by which it can decide which type of conversation should be continued with the person standing in front of it. Fig. 1. Ribo II. RELATED WORKS Gender classification, facial expression and facial gesture detection these three works were performed in different sectors previously. Authors’ concern is to integrate all these three functionality in Ribo robot.