International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 03 Issue: 09 | Sep-2016 www.irjet.net p-ISSN: 2395-0072 © 2016, IRJET | Impact Factor value: 4.45 | ISO 9001:2008 Certified Journal | Page 162 Object Segregation by an Autonomous Robot using Microsoft Kinect Shantanu Ingle 1 , Madhuri Phute 2 1 Department of E&TC, Pune Institute of Computer Technology, Pune, Maharashtra, India 2 Department of E&TC, Pune Institute of Computer Technology, Pune, Maharashtra, India ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract - Detecting and recognizing objects are fundamental components of robotic navigation. In order to effectively interact with its environment, a robot must be able to classify and distinguish between multiple objects, of same or distinct colors, and the surrounding environment. It should eventually identify the position as well as structure of these objects, and update the status in real-time. We propose an idea that uses Microsoft Kinect along with a 3 wheel drive autonomous robot to achieve these goals. The algorithm that was developed for this project was able to accurately segment objects of different shapes and colors placed at different distances from the Kinect and place the objects using a robot to its desired location. In this project, a self driving autonomous robot, with a gripper attached to it and the Kinect mounted on top, is used to detect, recognize, pick up and place objects at their predefined locations. Using the RGB- D information obtained from Kinect, we are able to identify objects of a variety of colors and shapes placed at different distances from the robot. Key Words: Autonomous Robot, Kinect, Object Recognition, MFCC, Neural Network, Bluetooth, Ultrasonic Sensor. 1.INTRODUCTION Ever since Microsoft launched the Kinect Xbox 360 [1], back in 2010, researchers and hobbyists have widely experimented with it to exploit its potential for applications other than gaming. An intriguing aspect of the Kinect is that it provides distance data, i.e. a depth component, along with RGB data. This allows a 3-Dimensional interpretation of the captured 2-Dimensional image. Taking that aspect into account, this project aims at developing an autonomous robot, capable of recognizing voice commands and identifying objects based on them. The goal behind our idea is to develop a general framework for classifying objects based on RGB-D data from Kinect. Using this framework, a robot equipped with Kinect will take the name of an object as an input from an inbuilt microphone array of the Kinect sensor, scan its surroundings, and move to the most likely matching object that it finds. As a proof of our concept, we demonstrate our algorithm in an office/school environment. The scope of this project currently spans 3 basic objects, namely 'cube', 'box' and 'sphere' of 3 basic colors; 'red', 'green' and 'blue'. Being in its inchoate stage, the objects for classification are limited, but further detailed design would lead to applications spanning a variety of fields. One such example of its use is in home automation as a voice enabled personal assistant. Designed such that the physically disabled people or the aged, who are incapable of moving around on their own volition, can instruct the robot to fetch certain objects or carry out certain tasks. Military applications may also find use of this project, such as deployment of a dummy robot in a war field to identify any ammunition planted and eventually diffuse it, or maneuver around mapping the area and ultimately gauging threats. Medical assistants such as a real-time patient’s body monitoring nurse are also in consideration, which will keep track of the patient’s condition and send information about the patient’s health to the doctor in real time, and also administer any emergency remedies if required. Use of such robots at a warehouse for its management and functioning is another feasible future for this project. Logging of consignments, retrieving packages from their locations and sorting of packages based on a variety of parameters can easily be achieved. Not only will it be efficient, but also highly cost effective. 2. RELATED WORK Object recognition and segregation is a widely researched topic in the field of cognitive robotics. The detection of objects and maneuvering of intelligent autonomous robots is the motive behind development of a plethora of techniques, each with its own merits and limitations along with its intended area of application. Most techniques for object recognition today rely on a set of descriptors. In essence, a picture is taken from a camera and individual entities within the picture are determined by computing certain values of descriptors. The relations between them, or the 'distance' between these descriptor vectors is what determines the object[2]. A major focus of this project is to combine the depth data along with the RGB data from the Kinect to simulate a 3-Dimensional environment and identify not only the presence of an object but also its location in terms of lateral distance from the robot. Kinect is now being used for applications far beyond