1 Hand Talk- Implementation of a Gesture Recognizing Glove Celestine Preetham, Girish Ramakrishnan, Sujan Kumar, Anish Tamse Dr. Nagendra Krishnapura Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai 600 036, India cpliitm@gmail.com Abstract—We present our prototype for a gesture recognizing glove (data glove). We use low cost packaging material (velostat) for making piezoresistive sensors. These flex sensors detect a bend in fingers and we map this data to a character set by implementing a Minimum Mean Square Error machine learning algorithm. The recognized character is transmitted via Bluetooth, to an Android phone, which performs a text to speech conversion. Our motivation for Hand Talk is to compare hand configurations with sign language charts and generate artificial speech which articulates the gestured words. This technology also has further applications as a 3D mouse, virtual keyboard, control for precision control of robotic arms. I. I NTRODUCTION Devices and gadgets that aid the differently-abled to lead normal and convenient lives has always been an area that has attracted innovation. Recent advancements in technology, like low-power electronics and wireless devices and ability to design both the analog front-end and digital processing back- ends as integrated circuits has inspired a new range of wearable micro-devices. We have developed a glove with a motive to provide a low-cost solution to enable speech impaired persons to communicate using an artificial voice. It is easy for the user to use the device as it converts sign language to speech, and a lot of people who are speech impaired communicate with hand gestures. Our glove also has the additional capability of being able to learn from user gestures, so that it can convert more gestures into speech across a wide pool of users and even in different languages. In this paper, we present the structure of our device, high- lighting on how the sensors were made. We also discuss the block level implementation of the gesture to speech translator and report a few results of our preliminary testing. A. Technical Background The Sayre Glove, created by Electronic Visualization Lab- oratory at the University of Illinois at Chicago in 1977, was the first data glove [1]. One of the first commercially available data gloves was the Nintendo Power Glove in the year 1987. This was designed as a gaming glove for the Nintendo Entertainment System. It had a crude tracker and finger bend sensors, plus buttons on the back. The sensors in the PowerGlove were also used by hobbyists to create their own datagloves [2]. We took this as an inspiration for building our own data gloves with inexpensive materials. B. Proposed solution The primary input of the system would be the pose and orientation of the hand. We focus on acquiring to what extent each of the finger joints are bent. Upon acquiring this data it is encoded and transmitted wirelessly to a mobile device. The software in the mobile device would guess the shape and orientation of the hand based on the received data. We achieve the same using a simple implementation of a Minimum mean squared estimation. A speech synthesizer adapted from the Android stack articulates the gestured words. We proposed to implement our solution using four major modules as shown above. The glove with sensors sends the data to a microcontroller which sends it to a bluetooth module to transmit wirelessly. The transmitted data is received by cell phone and converted to speech. We aimed at building a low cost solution. The major constraint was the cost of commercially available flex sensors. Hence we built our own flex sensors using ESD materials which changes its resistance depending on the degree of bend. The cost of commercially available flex sensor is $10 per sensor, whereas the flex sensor we built costs less than 1 rupee per sensor.