Shape Recognition for Irish Sign Language Understanding LIVIU VLADUTU Dublin City University School of Computing Glasnevin, Dublin 9 IRELAND lvladutu@computing.dcu.ie Abstract: The recognition of humans and their activities from video sequences is currently a very active area of research because of its many applications in video surveillance, multimedia communications, medical diagnosis, forensic research and sign language recognition. Our system is designed with the aim to precisely identify human gestures for Irish Sign Language (ISL) recognition. The system is to be developed and implemented on a standard personal computer (PC) connected to a colour video camera. The present paper tackles the problem of shape recognition for deformable objects like human hands using modern classification techniques derived from artificial intelligence. Key–Words: Statistical Learning, Shape recognition, Sign-Language 1 Introduction The purpose of the project is to develop a system for Irish Sign Language (ISL) understanding. Sign lan- guages are the native languages by which communi- ties of Deaf communicate throughout the world. De- spite the great deal of effort in Sign Language so far, most existing systems can achieve good performance only with small vocabularies or gesture datasets. In- creasing vocabulary inevitably incurs many difficul- ties for training and recognition, such as the large size of required training set, variations due to signers and to recording conditions and so on. Up to now the Deaf people had to communicate usually through an inter- preter or through written forms of spoken languages, which are not the native languages of the Deaf com- munity. The aim of the project is to develop this system using vision based techniques, independent of sensor- based technologies (using gloves) that can prove to be expensive, uncomfortable to wear, intru- sive and limit the natural motion of the hand. The images are extracted from simple ’one-shot’ video- streams, where only an individual gesture is executed, not linked to the consecutive signs (as in a normal sign-language conversation). 1.1 Main steps The classical steps of this type if human-computer interaction (HCI) system that were implemented by members of the team (see also [2, 3]) are: - hands and face detection; - tracking of the above mentioned human body parts using Hidden Markov Models; - shapes coding and classification using Machine Learning techniques ; - elimination of small area occlusion problems (hand-hand or hand-face occlusion) using motion estimation and compensation (Figure 1 below); - construction of faster implementation aiming the real-time ISL understanding system, using faster programming environments (mex programs based on C++ implementation); 1.2 Short description The work presented in the current paper investigates the detection of subunits (that compose a sign) from the viewpoint of human motion characteristics. In the model the subunit is seen as a continuous hand ac- tion in time and space; therefore, the clear shape un- derstanding at certain moments in time Representa- tive Frames (RFrames) for human action understand- ing is essential. One of the problems we faced is that the hand is a highly deformable articulate object with up to 28 degrees of freedom. Also the skin detec- tion for segmentation, see [2, 7] is based on the as- sumption that skin color is quite different from col- ors of other objects and its distribution might form a cluster in some specific color-spaces. Even in the case of specific difficult conditions, i.e. fast segmenta- tion imposed by the online requirement of the design, workarounds were found. The previous coding ap- proaches were dictated by classical implementation in Proceedings of the 9th WSEAS International Conference on SIMULATION, MODELLING AND OPTIMIZATION ISSN: 1790-2769 242 ISBN: 978-960-474-113-7