An English to Indian Sign Language Machine Translation System T. Dasgupta and A. Basu * Abstract—This paper presents a prototype English-To- Indian Sign Language (henceforth called, ISL) machine translation system. The system can be used to disseminate information to more than 1.5 million deaf people in India. Further, considering such a low literacy rate among the deaf population in India [Vasishta, Woodward and Santis (1998)], the system can also be used as an educational tool to learn ISL. Index Terms— Indian Sign Language, Hearing Impaired, Machine Translation I. I NTRODUCTION Indian Sign Language is the native language commonly practiced by the deaf community of India. It allows a deaf individual to convey thoughts and ideas using hands, arms, and face. Unlike spoken languages, ISL uses gestures instead of sounds in order to express a thought. Despite common misconceptions, recent research on ISL linguistics has established that sign languages are complete natural languages having their own grammatical structures, phonology, and complex set of morphological properties [Stokoe (1960)]. The morphology is complex in the sense that, it exhibits both sequential as well as simultaneous affixation of its manual as well as non- manual components [Sinha (2007)]. A sign is constructed by certain formational parameters like, dominant hand’s shape, location of the hand with respect to the body, extended finger orientation, palm orientation, movements, and non-manual activities. It has been estimated that more than 1 million deaf adults and around 0.5 million deaf children in India use ISL as a communication mode [Zeshan(2003)]. One out of every 5 people in the world uses ISL as a mode of communication. It is understandable that, information access for a deaf in common places like railway, bank and hospitals is very difficult. Hence, an automatic text to Indian sign language machine translation (Henceforth called, MT) system can help deaf people to access more information and services. In this paper, we present a framework for the syntactic transfer of English text to ISL. Our approach uses transfer grammar rules for the ISL sentence generation. The pro- totype MT system developed is a unidirectional system performing a structural transfer between English and ISL. II. RELATED WORKS ISL machine translation is still in its infancy. Most of the system encountered by us so far either performs * The authors are with the Dept. of Computer Science and Engineer- ing, Indian Institute of Technology, Kharagpur 721302. Fig. 1. System Architecture of the English to ISL MT system Sign Exact English (SEE) [Woodward (1973)] or they are domain specific. Further, lack of official status and linguis- tic studies makes the language a very poorly resourced [Vasishta, Woodward and Santis (1998)]. The only ISL MT system we encountered so far is the INGIT system. INGIT is a Hindi-To-Indian Sign Language MT system that has been built for the railway reservation domain [Kar, Reddy, Mukherjee, and Raina (2007)]. The system takes input from the reservation clerk and translates into ISL. The output of the system is an animated representation of the ISL-gloss strings via HamNoSys. III. OUR APPROACH The architecture of our system is illustrated in Figure 1. The input sentence is preprocessed and analyzed using a dependency parser to identify the higher syntactic and functional information of the sentence. This information is represented into a language independent intermediate recursive case frame structure. Each of the frames is asso- ciated with an attribute and a value corresponding to the attribute. This intermediate structure acts as an input to the generation module where, proper transfer grammar rules are applied to allow the structural transfer of the source sentence into target structure. The two main operations that are performed during the generation phase are: a) Lexical selection and b) Word order correspondence. Lexical selection is done using an English-ISL bilingual lexicon. For example, word like “Dinner” in English is replaced by “NIGHT FOOD” in ISL and “Mumbai” is replaced by the sign of “BOMBAY”. 44