Citation: Sharma, P.; Tulsian, D.; Verma, C.; Sharma, P.; Nancy, N. Translating Speech to Indian Sign Language Using Natural Language Processing. Future Internet 2022, 14, 253. https://doi.org/10.3390/ fi14090253 Academic Editors: Filipe Portela and Paolo Bellavista Received: 18 July 2022 Accepted: 23 August 2022 Published: 25 August 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affil- iations. Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). future internet Article Translating Speech to Indian Sign Language Using Natural Language Processing Purushottam Sharma 1, *, Devesh Tulsian 1 , Chaman Verma 2, * , Pratibha Sharma 1 and Nancy Nancy 1 1 Amity School of Engineering & Technology, Amity University, Noida 201301, India 2 Department of Media and Educational Informatics, Faculty of Informatics, Eotvos Lorand University, 1053 Budapest, Hungary * Correspondence: puru.mit2002@gmail.com(P.S.); chaman@inf.elte.hu (C.V.) Abstract: Language plays a vital role in the communication of ideas, thoughts, and information to others. Hearing-impaired people also understand our thoughts using a language known as sign language. Every country has a different sign language which is based on their native language. In our research paper, our major focus is on Indian Sign Language, which is mostly used by hearing- and speaking-impaired communities in India. While communicating our thoughts and views with others, one of the most essential factors is listening. What if the other party is not able to hear or grasp what you are talking about? This situation is faced by nearly every hearing-impaired person in our society. This led to the idea of introducing an audio to Indian Sign Language translation system which can erase this gap in communication between hearing-impaired people and society. The system accepts audio and text as input and matches it with the videos present in the database created by the authors. If matched, it shows corresponding sign movements based on the grammar rules of Indian Sign Language as output; if not, it then goes through the processes of tokenization and lemmatization. The heart of the system is natural language processing which equips the system with tokenization, parsing, lemmatization, and part-of-speech tagging. Keywords: Indian Sign Language; natural language processing; tokenization; lemmatization; parsing 1. Introduction Every country has a different sign language which is based on their native language. It is not easy for us to speak when we know the other person is not listening, let alone hearing- impaired. Even we with sufficient hearing abilities tend to ignore or avoid communication with those who do not hear, and for those who cannot hear it becomes even more difficult. Having the skill to talk to those who cannot hear can not only bridge the gap between the two but also help in the exchange of a lot of ideas and new thoughts which could encourage these people to contribute to the development of technology. Every mind can contribute to making unknowns into knowns and impossible possible. 1.1. Indian Sign Language Indian Sign Language can facilitate people to create an inclusive society in which people with disabilities have equal chances for growth and development so that they can live productive, safe, and dignified lives. In India’s hard-of-hearing community, Indian Sign Language (ISL) is widely utilized. However, ISL is not utilized to teach hard-of-hearing students in deaf schools. Teacher education programs do not train teachers to use ISL in their classrooms. Sign language is not included in any of the teaching materials. The parents of hard-of-hearing children are often unaware of sign language’s value in bridging communication gaps. ISL interpreters are in high demand at institutes and other locations where hard-of-hearing and hearing individuals communicate, yet India only has about 300 licensed interpreters. ISL aims to conquer the following points: Future Internet 2022, 14, 253. https://doi.org/10.3390/fi14090253 https://www.mdpi.com/journal/futureinternet