Journal of Applied Technology and Innovation (e -ISSN: 2600-7304) vol. 5, no. 4, (2021) 51 Wearable IOT based Malaysian sign language recognition and text translation system Kalleisel Van Murugiah School of Engineering Asia Pacific University of Technology and Innovation (APU) Kuala Lumpur, Malaysia TP044288@mail.apu.edu.my G. Subhashini School of Engineering Asia Pacific University of Technology and Innovation (APU) Kuala Lumpur, Malaysia subhashini@staffemail.apu.edu.my Raed Abdulla School of Engineering Asia Pacific University of Technology and Innovation (APU) Kuala Lumpur, Malaysia raed@staffemail.apu.edu.my Abstract—Sign language recognition devices are gaining tremendous attention in recent years for helping speech and hearing-impaired community. The idea of fusing technology and sign language knowledge together to create a smart system is still being tried and developed all over the world with implementation with many different sign languages. In this paper, Malaysian Sign Language is given importance with 5 Malaysian Sign Language words being selected for recognition and prediction with new combination of sensor used compared to previous researches done for Malaysian Sign Language recognition and prediction. The combination of sensors used are 1 MPU9250, 1 MyoWare and 2 Force Sensitive Resistor sensors. 1D CNN time-series model is implemented with prediction accuracy ranging from 80 to 91 percentage. Keywords— Wearable IOT, Malaysian sign language, text translation system, hand gestures, Force Sensitive Resistor. I. INTRODUCTION Communication is the first tool used by a human being to convey thoughts, opinions, ideas, emotions and generally spoken words as a contribution to an ongoing conversation. Unfortunately, some human beings have certain barriers to communicate in an ordinary way, these group of people falls among hearing or speech impaired groups. Hearing and speech impaired people are individuals whom has difficulties in listening to a voice or a sound and also those whom are not able to talk. It is stated that statistically 160,000 Malaysian are found to belong in the hearing-impaired category according to [1]. A direct solution for the obstacle faced is communicating through sign language which has different grammatical representation and also classified as a visual language. Sign language employs signs made with the hands and other movements, including facial expression and postures of the body, used primarily by people who are deaf [2]. According to [3] there are 300 different sign languages in the world, Malaysian Sign Language (MSL) is also one of them. As for this project, Malaysian Sign Language is the main element to be discussed and the chosen sign language for recognition process. The MSL has its own way of representation of alphabets and common words which can be categorized by static signs and dynamic signs that has motion in hand gesture representation. Referring back to history, Malaysia had signed the United Nations Convention on the Right of People with Disabilities (UNCRPD) and passed the people with Disability Act (Act 685) to give the people with disabilities the opportunity to live as a normal citizen of Malaysia. At first instance, sign language has a lot of benefits for hearing and speech impaired individuals to communicate or convey information to others whom also possess the same knowledge as the conveyer. As this situation is seen in a deeper manner, the impaired individuals suffer in conveying the thoughts, opinions, or having a normal conversation to the majority of people that does not possess the same sign language knowledge that requires hours and multiples class of trainings. This situation is usually the main reason for the implementation of sign language interpreter job although it is indicated by the Malaysian Federation of the Deaf in 2018, there are under 100 affirmed gesture-based communication mediators in Malaysia. The described problematic situation will create a social gap between those with and without the knowledge of Malaysian Sign Language. To overcome this barrier, a substantial research is gaining attention all over the world, it is exclusively about a creating a smart predictive system to recognize and translate sign language into text or speech through the implementation of sensor and microcontroller. The explained advancement inspired to create this project of developing a Malaysian Sign Language recognition and translation system that uses Surface Electromyography (sEMG) sensors, Force Sensitive Resistor (FSR) sensor and one 6-axis motion sensor where 3-axis accelerometer and 3- axis gyroscope are used which is also called inertial measurement unit (IMU) sensor. Hand orientation, position and individual fingers movements are crucial elements analysed through this proposed system. The analysing process is performed through machine learning algorithm to train and test the data collected for the sensors to make an accurate prediction of possible output datasets that will be translated to text and speech. As the research is based on sensor data analysing system, it explains that sensors are being used in countless applications as we move to even more connected world in terms of technology [4] Plenty of applications require various inputs from sensors with no less in performance and used with exceptionally low power consumption. Combination of the useful sensor and machine learning algorithms will be substantial components for the solution of targeted problems of this research as listed below. • Recognition of two types of hand gestures that are classified into static hand gesture and dynamic hand gestures (gesture with motion). • Training and testing the system model by choosing a suitable machine learning classification method to get