Citation: Hafeez, B.; Anwar, M.W.;
Jamal, M.H.; Fatima, T.; Espinosa,
J.C.M.; López, L.A.D.; Thompson,
E.B.; Ashraf, I. Contextual Urdu
Lemmatization using Recurrent
Neural Network Models. Mathematics
2023, 11, 435. https://doi.org/
10.3390/math11020435
Academic Editors: Nebojsa Bacanin
and Catalin Stoean
Received: 29 November 2022
Revised: 5 January 2023
Accepted: 8 January 2023
Published: 13 January 2023
Copyright: © 2023 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
mathematics
Article
Contextual Urdu Lemmatization using Recurrent Neural
Network Models
Rabab Hafeez
1
, Muhammad Waqas Anwar
1
, Muhammad Hasan Jamal
1
, Tayyaba Fatima
1
,
Julio César Martínez Espinosa
2,3,4
, Luis Alonso Dzul López
2,3,5
, Ernesto Bautista Thompson
2,3,6
and Imran Ashraf
7,
*
1
Department of Computer Science, COMSATS University Islamabad, Lahore Campus, Lahore 54000, Pakistan
2
Higher Polytechnic School, Universidad Europea del Atlántico, Isabel Torres 21, 39011 Santander, Spain
3
Department of Project Management, Universidad Internacional Iberoamericana, Campeche 24560, Mexico
4
Department of Project Management, Universidade Internacional do Cuanza, Cuito EN250, Bié, Angola
5
Project Management, Universidad Internacional Iberoamericana, Arecibo, PR 00613, USA
6
Fundación Universitaria Internacional de Colombia Bogotá, Bogota 111311, Colombia
7
Department of Information and Communication Engineering, Yeungnam University,
Gyeongsan 38541, Republic of Korea
* Correspondence: imranashraf@ynu.ac.kr
Abstract: In the field of natural language processing, machine translation is a colossally developing
research area that helps humans communicate more effectively by bridging the linguistic gap. In
machine translation, normalization and morphological analyses are the first and perhaps the most
important modules for information retrieval (IR). To build a morphological analyzer, or to complete
the normalization process, it is important to extract the correct root out of different words. Stemming
and lemmatization are techniques commonly used to find the correct root words in a language.
However, a few studies on IR systems for the Urdu language have shown that lemmatization is more
effective than stemming due to infixes found in Urdu words. This paper presents a lemmatization
algorithm based on recurrent neural network models for the Urdu language. However, lemmatization
techniques for resource-scarce languages such as Urdu are not very common. The proposed model is
trained and tested on two datasets, namely, the Urdu Monolingual Corpus (UMC) and the Universal
Dependencies Corpus of Urdu (UDU). The datasets are lemmatized with the help of recurrent
neural network models. The Word2Vec model and edit trees are used to generate semantic and
syntactic embedding. Bidirectional long short-term memory (BiLSTM), bidirectional gated recurrent
unit (BiGRU), bidirectional gated recurrent neural network (BiGRNN), and attention-free encoder–
decoder (AFED) models are trained under defined hyperparameters. Experimental results show that
the attention-free encoder-decoder model achieves an accuracy, precision, recall, and F-score of 0.96,
0.95, 0.95, and 0.95, respectively, and outperforms existing models.
Keywords: neural networks; natural language processing; inflectional morphology; derivational
morphology
MSC: 68T50
1. Introduction
In today’s modern world, almost every job involves the use of computers, resulting in
the production of a massive amount of data that needs to be processed and analyzed by
computers [1]. Natural language processing (NLP) is the computerized study of human
languages that have naturally evolved over time. NLP and machine translation (MT) are
fields that are constantly evolving and helping to bridge the linguistic gap between individ-
uals. To comprehend the development of MT algorithms, along with a strong vocabulary,
the knowledge of different modules including morphological analysis and normalization is
Mathematics 2023, 11, 435. https://doi.org/10.3390/math11020435 https://www.mdpi.com/journal/mathematics