80 The Scientific Journal of King Faisal University Basic and Applied Sciences 00966502240692, erefaie@jazanu.edu.sa Corresponding Author: Eshrag Ali Refaee Detecting Hadith Authenticity Using a Deep-learning Approach Eshrag Ali Refaee Department of Information Technology and Security, College of Computer Sciences and Information Technology, Jazan University, Jazan, Saudi Arabia ASSIGNED TO AN ISSUE 01/06/2022 PUBLISHED ONLINE 14/04/2022 ACCEPTED 14/04/2022 RECEIVED 19/12/2021 LINK https://doi.org/10.37575/b/sci/210084 ISSUE 1 VOLUME 23 YEAR 2022 NO. OF PAGES 5 NO. OF WORDS 4860 ABSTRACT Hadith is a collection of texts containing sayings of the prophet Muhammad, which, along with accounts of his daily practice, constitute the second major source of legislation for Muslims after the Holy Koran. The Hadith collection comprises thousands of text pieces transferred over the years by many narrators with varying degrees of credibility. Hadith scholars are faced with the challenge of assessing the degree of a specific Hadith’s authenticity to classify the Hadith as Sahih (fully authentic and accepted) or Daif (rejected). Automatic Hadith classification has been addressed in the literature; however, the results vary and are not directly comparable, as no dataset has been made available for benchmarking. In addition, no previous work has utilised deep-learning (DL) approaches for Hadith classification. This work contributes by 1) collecting and publicly releasing a benchmark Hadith dataset of almost 4,000 Hadith texts to facilitate future research, 2) exploring DL model performance on binary Hadith classification tasks, and 3) benchmarking traditional machine learning against DL models. Our best results were recorded with an ARBERT DL model that provided an accuracy score of 91.56%. KEYWORDS Hadith classification; deep learning; Classical Arabic; machine learning; Hadith science; Hadith authenticity CITATION Refaee, E.A. (2022). Detecting Hadith authenticity using a deep-learning approach. The Scientific Journal of King Faisal University: Basic and Applied Sciences, 23(1), 80–4. DOI: 10.37575/b/sci/210084 1. Introduction According to the United Nations Educational, Scientific and Cultural Organisation UNESCO (2021), Arabic is the language of more than 400 million people around the world. Arabic can be divided into three major classes: Classical Arabic (CA), Modern Standard Arabic, and Dialectal Arabic (Habash, 2010). Hadith is one of the most well- known CA texts. The literal meaning of the word Hadith in Arabic is “anything spoken or told among people” (Al Ma’ni Online Dictionary, 2021). The Oxford Dictionary definition of Hadith is “a collection of traditions containing sayings of the prophet Muhammad which, with accounts of his daily practice, constitute the major source of guidance for Muslims apart from the Koran”. Hadith science is a branch of the Islamic sciences concerned with studying the sayings and actions of the prophet Mohammad (Al Ma’ni Dictionary, 2021). It is crucial since Hadith is the second primary source of legislation, namely a constitution, after the Holy Koran for almost 1.6 billion Muslims worldwide (Desilver and Masci, 2017). The Hadith corpus is vast and has been recognised in the universal European Language Resources Association catalogue (European Language Resources Association, 2021). Each Hadith consists of two main parts: a Matn, which is the body text of the Hadith, and an Isnad, which is the chain of narrators who have transmitted the referenced Hadith from the days of the prophet until the day that the Hadith was documented in one of the significant Hadith books (Duderrija, 2021). The most well-known Hadith books are Imam Al Bukhari, Imam Muslim and Ibn Majah. During Hadith collection and before any Hadith is written in one of the crucial Hadith books, Hadith collectors perform an extensive verification process, checking the degree of Hadith authenticity to avoid documenting any Hadith that is not genuine. Due to the Hadith’s vital role as a leading source of legislation, the Hadith collectors – Imam Al Bukhari, Imam Muslim and Ibn Majah, among others – take their responsibility for verifying Hadith authenticity seriously. For instance, they check whether all the chains of narrators of a particular Hadith are reliable, that is, they verify the absence of a reason to doubt their credibility. They also identify any gap between narrators, for example, if the lives of two consecutive narrators overlap and whether they have met in their lifetime (Azmi et al., 2019). Based on its level of authenticity, the Hadith corpus can be classified into three major categories: Sahih, which is fully verified to be genuine; Hasan, which is highly likely to be genuine and usually accepted by most scholars but does not qualify to be Sahih because of a minor issue; and Daif, or weak Hadith. The latter category refers to Hadith that does not have the qualifications of the Sahih or Hasan Hadiths. Weak Hadith is not used as evidence since at least one of its narrators has committed a transgression or been accused of lying or another act that negatively affects the narrator’s credibility. In this work, the focus is on the automatic classification of Hadith based on its level of authenticity. Specifically, a deep-leaning (DL) approach is employed to perform automatic classification to determine whether a specific Hadith text is Sahih, namely genuine, or Daif. To do this, a publicly available dataset of the Hadith corpus is utilised. To the best of our knowledge, no previous work has focused on the use of DL models for automatic detection and classification of Hadith categories based on the Hadith’s level of authenticity. This work also contributes to the literature by making the collected and pre-processed dataset publicly available for further research. To the best of our knowledge, there is no publicly available benchmark Hadith dataset that has been prepared for this task. The remainder of this paper is structured as follows. A review of previous natural language processing (NLP) research that focuses on Hadith science is provided in section 2. The subsequent section outlines the dataset used in this work and the experimental setup. This is followed by the methodology section, which describes our methodology in detail, explains the experimental results, and discusses error analysis. The final section summarises the main findings and highlights potential directions for future research. 2. Related Work Computational NLP research has approached Hadith science in several ways, including automatic Hadith topic classification, automatic Hadith question answering systems, and a graphing narration tree designed to ascertain how the Hadith text spread (Azmi