Research Article The Reliability Analysis of Speaking Test in Computer-Assisted Language Learning (CALL) Environment Raja Muhammad Ishtiaq Khan , 1 Tribhuwan Kumar , 2 Ahmed Benyo , 2 Syed Farhat Jahara , 3 and Mir Mohammad Farooq Haidari 4 1 Majma’ah University, Zulfi, Saudi Arabia 2 College of Science and Humanities at Sulail, Prince Sattam Bin Abdulaziz University, Al Kharj, Saudi Arabia 3 Department of English Language and Translation, College of Sciences and Arts at Al-Asyah, Qassim University, Saudi Arabia 4 Economic Faculty and Research Deputy of Taj University, Afghanistan Correspondence should be addressed to Tribhuwan Kumar; t.kumar@psau.edu.sa and Mir Mohammad Farooq Haidari; mir.m.farooqhaidari@gmail.com Received 9 February 2022; Revised 27 February 2022; Accepted 2 March 2022; Published 15 March 2022 Academic Editor: Ehsan Rezvani Copyright © 2022 Raja Muhammad Ishtiaq Khan et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Speaking ability is regarded as among the essential aspects of language development. Oral examination appeared challenging to evaluate due to the presence of human evaluators. The speaking method depends on the test’s reliability, determined by the raters’ scores. The current study is aimed at evaluating the speaking test’s interrater reliability utilized to measure the speaking performance of Common First Year (CFY) students during remote learning. The data were obtained from 56 EFL learners using a scoring sheet and rubrics. Eight raters were responsible for rating the study. The speaking test’s reliability was estimated using quantitative data analysis. Correlation coefficients and the Bland-Altman test were employed to assess raters’ agreement. SPSS was utilized to analyze the data in this investigation. The study’s findings suggested that the speaking exam used in the CFY program during remote learning has shown some reliability on correlations and acceptable norms on the Bland-Altman test. 1. Introduction Education has changed rapidly in recent years, with a notice- able increase in e-learning, which involves teaching distantly or remotely and via online platforms [1–3]. Following the closure of all educational facilities in Saudi Arabia due to the COVID-19 breakout, an unforeseen quick transition from the typical “traditional” learning method [4, 5] to the newly e-learning supported, namely, online learning, has happened. Adedoyin and Soykan [6] argue that “without the pandemic’s emergence, our education institutions would not have been so adept at online instruction.” This suspen- sion has also impacted the testing and evaluation system, partially testing language skills. Speaking is an essential skill in learning any language. It involves interaction and group discussion, which was significantly impacted by the outbreak of COVID-19. Institutions were bound to have an online evaluation during this era. Language proficiency fosters collaboration and interac- tion among individuals from varied cultural origins in all spheres of life, education, and business in the globalized twenty-first century Kukulska-Hulme et al. [7]. Learning languages must consequently be a long-term commitment, implemented in a variety of ways to fulfill societal, profes- sional, and educational objectives, as well as individual desires and needs. English is commonly recognized as the globe’s lingua franca and the most widely spoken language [8–10]. Due to the importance and demand for the English language in the modern global era, English as a second language (ESL) students travel from all over the world to study the language [11, 12]. Consequently, significant effort has been expended in developing effective methods for learning Hindawi Education Research International Volume 2022, Article ID 8984330, 10 pages https://doi.org/10.1155/2022/8984330