Citation: Belda-Medina, J.; Calvo-Ferrer, J.R. Using Chatbots as AI Conversational Partners in Language Learning. Appl. Sci. 2022, 12, 8427. https://doi.org/10.3390/ app12178427 Academic Editors: Tao Xie and Ming Liu Received: 31 July 2022 Accepted: 22 August 2022 Published: 24 August 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affil- iations. Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). applied sciences Article Using Chatbots as AI Conversational Partners in Language Learning Jose Belda-Medina * and José Ramón Calvo-Ferrer Digital Language Learning (DL2) Research Group, University of Alicante, 03690 Alicante, Spain * Correspondence: jr.belda@ua.es; Tel.: +34-965-909-438 Abstract: Recent advances in Artificial Intelligence (AI) and machine learning have paved the way for the increasing adoption of chatbots in language learning. Research published to date has mostly focused on chatbot accuracy and chatbot–human communication from students’ or in- service teachers’ perspectives. This study aims to examine the knowledge, level of satisfaction and perceptions concerning the integration of conversational AI in language learning among future educators. In this mixed method research based on convenience sampling, 176 undergraduates from two educational settings, Spain (n = 115) and Poland (n = 61), interacted autonomously with three conversational agents (Replika, Kuki, Wysa) over a four-week period. A learning module about Artificial Intelligence and language learning was specifically designed for this research, including an ad hoc model named the Chatbot–Human Interaction Satisfaction Model (CHISM), which was used by teacher candidates to evaluate different linguistic and technological features of the three conversational agents. Quantitative and qualitative data were gathered through a pre-post-survey based on the CHISM and the TAM2 (technology acceptance) models and a template analysis (TA), and analyzed through IBM SPSS 22 and QDA Miner software. The analysis yielded positive results regarding perceptions concerning the integration of conversational agents in language learning, particularly in relation to perceived ease of use (PeU) and attitudes (AT), but the scores for behavioral intention (BI) were more moderate. The findings also unveiled some gender-related differences regarding participants’ satisfaction with chatbot design and topics of interaction. Keywords: chatbots; intelligent conversational agents; language learning; pre-service teachers; perceptions and satisfaction 1. Introduction Nowadays, there is a growing interest in Artificial Intelligence (AI) and chatbots, which have been widely adopted in different areas such as e-commerce, healthcare and education [1,2]. Chatbot technology has rapidly evolved over the last decades, partly thanks to modern advances in Natural Language Processing (NLP) and machine learning [3,4]. The history of chatbots can be traced back to the 1950s when Alan Touring formulated his complex question ‘Can machines think?’, published as an article in Computing Machinery and Intelligence (1950). Since then, a good amount of chatbots have emerged such as Eliza (1966), Parry (1972), Racter (1983), Jabberwacky (1988) and A.L.I.C.E. (1995), some of them still in use today. These chatbots were originally text-based and preset, based on Q&A scripts, so their responses were considered predictable, and the interaction was not perceived as natural by human standards. However, modern chatbots have incorporated new functionalities such as R&S tech- nologies (voice recognition and synthesis), customized interaction, integration with third- party apps, omnichannel deployment, context-awareness and multi-turn capability [3,5,6]. As a result, there is today a wide range of chatbots integrated in all shapes and forms into different electronic devices, programs and applications, for example messaging apps (Whatsapp, Telegram, Kik, Slack), video games and gaming platforms (Xbox, Roblox) and Appl. Sci. 2022, 12, 8427. https://doi.org/10.3390/app12178427 https://www.mdpi.com/journal/applsci