Edelweiss Applied Science and Technology ISSN: 2576-8484 Vol. 8, No. 6, 9328-9344 2024 Publisher: Learning Gate DOI: 10.55214/25768484.v8i6.3996 © 2024 by the authors; licensee Learning Gate © 2024 by the authors; licensee Learning Gate * Correspondence: paulo.etchegaray@uautonoma.cl Learning to assess foreign language listening: Analyzing the assessment items designed by Chilean pre-service EFL teachers and the cognitive load they experience in this process Tania Tagle 1 , Paulo Etchegaray 2* , Claudio Díaz 3 , Mabel Ortiz 4 , Lucía Ramos 5 , Marcela Quintana 6 1 Universidad Católica de Temuco, Chile; ttagle@uct.cl (T.T.). 2 Universidad Autónoma de Chile; paulo.etchegaray@uautonoma.cl (P.E.). 3 Universidad de Concepción, Chile; claudiodiaz@udec.cl (C.D.). 4 Universidad Católica de la Santísima Concepción, Chile; mortiz@ucsc.cl (M.O.). 5 Universidad Católica del Norte, Chile; luramos@ucn.cl (L.R.). 6 Universidad Arturo Prat, Chile; maquinta@unap.cl (M.Q.). Abstract: The general objective of this study is to analyze the types of items designed by Chilean pre- service EFL teachers to assess listening comprehension skills in English and the cognitive load they experience during the test design process. This study employs a mixed-method design, including 125 Chilean pre-service teachers of English in their third, fourth, and fifth year of professional preparation. As data collection techniques, the participants created a listening comprehension test for secondary education English language learners. Moreover, they completed the Cognitive Load Scale (Leppink et al., 2013). The findings indicate that all pre-service EFL teachers create close and open-ended questions to evaluate students' listening comprehension, focusing on identifying specific information. Fourth and fifth-year pre-service teachers design close and open-ended questions to identify general and specific information, encompassing higher levels of comprehension. On the other hand, regarding the cognitive load experienced by the participants in the test design, they exhibited moderate intrinsic cognitive load, demonstrating that the task was reasonably challenging. The extraneous load was generally low, suggesting minimal external distractions. Notably, the germane load was high across all groups, indicating that the task effectively promoted learning and competency development. Pre-service language teacher education programs should consider these findings to enhance listening assessment practices. Incorporating cognitive load theory can assist teacher candidates in building both pedagogical and disciplinary teaching competencies. Keywords: Cognitive load theory, Evaluation, Language assessment, Listening skills, Pre-service teacher education, Test design. 1. Introduction Following communicative practices, English language teaching and learning practices should focus on developing receptive (reading and listening) and productive (speaking and writing) language skills (Ashoori Tootkaboni, 2019). Implementing communicative approaches aids students in enhancing their abilities to understand and produce information in a foreign language (Porto, 2018). The evidence suggests that successful assessment practices could aid students in learning critical skills as teachers of English apply different assessment practices to collect data regarding students' learning progress (Fitriyah & Jannah, 2021; Fröjdendahl, 2018). This helps education professionals evaluate their teaching and make proper decisions.