An Architecture and Data Model to Process Multimodal Evidence of Learning Shashi Kant Shankar 1(B ) , Adolfo Ruiz-Calleja 2 , Luis P. Prieto 1 , Mar´ ıa Jes´ us Rodr´ ıguez-Triana 1 , and Pankaj Chejara 1 1 Tallinn University, Tallinn, Estonia {shashik,lprisan,mjrt,pankajch}@tlu.ee 2 GSIC-EMIC Group, University of Valladolid, Valladolid, Spain adolfo@gsic.uva.es Abstract. In learning situations that do not occur exclusively online, the analysis of multimodal evidence can help multiple stakeholders to better understand the learning process and the environment where it occurs. However, Multimodal Learning Analytics (MMLA) solutions are often not directly applicable outside the specific data gathering setup and conditions they were developed for. This paper focuses specifically on authentic situations where MMLA solutions are used by multiple stakeholders (e.g., teachers and researchers). In this paper, we propose an architecture to process multimodal evidence of learning taking into account the situation’s contextual information. Our adapter-based archi- tecture supports the preparation, organisation, and fusion of multimodal evidence, and is designed to be reusable in different learning situa- tions. Moreover, to structure and organise such contextual information, a data model is proposed. Finally, to evaluate the architecture and the data model, we apply them to four authentic learning situations where multimodal learning data was collected collaboratively by teachers and researchers. Keywords: MMLA · Architecture · Data model · Multimodal Learning Analytics 1 Introduction There has been an explosive growth in the use of Learning Analytics (LA) to support evidence-based decision making in learning and teaching practice, tar- geting multiple stakeholders (e.g., teachers, students or researchers) [13]. To support such decision making, most LA solutions use learning evidence from the digital space, such as logs from Learning Management Systems (LMSs) [15]. Such emphasis on digital traces, however, provides only a partial view of the learning process and the environment where it occurs. To overcome this issue, Multimodal Learning Analytics (MMLA) [12] enables the data collection from digital as well as physical spaces, using multiple modalities [16] to understand the learning context (e.g., adding sensors or observations to the logs). c  Springer Nature Switzerland AG 2019 M. A. Herzog et al. (Eds.): ICWL 2019, LNCS 11841, pp. 72–83, 2019. https://doi.org/10.1007/978-3-030-35758-0_7