Algorithmic Driven Decision-Making Systems in Education Analyzing Bias from the Sociocultural Perspective Federico Ferrero Universidad Nacional de Córdoba Stellae Research Group Córdoba, Argentina federicojferrero@gmail.com Adriana Gewerc Universidad de Santiago de Compostela Stellae Research Group Santiago de Compostela, Spain adriana.gewerc@gmail.com Abstract— This article analyzes biases in algorithmic driven decision-making systems in education considering some contributions from the Activity Theory of sociocultural tradition. First, it is identified how the sources of biases (theoretical, methodological, by interpretation, by decontextualization and by data training) are distributed in the elements of the analytical unit as well as in the systemic time. Consequently, the algorithm is not treated as a mediating artifact biased in itself, but biases are reflected in it and are linked to the practices carried out by the subjects involved in the systemic reality studied. Second, it is presented the results of a Systematic Literature Review that allows us to explore the ways in which the Journal of Learning Analytics community approaches the subject of biases according to the bias-sources classification previously constructed. Index Terms— bias, algorithms, decision-making process, Artificial Intelligence, educational contexts, Activity Theory. I. INTRODUCTION The proliferation of automated decision-making processes in different areas of our societies is becoming more and more evident and the governments arranged with algorithms seem to be unavoidable phenomena [1] [2] [3]. As it is increased the computer capacity and the accumulation and constant updating of large quantities of data coming from the actions of users in virtual surroundings; the already known Artificial Intelligence techniques take on a determined new boost [4]. Within this “datafication” context [5] [6] [7] these systems use algorithms to assess situations and make decisions covering a large variety of areas that have an impact at private individual level. In particular, these automated systems are present in business, since it was there where the so-called “predictive risk analytics” [8] first found an applicability. However, algorithms are used more and more to make decisions in the area of health, in justice and in the organisation of prisons, in urban design and its mapping, in government and bureaucratic systems [9] [10] [11] [12] and, obviously, in the pedagogical field. In this field, algorithms are usually used to predict performances, choose students and assess teachers, or to develop “Intelligent Mentoring Systems” or “Adaptive Learning Systems” to recommend lessons and contents to pupils, amongst others [13] [14] [15] [16]. In the field of education, a growing interest is becoming evident, to show how algorithmic-driven decision-making is applied in specific cases, in which they usually interact with traditional decision-making systems. The use of virtually originated educational data no longer implies only the description of the performances and practices of students, teachers, and administrators, but also the search for automation in decision-making at different levels (national education systems, institutions and even, sometimes, the classes as such). This situation draws the attention on the centrality of the assessment, a classic pedagogical concept, that acquires new connotations and manifests itself by means of new procedures and mechanisms. Around this topic, it is recognized that the inclusion of this kind of systems is crossed by different controversies that call for new types of exchanges between different professionals, trained in different logics (predominantly specialists in pedagogy, in computer sciences, and statistics). Within this framework, this article deals with the biases in automated decision-making systems in educational contexts from a pedagogical perspective well-stocked in the Sociocultural Approach to Learning Theories. The aim is to produce a theory-oriented analysis from the field of education that is able to engage in dialogue with specialists from other fields. Although analyzes have recently begun to take place on this type of decision-making system in education [4] [13] [14], there are little meta-analytical studies on bias in such systems addressed from the Sociocultural Approach. Therefore, two intentions are brought up to this end: 1) to present a conceptualisation and classification of biases in algorithmic- driven decision-making systems, considering some 166 2019 XIV Latin American Conference on Learning Technologies (LACLO) 978-1-7281-4286-9/19/$31.00 ©2019 IEEE DOI 10.1109/LACLO49268.2019.00038 Authorized licensed use limited to: Univ of Texas at Dallas. Downloaded on November 21,2021 at 13:07:58 UTC from IEEE Xplore. Restrictions apply.