STATISTICAL TOOLS TO ENHANCE PEER-REVIEW PROCESSES IN LARGE GROUPS * Joan Codina Universitat Pompeu Fabra P. Circunval·lació 8 -08003- Barcelona Josep M. Fontana Universitat Pompeu Fabra La Rambla 30-32 -08302 - Barcelona ABSTRACT The use of peer-review methods in the evaluation of students’ work is becoming increasingly popular due mainly to pedagogical considerations. This paper argues for the integration of several statistical techniques in the peer-review process that would increase the efficiency and reliability of this process by providing the teacher with tools that make the necessary supervision task less daunting and more productive. Crucially, the model proposed here makes it possible for students to be not only evaluated by the quality of their work but also by the quality of their evaluation of the work of their peers. The statistical tools are intended to minimize the need for direct intervention on the part of the teacher by establishing a ranking of reviewers according to their reliability and quickly identifying significant discrepancies in the evaluation of specific assignments. The system thus allows the teacher to be able to focus his/her attention on the subset of assignments whose evaluation has been signaled as problematic. This is especially useful in courses with large groups of students where the evaluation of assignments can be enormously tedious and labour intensive. This approach allows computers to play a more useful role in the continuous assessment of the student’s work, restricted until now mainly to certain types of activities such as cloze tests or multiple-choice quizzes whose pedagogical value and possibilities are rather limited. The work presented here is intended as a preliminary proposal before its implementation in an LMS for further testing and improvement. KEYWORDS peer-to-peer review, assessment, LMS, collaborative learning 1. INTRODUCTION Peer review is defined as “assessment of the work of others by people of equal status and power. In the context of student learning, peer assessment is used in making formal estimates of worth of other students’ work and giving and receiving feedback. Making estimates of other students’ work is sometimes known as peer marking” Murray (2003). The use of peer evaluation methods in the context of higher education is well established and there a number of studies that confirm their positive contribution to the learning process (eg see Boud 2000, McConnell, 2000, Wondrak 1993, Opern 1982). One of the main pedagogical advantages of this method of assessment is that it provides an additional opportunity for the learner to participate in the shared construction of knowledge. If properly implemented, the use of peer review techniques allows students to learn from each other, above all when they are used with activities or exercises where there is not a unique and clear cut solution. Thus, it can be argued that by reviewing the work of his/her peers, the learner is also actively revising his/her own representation of knowledge and re-evaluating it. * The research presented in this paper has been partially financed by the Multilingual eLearning in Language Engineering (MeLLANGE) project, European Union – LEONARDO DA VINCI Community Vocational Training Action Programme. IADIS International Conference on Cognition and Exploratory Learning in Digital Age (CELDA 2006) 313