Capstone Project: Fair, Just and Accountable Assessment Vivienne Farrell Faculty of ICT Swinburne University Melbourne, Australia +61 3 92145000 vfarrell@swin. edu.au Gilbert Ravalli Faculty of ICT Swinburne University Melbourne, Australia +61 3 92145000 gravalli@swin. edu.au Graham Farrell Faculty of ICT Swinburne University Melbourne, Australia +61 3 92145000 gfarrell@swin. edu.au Paul Kindler Faculty of ICT Swinburne University Melbourne, Australia +61 3 92145000 pkindler@swin. edu.au David Hall Faculty of ICT Swinburne University Melbourne, Australia +61 3 92145000 djhall@swin. edu.au ABSTRACT Fair, just and accountable assessment for the individual student in the capstone project requires establishing criteria beyond the final project output to meet the objectives of the capstone project experience. Current research is limited and requires solutions that are not subjective, enable formative and summative assessment with student input and are not onerous on supervisors and/or students. This paper draws on input from highly experienced academics supervising capstone projects in ICT, critically reviews recent publications pertaining to capstone project assessment and current capstone students opinions. The outcome of this input led to the development, trial and evaluation of a toolkit to assist supervisors in both formative and summative assessment of capstone projects. Categories and Subject Descriptors K.3 COMPUTERS AND EDUCATION Keywords Capstone, Project, Assessment, TCS, Peer, Group, Team, IT 1. INTRODUCTION The capstone project experience is the culmination of a student’s knowledge and skills in preparation for impending integration into the workplace. It not only offers students the opportunity to piece together their academic and technical learning but also to develop and practice a maturity towards team work, communication, client liaison, role playing, ethics, peer assessment and understanding the impact of the integration of their project on the wider community. In offering all these benefits there are identifiable challenges of assessing the individual that have been a long-standing issue for instructors and consequently students. In order to evaluate the individual within the group it is essential to provide assessment that is not only fair and just for the individual but also accountable. Individual assessment for the capstone project or in general any team project has historically created a situation where subjective assessment is required to differentiate individual student achievements. Clear [1]identified the “3Ps” in an IT disciplined capstone projects as pertaining to Product, Process and Progression. Clear [1] recommends Product assessment (scope, expectations and outcomes) to be assessed objectively; Process (portfolio, quality management, communication etc) to be assessed objectively; Progression (professionalism, initiative, perseverance and responsibility) to be assessed subjectively, with contributions from reflective reports, evidence of improvement and “observations and opinions of supervisors..” While Clear [1] is comfortable with this subjective assessment, markers often find it difficult to justify the final grade. This is exacerbated in capstone projects where the student’s individual marks can vary from a Distinction to a Fail, a situation that requires justification. The motivation for introducing a system that identified the individual student contribution was to address the issue of “free- riders” identified by Aggarwal and O’Brien [2]. Latu [3]in their study interviewed students to identify their preferences when considering the group work grading. They found that 94% of their students wanted to identify “free-riders”, defined as “an individual working in a group setting fails to contribute his or her fair share.” [2]. In this study (79%) felt that individual grades were preferable to group marks. The scope of this research has been restricted to capstone projects where students undertake a real world project within the ICT industry. We restrict the student group size from 3 to 5. The aim of this research is to design and develop a system with supporting documentation that will enable a fair allocation of grades to individuals undertaking group work. This system is to ensure that the assessment considers the main objectives of the capstone project experience and not just the final project outcome as discussed earlier. This paper considers the existing literature and present practices of assessment methods for capstone projects. Next it discusses the outcomes from focus groups held with experienced capstone project supervisors, representative student focus groups and surveys (pre, during and post the capstone experience). From the literature review, focus groups and surveys we next derive the criteria to optimize assessment practices. We then report on the designing and development of a manageable, verifiable Team Contribution System (TCS) as a solution to meet our derived criteria. The required documentation to support the TCS is then developed to assist capstone project assessors. Finally we discuss identified further work. 2. DEVELOPMENT PROCESS OF THE TCS In the initial stages of this research discussions were held in the form of a focus group with academics experienced with team project supervision. The brief was to set the parameters of the study, identify issues with team project assessment, identify desired outcomes from studies and develop alternative assessment methods. Further, a preliminary literature review of capstone project assessment (CPA) research was carried out by drawing on recent studies from prominent journals and conference proceedings. Our primary focus was on identifying novel, original and grounded studies as opposed to adaptations and extensions of existing research. These studies were subsequently Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. ITiCSE’12, July 3–5, 2012, Haifa, Israel. Copyright 2012 ACM 978-1-4503- 1246-2/12/07...$10.00.