A comparative analysis of the consistency and difference among teacher-assessment, student self-assessment and peer-assessment in a Web-based portfolio assessment environment for high school students Chi-Cheng Chang a, * , Kuo-Hung Tseng b , Shi-Jer Lou c a Department of Technology Application and Human Resource Development, National Taiwan Normal University, No.162, He-Ping E. Road, Section 1, Taipei 106, Taiwan, ROC b Graduate Institute of Business and Management, Meiho University, No. 23, Pingguang Rd., Neipu, Pingtung, Taiwan, ROC c Graduate Institute of Technological and Vocational Education, National Pingtung University of Technology and Science, No.1, Xue Fu Road, Lao Bei Village, Nei-Pu Township, Pingtung, Taiwan, ROC article info Article history: Received 12 February 2011 Received in revised form 28 June 2011 Accepted 4 August 2011 Keywords: Web-based portfolio Web-based portfolio assessment Self-assessment Peer-assessment abstract This study explored the consistency and difference of teacher-, student self- and peer-assessment in the context of Web-based portfolio assessment. Participants were 72 senior high school students enrolled in a computer application course. Through the assessment system, the students performed portfolio creation, inspection, self- and peer-assessment; three teachers reviewed portfolios and evaluated learning performances. There were significant differences in the results of the three assessment methods, among which teacher-raters adopted the most rigorous scoring standards, while peer-raters tended to use the most lax standards. The results of self- and teacher-assessment were discovered to be consistent; however, consistency was not discovered between self- and peer-assessment as well as peer- and teacher-assessment. In analyzing their consistency with the end-of-course examination, teacher- and self-assessment demonstrated high consistency, whereas peer-assessment showed a low level of consistency. Crown Copyright Ó 2011 Published by Elsevier Ltd. All rights reserved. 1. Introduction A learning portfolio could be considered as portfolio assessment when its primary purpose involves assessing learning outcomes. A Web- based portfolio assessment refers to a teacher’s or peers’ authentic evaluation of a student’s efforts, progress and achievements through an online portfolio assessment system platform where the learning processes and outcomes are exhibited (Chang, & Tseng, 2009a; 2009b; Barbera, 2009). Authentic assessment involves student engagement in the evaluation process by using authentic evidences of learning processes and outcomes (Barbera, 2009; Barrett, 2007). There are three most common methods used in Web-based portfolio assessment (i.e. teacher-assessment, student self-assessment and peer-assessment), with which teachers may have options to select which method(s) to adopt in order to serve pedagogical needs. In their Web-based portfolio assessment research, Chang, & Tseng (2009a; 2009b) incorporated the three methods in order to satisfy the nature of authentic assessment. Teacher-assessment is the most used approach, but the sole dependence on which might overlook students’ real thoughts. According to Barbera (2009) and Jenkins (2004), peer feedback to some extent is more influential and powerful than teacher feedback. Therefore, the integration of self- or peer-assessment will effectively increase students’ involvement, motivation and incentives. A learning portfolio, when serving as assessment tool, is characterized by the following aspects: 1) it is a dynamic, authentic and comprehensive assessment method; 2) it focuses on both the processes and outcomes of learning; 3) scoring participation spurs students to improve their overall knowledge and skills; 4) past performance is considered, which respects individual differences of students; 5) diverse and multi-dimensional evidence of learning helps to ensure reliable assessment results (Barrett, 2010; Barrett & Garrett, 2009). If the advantages of portfolio assessment are to be taken, none of the three assessment methods should be neglected. Although teacher- assessment plays an indispensable role, the authenticity of portfolio assessment is proved to be readily enhanced when using self- and * Corresponding author. Tel.: þ886 2 77343439; fax: þ886 2 23921015. E-mail addresses: samchang@ntnu.edu.tw, samchichengchang@gmail.com (C.-C. Chang), gohome8515@gmail.com (K.-H. Tseng), lousj@ms22.hinet.net (S.-J. Lou). Contents lists available at ScienceDirect Computers & Education journal homepage: www.elsevier.com/locate/compedu 0360-1315/$ – see front matter Crown Copyright Ó 2011 Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2011.08.005 Computers & Education 58 (2012) 303–320