An innovative parallel test sheet composition approach to meet multiple assessment criteria for national tests Gwo-Jen Hwang a, * , Hui-Chun Chu a , Peng-Yeng Yin b , Ji-Yu Lin b a Department of Information and Learning Technology, National University of Tainan 33, Sec. 2, Shulin Street, Tainan City 70005, Taiwan, ROC b Department of Information Management, National Chi Nan University Pu-Li, Nan-Tou County, Taiwan 545, ROC Received 11 August 2007; received in revised form 13 October 2007; accepted 21 October 2007 Abstract The national certification tests and entrance examinations are the most important tests for proving the ability or knowl- edge level of a person. To accurately evaluate the professional skills or knowledge level, the composed test sheets must meet multiple assessment criteria such as the ratio of relevant concepts to be evaluated and the estimated testing time. Further- more, to fairly compare the knowledge levels of the persons who receive different national tests and to avoid unexpected situations due to weather, accidents, etc., a set of parallel test sheets with identical test ability need to be composed, which is almost impossible to be accomplished manually. To cope with this problem, an innovative approach is proposed to effi- ciently compositing parallel test sheets from very large item banks, while meeting multiple assessment criteria. Moreover, a computer-assisted testing system was developed and a series of experiments were conducted to show the efficiency and effectiveness of the innovative approach. Ó 2007 Elsevier Ltd. All rights reserved. Keywords: Evaluation methodologies; Public spaces and computing; Human-computer interface; Media in education; Computer-assisted testing 1. Introduction As the efficiency and effectiveness of the deployment of computer-based tests have been confirmed by many early studies, many researchers in both technical and educational fields have engaged in the development of computerized testing systems (Fan, Mak, & Shue, 1996; Olsen, Maynes, Slawson, & Ho, 1986). Some researchers have even proposed computerized adaptive testing, which uses prediction methodologies to shorten the length of the test sheets without sacrificing their precision (Wainer, 2000). 0360-1315/$ - see front matter Ó 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2007.10.006 * Corresponding author. Tel.: +886 915396558; fax: +886 6 3017001. E-mail addresses: gjhwang@mail.nutn.edu.tw (G.-J. Hwang), carolchu@mail.nutn.edu.tw (H.-C. Chu), pyyin@ncnu.edu.tw (P.-Y. Yin), s94213529@ncnu.edu.tw (J.-Y. Lin). Available online at www.sciencedirect.com Computers & Education 51 (2008) 1058–1072 www.elsevier.com/locate/compedu