Use of ICT and its relationship with performance in examinations: a comparison of the ImpaCT2 project’s research findings using pupil-level, school-level and multilevel modelling data Colin Harrison à , Eric A. Lunzer à , Peter Tymmsw, Carol Taylor Fitz-Gibbonw & Jane Restorick à à School of Education, University of Nottingham, Nottingham, UK wSchool of Education, University of Durham, Durham, UK Abstract This paper reports a previously unpublished comparative analysis of data from the ImpaCT2 study investigating the relationship between students’ performance in England on national tests and their reported use of information technology (particularly networked technology) for school work, at three age levels (11, 14 and 16), in English, Maths and Science (and in additional subjects at age 16). The new analysis compares three separate analyses of the same data set based on approximately 1100 children drawn from 27 primary and 28 sec- ondary schools: the first analysis considers the individual pupil data and looks particularly for pupil-level (and particularly gender) effects; the second analysis groups the pupils by school, and looks particularly for school-level effects; the third analysis was a more con- servative multilevel modelling analysis, carried out by an independent team at the University of Durham, which split the schools into three groups, based on overall high, medium or low level of use of information and communications technology (ICT) in each school. All three analyses reported findings based on mean achievement measured in terms of standardised residual gain scores, where each student’s progress was measured against predicted gains calculated from baseline data on reading and Maths ability gathered 2 years earlier. The results indicated broad agreement between each of the three modes of analysis, though effect sizes were slightly smaller in the case of the multilevel modelling. The findings are inter- preted in relation to case study data on ICT use in schools gathered as part of the wider ImpaCT2 investigation, but the case is also made that the common rationale for using multilevel modelling with large data sets, namely that whole class effects are more sig- nificant determinants of performance than individual effects, is less applicable in this study, because of wide variation in the use by children of ICT for school work at home. Keywords technology, assessment, primary, secondary, value-added, pedagogy Background The National Grid for Learning (NGfL) has been one of the UK Government’s key initiatives to stimulate and support the use of information and communications technology (ICT) in England to improve standards and to encourage new ways of teaching and learning (DfEE, 1998, 2002). ImpaCT2 was one of a number of Correspondence: Colin Harrison, School of Education, University of Nottingham, Jubilee Campus, Wollaton Road, Nottingham NG8 1BB, UK. Email: colin.harrison@nottingham.ac.uk Accepted: 3 August 2004 & Blackwell Publishing Ltd 2004 Journal of Computer Assisted Learning 20, pp319–337 319 Original article