M. Kingston and P. Grimes (Eds.) Proceedings of the Eighth Conference on Research in Mathematics Education in Ireland (MEI 8) Validation of Task Items on a Screener for Initial Algebra Aoife OBrien 1 and Máire Ní Ríordáin 2 1 Galway-Mayo Institute of Technology and 2 University College Cork A standardised criterion referenced assessment known as a screener has been developed for initial algebra. This screener is a formative assessment which has been designed to be of use to teachers in the second-year Irish post-primary mathematics classroom. This paper outlines the design of the screener as guided by the assessment triangle and the subsequent steps in selecting and validating task items for use on the screener. There are two types of items included - selected response (SR) and constructed response (CR). Classical Test Theory (CTT) is used to analyze the results of the SR task items in terms of item difficulty, discrimination, and functioning distractors. Important construct validity evidence arises from the statistical analyses of the item responses. The results of these analyses inform the presentation and layout of the SR task items on the final draft of the screener to ensure the construct of initial algebra is operationalised fairly. Introduction Initial algebra is the period when students transition from arithmetic to algebra, an area of mathematics education known for its difficulties (Kieran, 2007). Much research and effort has been focused on the area of algebraic thinking and initial algebra to improve students’ engagement and attainment with the subject (Kieran et al., 2016). Formative assessment (FA) should now be a vital part of mathematics teaching and learning and it should form part of the “Assessment toolkit” for all classrooms (NCCA, 2019). However, it is noted in the literature that few adequate assessments are available to provide formative information on a student’s progress with algebra, but they are essential to allow timely and informed instructional decisions for teachers (Ketterlin-Geller et al., 2019). The overall aim of this study was to profile second year post-primary students’ (approx. age 14) knowledge of initial algebra and to do so an appropriate measurement instrument was required. This standardised screener developed for this study takes the form of a summative assessment (SA) to be used in a formative manner, as it can provide information to teachers about their students’ current understanding of algebra and identify any gaps in their knowledge and understanding. The focus of this paper is on the development and validation of selected response (SR) task items for the standardised screener. First the design of the screener is outlined which explains how the task items were identified and selected for inclusion. Subsequently, the methodology employed to establish construct validity using item level analysis is given. Finally, an overview of the results of item analyses are presented and the resulting changes to the screener are documented. Screener Design and Development Nichols et al., (2017) state that assessment design is a “sequence of development actions aimed at accomplishing specific goals” and assessment development is “the execution of the planned course of action” (p. 15). The assessment triangle as described by Pellegrino et Aoife OBrien and Máire Ní Ríordáin 316