JPII 10 (4) (2021) 571-584 Jurnal Pendidikan IPA Indonesia http://journal.unnes.ac.id/index.php/jpii EXAMINING THE STEM-SCIENCE ACHIEVEMENT TEST (SSAT) USING RASCH DICHOTOMOUS MEASUREMENT MODEL J. Jamaludin 1 , Y. F. Lay* 2 , C. H. Khoo 3 , A. S. Y. Leong 4 1,2 Universiti Malaysia Sabah, Sabah, Malaysia 3,4 Institut Pendidikan Guru Kampus Kent, Tuaran, Malaysia DOI: 10.15294/jpii.v10i4.29405 Accepted: March 8 th 2021. Approved: December 27 th 2021. Published: December 31 st 2021 ABSTRACT The main purpose of this study is to develop a valid and reliable instrument to measure the STEM-Science achievement of primary school students in Malaysia. Six Year 4 Science topics (Scientific Skills, Life Processes of Human, Properties of Materials, Measurement, Solar System, Importance of Technologies in Life) and Six Year 5 Science topics (Rules and Regulation in Science Lab, Life Processes of Plants, Acids and alkali, Electricity, Earth and Space Science, Technology and Sustainable Life) have been included in the development of the STEM- Science Achievement Test (SSAT). 226 Year 4 and 226 Year 5 primary school students in Sabah responded to the developed instrument to test their STEM-Science knowledge. The Rasch dichotomous measurement model approach was used to evaluate the validity and reliability of the SSAT. The validity assessed the Point-Measure Correlation (PTMEA CORR), Principal Component Analysis of Residuals (PCAR), as well as Infit and Outfit Mean Squares (MNSQ). In terms of reliability, Cronbach’s alpha, item reliability, and item separation index were analysed. The analysis results revealed the presence of unidimensionality for both objective and subjective items. For objective items, the reported values for Cronbach’s Alpha are .81 and .83; item reliability are .95 and .95; item separation are 4.21 and 4.25 for Year 4 and Year 5 students, respectively. Standardised residual correlations for Year 4 and Year 5 subjective items also showed satisfactory values. The assessment using Rasch measurement model has proven that SSAT is a valid and reliable instrument to measure Malaysian primary students’ STEM- Science-related knowledge. © 2021 Science Education Study Program FMIPA UNNES Semarang Keywords: validation; Rasch measurement model; science achievement test *Correspondence Address E-mail: layyf@ums.edu.my INTRODUCTION Validity is considered by numerous experts to be critical for any effective assessment (Bond & Fox, 2015). There have been many definitions of what validation of a test entails. For some, a valid test is simply one that measures what it was designed to measure. However, in order to eva- luate the quality of these questions, a discussion of reliability is essential (Reardon et al., 2021). Reliability is the degree to which an instrument consistently measures the ability of an individual or group (Morales, 2009; Taber, 2018). Lamen- tably, not all countries evaluate the validity and reliability of their achievement test instrument. Hence, because of that, the use of poorly desig- ned achievement test affects students’ interest and achievement (Osadebe, 2015). Science is considered as a core subject in school, and Malaysia is formulating its education policy to improve the system based on research findings by experts and consequently improve student achievements in science (Sumintono, 2017). According to Bhagat & Baliya (2016), an achievement test is a tool for teachers to measure the development of skills or knowledge proficien- cy (Anwer, 2019) of an individual into a certain topic that has been taught. Apart from that, a test design can determine the strengths and weaknes-