JPII 5 (2) (2016) 268-277 Jurnal Pendidikan IPA Indonesia http://journal.unnes.ac.id/index.php/jpii VALIDATION OF SCIENCE ACHIEVEMENT TEST WITH THE RASCH MODEL P. Susongko* Pancasakti University, Tegal, Indonesia DOI: 10.15294/jpii.v5i2.7690 Accepted: August 25 th 2016. Approved: September 4 th 2016. Published: October 2016 ABSTRACT The purpose of this study was to validate the test items of science achievement, which were used as a test at Pancasakti Science Competition, in order to obtain enough valid test items with the Rasch model application. Validation model used in this research is Messick validity covering aspects such as (1) content, (2) substantive, (3) structural, (4) external and (5) consequential. To achieve these objectives, this study investigates the quality of the items that include matching items, Person-Items Folder, Person/Item Folder, Information Function Tests, Person Fit Statistic, Collapsed Deviance, Casewise Deviance, Hosmer-Lemeshow, accuracy, sensitivity, specific- ity, unidimensional, invariance, separation and DIF. The test was given in the form of multiple choice as many as 40 items consisting of 15 items of physics, 10 items of chemistry, and 15 items of biology. The participants were 85 biology students who were given 60 minutes to do the test. Item analysis was conducted by using R 3:12 Program, eRm package version 0.15-6. The study results showed that the test items of science achievement were proven valid by the application of Rasch Model. The test items have met construct validity according to Messick (1996) which includes such aspects: (1) content, (2) substantive, (3) Structural, (4) External and (5) consequential. © 2016 Science Education Study Program FMIPA UNNES Semarang Keywords: validation, Messick, Rasch model, science achievement test INTRODUCTION Measurement is one of the initial steps in the evaluation program, which is a process for determining the characteristics of a number of attributes of the students, especially their learn- ing achievement. In relation to student achieve- ment, measurement is the process of giving the numbers that are expected to demonstrate the ability of students about a subject. To perform measurements, a measuring instrument is need- ed. Measuring tool that gives information about a person’s position in the measured attribute.A good measurement tool would ensure valid and reliable results in order to accurately quantify the ability of students. Validity indicates the degree of evidence and theory that supports the inter- pretation of test scores as the intended use of the test (APA, 1999). Thus, validation is basically a collection of evidence to provide the scientific basis of test scores interpretation. The valida- tion process used by the association of education and psychology in the United States (AERA and APA) can be divided into five types: (1) evidence based on the content of the test, (2) evidence based on the response, (3) evidence based on the internal structure, (4 ) evidence based on the link- ages with other variables, and (5) evidence based on the consequences of the test (APA, 1999). Re- liability is the coefficient which shows the degree of regularity or consistency of the measurement results of a test (Mardapi, 2012). Therefore, the reliability is not related to the test, but rather to the level of errors in the results of a test in the form of a score. Messick (1996) argues that validity is a *Alamat korespondensi: Email: kusumatirto@gmail.com