1 Using Psychometrics to Advance Assessment in Mathematics Education: The Does it Work Project 1 Chandra Hawley Orrill University of Massachusetts Dartmouth The Does it Work: Building Methods for Understanding the Effects of Professional Development (DiW) project is focused on understanding what teachers learn in professional development, how they use that learning in their classrooms, and whether the teachers’ learning has impact on student performance. To this end, DiW has developed an item pool to help assess teacher learning from the professional development experience. How are theoretical perspectives on knowledge reflected in the assessment? The DiW project, and the assessment we developed for it, uses two perspectives in the design of the instrument that are unique. First, we focus on fine-grained understandings of teacher knowledge and, second, we applied a systematic mapping of the domain as the organizational structure for thinking about both the content we were interested in and what it means to know this mathematics. Further, we moved beyond IRT models based solely on scaling and used the mixture Rasch model that classifies and scales examinees. While there are a number of studies that have considered teacher knowledge, too often, those studies focus on deficits in understandings. For example, Ma’s (1999) work compares Chinese teachers’ knowledge to American teachers’ knowledge to determine the places where the knowledge is lacking about particular concepts. Similarly, in Borko et al. (1992), the authors followed a single preservice teacher as she attempted to teach fraction division. They found that she lacked the conceptual understandings to provide answers to students’ questions about why the algorithmic approach to solving such problems works. In contrast, research on students’ reasoning about rational numbers has long been focused on understanding how knowledge develops over time and how various schemes support the development of more robust understandings of these concepts (e.g., Mack, 1990, 1995, Olive, 1999; Steffe, 2001, 2003; Tzur, 1999, 2000). In this work, there is often, though not always, a developmental aspect but, more important for our study, there is an effort to gauge what the students do understand and where that understanding breaks down. To this end, the researchers have tended to develop tasks and methods that help to better delineate what students do and do not understand about fine-grained mathematics rather than clustering topics together into a coarser division of the content. 1 Presented as part of Izsák, A., Confrey, J. E., Orrill, C., McCrory, R., & Kelly, A. (April, 2010). Using psychometrics to advance assessment in mathematics education. Symposium presented at the Research Presession of the 88 th Annual Meeting of the National Council of Teachers of Mathematics: San Diego. The work reported here is supported by the National Science Foundation under grant number REC-0633975. The results reported here are the opinions of the author and may not reflect those of NSF. Special thanks goes to the Does it Work research team for their support in every aspect of the reported work.