Creating coupled-multiple response test items in physics and engineering for use in adaptive formative assessments Laura R´ ıos *‡ , Benjamin Lutz † , Eileen Rossman † , Christina Yee * , Dominic Trageser † , Maggie Nevrly † , Megan Philips † , Brian Self † * Physics Department California Polytechnic State University San Luis Obispo, CA, USA † Department of Mechanical Engineering California Polytechnic State University San Luis Obispo, CA, USA ‡ Corresponding author: lrios02@calpoly.edu Abstract—This Research Work-in-Progress paper presents ongoing work in engineering and physics courses to create online formative assessments. Mechanics courses are full of non- intuitive, conceptually challenging principles that are difficult to understand. In order to help students with conceptual growth, particularly if we want to develop individualized online learning modules, we first must determine student’s prior understanding. To do this, we are using coupled-multiple response (CMR) ques- tions in introductory physics (a mechanics course), engineering statics, and engineering dynamics. CMR tests are assessments that use a nuanced rubric to examine underlying reasoning elements students may have for decisions on a multiple-choice test and, as such, are uniquely situated to pinpoint specific student alternate conceptions. In this paper, we describe our research efforts to create CMR test items for use in an adaptive learning technology for targeted intervention for a student’s unique learning needs. Our analysis was able to pinpoint some reasoning patterns that students employ to solve problems. We observed reasoning patterns that yield incorrect, partially correct, and correct answers. Our burgeoning process would aid instructors in tailoring their instruction or curricular materials for specific student needs. Index Terms—formative assessment, coupled multiple re- sponses, adaptive learning, statics, dynamics, mechanics, engi- neering education research, physics education research I. I NTRODUCTION In introductory courses in physics and engineering, instruc- tors are often met with persistent student misconceptions, or robust belief systems drawn from their visceral experience with physical phenomena. Because these experiences form a student’s mental model of physical phenomena, alternate conceptions of canonical mechanics are difficult to pinpoint and even more difficult to assess and address [1], [2], [3], [4]. One way to aid instructors in using class time productively (e.g., in active-learning endeavors involving peer-assisted learning) is to diagnose student alternate conceptions ahead of time via online formative learning tools, specifically adaptive State of California Governor’s Office of Research and Planning. technologies [5], [6], [7]. In this work-in-progress, we describe our research efforts which leverage existing online learning infrastructure (Concept Warehouse) [8]. The Concept Ware- house is an online instructional and assessment tool that helps students develop a deeper understanding of relevant concepts across a range of STEM courses and topics. We begin by briefly reviewing student difficulties in statics and dynamics, in addition to our motivation for developing specifically coupled-multiple response (CMR) test items to tailor online learning tools. CMR items leverage a combination of student answers and explanations to identify specific ways of understanding, which can then be used to create targeted interventions to address robust misconceptions. Because this is a work-in-progress paper, we then go on to describe the methodology of creating data-driven CMR test items to add a level of nuance in evaluating the students that is often missing from multiple-choice and online assessments. Our focus on methods is to communicate with interested researchers how our interdisciplinary team is undertaking the creation of CMR test items in different classroom contexts. We describe those contexts and provide examples from our ongoing work. We then discuss future work on validating the items. Ultimately, we expect CMR test items will guide how and when to deploy supplemental instruction or interventions that are tailored to a student’s particular reasoning. II. BACKGROUND A. Student difficulties in engineering and physics Statics is often students’ first engineering mechanics course and typically poses a number of difficulties related to con- ceptual understanding [9]. As a result, a number of research efforts have examined both the relevant concepts within statics [10] and strategies for helping students overcome conceptual difficulties [11], [12]. One reason statics is challenging for students is the presence of threshold concepts [13], [14] and topics in the course that run counter to many students’