Evaluating the applicability of Approaches to Teaching Inventory (ATI) in higher engineering education research and development Johanna Naukkarinen School of Energy Systems LUT University Lappeenranta, FINLAND johanna.naukkarinen@lut.fi Sonja Niiranen Faculty of Education and Culture Tampere University Tampere, FINLAND sonja.niiranen@tuni.fi Abstract— This Work-in-Progress research paper presents the rationale and a plan to evaluate the applicability, validity, and reliability of Approaches to Teaching Inventory (ATI) in higher engineering education research and development. The ATI is a widely used, but also criticized, tool in higher education research in different disciplines, including engineering. This paper discusses the validity and reliability of the ATI as an instrument in general, and the applicability of the ATI in the discipline of engineering. A research plan to scrutinize the validity, reliability, and applicability of the ATI in engineering education is proposed, and the FIE community is invited to offer their expertise in developing and executing the plan. Keywords—Approaches to Teaching Inventory, content validity, construct validity, reliability, engineering education research I. INTRODUCTION A. Development of ATI Approach to Teaching Inventory (ATI) is an instrument developed in the 1990s by Keith Trigwell and Michael Prosser [1] to measure the “key aspects of variation in approaches to university teaching” in order to relate them to the students’ approaches to learning. The creation of the inventory originated from a phenomenographical study, where five qualitatively different approaches to teaching were discovered based on interviews of 24 university science teachers. The five categories exhibited four different intentions behind teaching (information transmission, concept acquisition, conceptual development, and conceptual change) and three different teaching strategies (teacher-focused, student-teacher interaction, and student- focused). During the construction of the inventory, three of the original five categories were dismissed resulting in a survey with 16 statements scoring the respondents on two scales: the Information transmission/Teacher-focused (ITTF) and the Conceptual change/Student-focused (CCSF) approach scale. Both scales were composed of the subscales of intention and strategy. B. Current use of ATI Currently, the ATI is a widely used tool in higher education research and development. In addition to the original intent of relating the teachers’ teaching approaches and the students’ learning approaches, it has been used, for example, to measure and explain teachers’ repertoire of teaching methods [2], to evaluate the impact of pedagogical training on teachers’ conceptions [3], to study the development of teacher identity [4], and to understand the variety of teaching conceptions [5]. Although the instrument was originally developed in the contexts of science teaching, it has been applied in various disciplinary contexts, and also used to understand the disciplinary variation in teachers’ thinking [5–8]. In engineering, the ATI has been applied at least to develop training in university pedagogy [9], to understand the diversity of academic community [10], to explore the impact of pedagogical training [11], to examine the relationship between beliefs and practices [12, 13], and to examine university students’ approaches to learning [14]. C. Applicability of the ATI to engineering education research However, the applicability of the instrument to engineering education remains somewhat unclear. The validity of the instrument has been challenged [15], and the development process and applicability have been criticized [16]. The results regarding disciplinary variation are contradictory, with some studies showing evidence on disciplinary differences [6, 7], and some not [8]. In addition, many of these studies included no engineering teachers, or the studied disciplines were clustered in such a way that it was not clear whether engineering was among the disciplines under study. Hence, it is difficult to evaluate the generalizability of the findings from the ATI studies in other disciplines to engineering. First, this paper discusses the validity and reliability of the ATI based on previous research and draws attention to some important issues. After that, the paper presents a plan to empirically investigate the issue further. As a work-in- progress research paper, it also invites the engineering education research community to comment on the assumptions on which the research scheme is based, and suggest improvements in the research plan. II. VALIDITY AND RELIABILITY OF ATI Validity of a research instrument is a multifaceted issue, and there appears not to be an agreement on a common conceptualization or a theory of the subject. This paper follows the footsteps of Cronbach and Meehl [17] and discusses validity from the viewpoints of content validity, criterion validity, and construct validity. Reliability is viewed mainly from the aspect of internal consistency. A. Evaluating different types of validity Criterion validity is evaluated by obtaining a criterion measure independent of the instrument applied to the same subjects and computing a correlation between them [17]. If the independent criterion measure is obtained after the use of the instrument, the validity is called predictive, and if the