Quantitative Conceptual Model Analysis for Evaluating Simple Class Diagrams made by Novices Mizue Kayama, Shinpei Ogata, David K. Asano & Masami Hashimoto Shinshu University {kayama, ogata, david, hasimoto}@cs.shinshu-u.ac.jp Abstract. In this paper, we aim to propose criteria for evaluating conceptual modeling errors made by university freshmen. We quantitatively analyzed class diagrams made by novice learners. Based on the results of three types of exper- iments, we propose 12 criteria, which are divided into 4 types, for evaluating class diagrams made by novices. Keywords: conceptual modeling, class diagram, criteria, quantitative analysis. 1 Introduction These days, educational methods or learning courses related to conceptual modeling have been explored in many educational institutes, academic conferences and aca- demic journals [1-3]. The learners who were subjects of previous research were mainly third or fourth-year undergraduate students, graduate students and/or young engineers in computer science (CS). They already had finished many specialized classes related to programming, object-oriented analysis and design, databases and so on. Especially, almost all of the students were in a CS program as their major[4,5]. A few studies whose subjects had little prerequisite knowledge, for example, high school students (pre-university), undergraduate students in non-CS programs and CS freshmen in their first semester, also reported their teaching experience or teaching methods related to modeling education [6-8]. However, quantitative evaluation re- sults were not shown in their reports. Especially for students in pre-university, Non- CS and CS freshmen, there are no quantitative studies about model based thinking and subsequent curricula of conceptual modeling. In this paper, we aim to propose criteria for evaluating conceptual modeling er- rors made by university freshmen. To achieve this research goal, we quantitatively analyzed class diagrams made by students. During this analysis, we asked ourselves “What kind of criteria are suitable for novice learners when they create conceptual models? ” and “Are there any differences between the scores of novice learners with and without programming knowledge? ” There are two differences between previous research and our research. The first difference is our subjects. We focus on universi- ty freshmen and pre-university students, who have not taken any kind of CS specific courses. The other difference is the empirical and continual research method. We have been engaged in this research since 2010 [9,10]. Joint Proceedings of EduSymp 2016 and OSS4MDE 2016 Page 6 Copyright © 2016 held by the author(s)