Using Static Analysis Tools to Assist Student Project Evaluation Arthur-Jozsef Molnar arthur@cs.ubbcluj.ro BabeşśBolyai University Cluj-Napoca, Romania Simona Motogna motogna@cs.ubbcluj.ro BabeşśBolyai University Cluj-Napoca, Romania Cristina Vlad vcis1860@scs.ubbcluj.ro BabeşśBolyai University Cluj-Napoca, Romania ABSTRACT Code review and static analysis tools are acknowledged as impor- tant instruments in software quality control and are used in the industry on a daily basis. In this exploratory study we examine how a well-known static analysis tool can be employed to assess the quality of student solutions to coding assignments. We examine all student solutions submitted to fulfll coding assignments re- quired as part of an introductory programming course taught using Python. We show how teaching staf can evaluate the progress of individual students and how coding mistakes common to many students can be highlighted. We also show how teaching staf can improve their own understanding of perceived assignment com- plexity by evaluating the aggregate quality of student submitted source code. CCS CONCEPTS · Social and professional topics → Student assessment. KEYWORDS static analysis, student evaluation, Computer Science education ACM Reference Format: Arthur-Jozsef Molnar, Simona Motogna, and Cristina Vlad. 2020. Using Static Analysis Tools to Assist Student Project Evaluation. In Proceedings of the 2nd ACM SIGSOFT International Workshop on Education through Ad- vanced Software Engineering and Artifcial Intelligence (EASEAI ’20), No- vember 9, 2020, Virtual, USA. ACM, New York, NY, USA, 6 pages. https: //doi.org/10.1145/3412453.3423195 1 INTRODUCTION We know that source code quality has an important infuence on the software development process as a whole. While experience plays a central role in writing quality code, at the same time the principles of code quality should be taught early on. This leads to many introductory programming courses also addressing issues related to code quality in the form of observing coding guidelines, following a set of established best practices, and even adopting methodologies such as feature-driven or test-driven development. Code review and static analysis tools support this approach by allowing automatic detection of many types of quality defciencies such as software defects, security vulnerabilities, or the breaking of Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proft or commercial advantage and that copies bear this notice and the full citation on the frst page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specifc permission and/or a fee. Request permissions from permissions@acm.org. EASEAI ’20, November 9, 2020, Virtual, USA © 2020 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-8102-4/20/11. . . $15.00 https://doi.org/10.1145/3412453.3423195 best practices in early development phases. These tools are widely used in the industry and their efciency in development and main- tenance has been proven to be signifcant, with existing research [14] showing average defect detection of up to 55% to 60%. How- ever, the usage of these tools in an educational setting is usually limited to quality control as a software engineering process, or within coursework dedicated to software quality, which is usually taught at a more advanced level. As such, the main objective of our work is to "assist teaching staf in monitoring the learning evolution of students in a programming language course". As many academic courses touch upon the most important issues of programming style and best practices, we be- lieve that special attention should be dedicated to corresponding learning outcomes and the way they are assessed. At the same time, evaluating the quality of student code is a tedious and time consuming activity, reasons for which it is often overlooked. This can be changed through automation, which brings additional ad- vantages such as the ability to track student progress through all completed assignments, compare student performance for a certain assignment as well as build an overall assessment of the student projects themselves, and improve the understanding teaching staf have about the aspects students found most difcult to address. We study how our main objective can be fulflled through an experience report that covers all student solutions to the coding assignments required within the Fundamentals of Programming introductory course that is taught using Python 3 and is attended by 200 frst-year undergraduate students in computer science from the Babeş-Bolyai University in Cluj-Napoca, Romania. 2 TOOLS FOR STATIC ANALYSIS Static code analysis allows developers to gain important insight into source code without having to run it. Its benefts are the subject of notable recent studies, mainly caused by the growth of static code analysis tools [10] and their tight link with software quality through implementation of known quality models such as SQALE [12] and QMOOD [3]. Current static analysis tools are considered to belong to the third generation [11], with the major shift in the fact that analysis is performed on the target program’s abstract syntax tree. Focusing on the logic of the code allows improved tool efciency in detecting existing issues. Most of the widely used tools are available for several programming languages and can be easily integrated into development environments in the form of plug-ins such as those for Pylint [16] and SonarQube [20]. Static analysis tools such as SonarQube were also used to characterize quality of large-scale code bases in recent research targeting software quality [8, 15]. The tight integration between well-known software quality models and static analysis tools have helped promote increased formalism in the study of software quality [1]. Boehm et al. [6] have shown that 7