Visual vs. Textual Programming Languages in CS0.5 Comparing Student Learning with and Student Perception of RAPTOR and Python Joel Coffman United States Air Force Academy Department of Computer and Cyber Sciences joel.coffman@usafa.edu Adrian A. de Freitas United States Air Force Academy Department of Computer and Cyber Sciences adrian.defreitas@usafa.edu Justin M. Hill United States Air Force Academy Department of Computer and Cyber Sciences justin.hill@usafa.edu Troy Weingart United States Air Force Academy Department of Computer and Cyber Sciences troy.weingart@usafa.edu ABSTRACT Much debate surrounds the choice of programming language for teaching computer science. Our institution’s replacement of a visual programming language (RAPTOR) with a textual programming lan- guage (Python) provided a novel opportunity to explore the impacts of the programming language on students’ learning and percep- tion of programming. We conducted a randomized comparative study that involved 1083 students who took our introductory com- puting course in the 2019–2020 academic year. A unique aspect of our work stems from our course being a general education re- quirement; thus, our study includes students with a wide variety of backgrounds and majors. This report presents a comparison of student performance in each version of the course, including the impact of the programming language on underrepresented groups, and provides a summary of student feedback. Our results show that students in our introductory course performed similarly overall, but overwhelmingly perceived Python to be more valuable. CCS CONCEPTS • Social and professional topics → Computer science educa- tion; Computational thinking; Computing literacy; • Software and its engineering → Visual languages; Scripting languages. KEYWORDS visual programming languages, textual programming languages, RAPTOR, Python, introduction to computing ACM Reference Format: Joel Coffman, Adrian A. de Freitas, Justin M. Hill, and Troy Weingart. 2023. Visual vs. Textual Programming Languages in CS0.5: Comparing Student Learning with and Student Perception of RAPTOR and Python. In Proceed- ings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2023), March 15–18, 2023, Toronto, ON, Canada. ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/3545945.3569722 This paper is authored by an employee(s) of the United States Government and is in the public domain. Non-exclusive copying or redistribution is allowed, provided that the article citation is given and the authors and agency are clearly identified as its source. SIGCSE 2023, March 15–18, 2023, Toronto, ON, Canada. 2023. ACM ISBN 978-1-4503-9431-4/23/03. https://doi.org/10.1145/3545945.3569722 1 INTRODUCTION For many students pursuing post-secondary education, an intro- ductory computing course is the only formal exposure to computer science or programming. Surprisingly, little empirical research un- dergirds which programming language (or, more generally speak- ing, programming paradigm) is most appropriate in the context of an introductory programming course [28]. Visual programming environments are known to lower the barrier to entry and increase engagement among historically underrepresented groups in com- puting [21], yet research also shows that students find visual lan- guages to be inauthentic compared to their textual counterparts [34]. Consequently, using a visual language to teach programming may diminish the societal relevance and perceived usefulness of pro- gramming, two factors that are important to engaging and retaining students from historically underrepresented groups [11, 29]. In the 2019–2020 academic year, we had the opportunity to study the impact of the programming language in our introduc- tory computing course as we converted the course to Python from RAPTOR [7], a flowchart-based programming environment. This change was driven by several factors, including perceived limita- tions of visual programming (e.g., support for automated testing) and perceived advantages to introducing students to a program- ming language they may use elsewhere. In contrast with other studies of this type, our work differs in several key ways: • our introductory computing course is required for all stu- dents at our institution; • our study naturally includes students of all majors, not just those pursuing computing or STEM majors; and • two versions of the course were offered that differed in the programming language used for instruction and assign- ments. The result is a randomized between-subjects experimental design with a relatively large N (> 1,000) to examine the impact of the programming language in the course. Our contributions are as follows: • We compare students’ performance in our introductory com- puting course based on the programming language used (i.e., RAPTOR or Python). • We compare the performance of underrepresented groups in our introductory computing course based on the program- ming language used. 32