1932-8540 (c) 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/RITA.2020.3008132, IEEE Revista Iberoamericana de Technologias del Aprendizaje Abstract—This paper reports our experience in flipping a second- year undergraduate course on software architecture and integration, taught in the second course of a Software Engineering degree. We compare the application of the flipped- classroom methodology with a traditional methodology. Our study encompasses two academic courses, in the years 2017 and 2018, and involves a total number of 434 students and 6 lecturers, placing this among the largest studies on flipped- classroom to date. The paper also reports on the production of the videos used with the flipped-classroom methodology, recorded by the lecturers in informal settings, and provides several lessons learned in this regard. The results of the study, backed by a solid statistical analysis of the data, demonstrate the suitability of the flipped-classroom methodology for laboratory sessions in the subject course. Among other results, our analysis concluded that students had on average 24 more minutes per session to solve in-class exercises with the flipped-classroom methodology; more than 70% of the students considered that the quantity, duration and didactic content of the videos were (very) appropriate; and 9 out of every 10 students would prefer this methodology in the laboratory sessions of future courses rather than a traditional face-to-face approach. Index Terms— Flipped Classroom, Software Engineering, Comparative Study. I. INTRODUCTION N increasing number of papers [1, 8, 18] advocate the application of the flipped-classroom methodology conceived by Bergman and Sams in 2012 [4] as an option to optimally organize the learning time. This methodology consists in transferring to the student the responsibility of acquiring the most theoretical concepts before the class and devoting the in-person class to more practical activities such as solving exercises and discussions, where the assistance of the instructor is much more valuable. To this end, instructors typically transfer part of the content to videos that students must view before the face-to-face sessions. In the context of higher education, and in computer science in particular, many studies have evaluated the results of applying the flipped-classroom methodology both at the national level [12, 13, 17, 19, 20] as well as at the international level [3, 5, 14–16]. In most cases, there seems to be a positive effect of the application of the flipped-classroom on students’ perception of the course and on their academic performance in it. Motivated by these results, the authors of this article Authors are with the Universidad de Sevilla (Seville, Spain), {japarejo,jtroya,sergiosegura,adeladelrio,antoniogamez,amarquez6}@us. es proposed to solve a problem that was pressing in the course of Software Architecture and Integration, in the second year of the Software Engineering degree at the University of Seville. There was a lack of time to carry out exercises in laboratory sessions, partially due to the time invested to explain the technical details at the beginning of each lab. This explanation was aimed at technical concepts needed before starting to work on the lab exercises, such as the step-by-step configuration of the programming environment, instructions for cloud deployment, use of libraries, etc. In this article, we present our experience after having applied the flipped-classroom methodology in the laboratory sessions of the above-mentioned course. We also describe in detail the data collection and analysis carried out and evaluate the impact on students. Our study differs from the existing ones in several aspects. Firstly, the number of students under study. In most cases this number is not very high, ranging from 12 [13] to 200 [3, 5, 15–17]. Only a small portion of the works so far has more than 300 students: 364 in [19] or almost 400 in [14], being our study one of the largest, with 434 students. Another aspect to take into account is the period of time to be considered, since in many cases only one course and year were considered, being it is difficult to compare the flipped-classroom with the traditional methodology. In contrast, our study encompasses two consecutive years in which we used the same contents and evaluation methods for a fair comparison of the results. It is also worth mentioning the large amount of data we collected, such as that related to class attendance, video views, perceived quality of the videos, number of exercises solved, duration of tutoring sessions, grades obtained, and degree of student satisfaction, among others. In total, we collected and analyzed about 2500 questionnaires and surveys, and the results of more than 1000 tests in the gamification platform Kahoot! [10], among other data, which allowed us to draw statistically significant conclusions. Another key aspect of our work, partly required by the volatility of the contents to be used in the course under study, was the need to seek a flexible approach to the methodology. Much of the content shown in the videos changes frequently, mainly due to the rapid evolution of the technologies used. This made us discard any process that could be slow and heavy in the recording of videos, so we propose a flexible guide for the elaboration of the videos, following existing recommendations [9]. DOI (Digital Object Identifier) Pendiente Flipping Laboratory Sessions: An Experience in Computer Science José A. Parejo, Javier Troya, Sergio Segura, Adela del-Río-Ortega, Antonio Gámez-Díaz, Alfonso E. Márquez-Chamorro A Authorized licensed use limited to: Universidad de Sevilla. Downloaded on July 13,2020 at 07:40:56 UTC from IEEE Xplore. Restrictions apply. The final published version of this paper is available from IEEE's website: https://ieeexplore.ieee.org/document/9137231