Analysing and Interpreting Quantitative Eye-Tracking Data in Studies of Programming: Phases of Debugging with Multiple Representations Roman Bednarik and Markku Tukiainen Department of Computer Science and Statistics, University of Joensuu, PO Box 111, Joensuu, Finland firstname.surname@cs.joensuu.fi Abstract. While eye-tracking systems become gradually improved and easier to apply, the methodological challenges of how to analyze, inter- pret and relate the eye-tracking data to user processing remain. Studies of programming behavior are not an exception. We have conducted a reanalysis of eye-tracking data from a previous study that involved pro- grammers of two experience groups debugging a program with the help of graphical representation. Proportional fixation time on each representa- tion, frequency of visual attention switches between the representations and type of switch were investigated in relation to five consequential phases of ten minutes of debugging. Therefore, we have increased the granularity of focus on debugging process. We found some repetitive patterns of visual strategies that were asso- ciated with less experienced programmers finding fewer errors. We also discovered that at the beginning of the process programmers make use of both the code- and graphical representations while frequently switching between them. During the process, more experienced programmers be- gan to integrate also the output of the program and finish the debugging with frequent switching between the three representations. We discuss benefits and limitations of this approach to analyzing and interpreting the quantitative eye-tracking data. As part of future research we propose to investigate the symmetries of representation switching behavior. 1 Introduction While the technological problems of eye-tracking systems are being continually resolved, granting the increasing usability of the technique, the methodological issues prevent the technology from yet spreading wider. Most challenging – apart from the somewhat remaining technical problems – [1] list two methodological problems with eye-tracking: labor-intensive data extraction and difficulties in their interpretation. Modern eye-tracking systems are easy to operate, make no interference with participants, and can capture up to 90% of population. Com- mercially available eye-tracking systems are often supplied with recording and