Bugs and Emotion: A Content Analysis of Qality Assurance Player Feedback Extended Abstract Luke Thominet Florida International University Miami, Florida luke.thominet@fu.edu ABSTRACT This study uses content analysis to explore how players provide quality assurance (QA) feedback of games in progress. It identifes both the most commonly reported bugs and the bug types that caused the most positive and negative experiences. In doing so, it seeks to build stronger connections between QA and UX research. CCS CONCEPTS · Human-centered computing → HCI design and evaluation methods; KEYWORDS User Experience, Video Games, Content Analysis, Quality Assur- ance ACM Reference Format: Luke Thominet. 2018. Bugs and Emotion: A Content Analysis of Quality Assurance Player Feedback: Extended Abstract. In SIGDOC ’18: The 36th ACM International Conference on the Design of Communication, August 3ś 5, 2018, Milwaukee, WI, USA. ACM, New York, NY, USA, 2 pages. https: //doi.org/10.1145/3233756.3233934 1 INTRODUCTION There are conceptual and procedural divisions between quality as- surance (QA) testing and playtesting in games. While playtesting, a closer analogue of general user experience (UX) research, usually involves players/users, QA testing is primarily the work of inter- nal, specialized testing teams [13]. Yet QA issues have a signifcant impact on UX [10], suggesting the need to build connections be- tween these two areas. This study ties these testing areas together by examining player feedback tickets from an open development game. Open development games are publicly released during the de- velopment process in order to gather player feedback and iterate on design. These games provide valuable data on how users can engage in technology development. Elsewhere, I have argued that open development projects show how we can design prolonged UX research projects as an engaging experience [2]. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for proft or commercial advantage and that copies bear this notice and the full citation on the frst page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). SIGDOC ’18, August 3ś5, 2018, Milwaukee, WI, USA © 2018 Copyright held by the owner/author(s). ACM ISBN 978-1-4503-5935-1/18/08. https://doi.org/10.1145/3233756.3233934 Notably, open developers have also debated the role of players in QA testing. Some developers argued categorically that players should not be used for QA testing [12], and others argued that players are inefective at fnding bugs [15]. Still, many developers implicitly supported using players as QA testers by recommending centralized locations for bug reporting [7] or acknowledging how bugs afect players’ experiences [3]. This study responds to the debate by arguing that rhetorical discourse methods ofer insight into how game bugs afect player experience. As such, this study contributes to technical communica- tion’s ongoing work to use rhetorical discourse analysis to analyze UX practice [11] and video game play and development [4, 5]. 2 METHODS This study used a data set of 300,000 player feedback tickets for Sub- nautica, a scuba diving, exploration game. The tickets were created through an in-game widget where players submitted textual feed- back and reported their emotional state (very unhappy, unhappy, happy, very happy). In a previous study, I analyzed a sample of 3,250 feedback tickets to trace broader UX topics and found that the majority (63.3%) were focused on QA topics [1]. These 2,296 QA player feedback statements were the sample for the current study. This study sought to answer two research questions: 1) What types of bugs did Subnautica players frequently report? and 2) What types of bugs most negatively afected players’ experiences? This study used conventional content analysis [6] to identify the types of bugs that players reported with the initial coding categories drawn from existing game bug taxonomies [8, 9]. This frst coding system was developed further during an initial coding pass. The fnal coding system can be seen in Table 1 below. To analyze the results, overall frequencies of each bug type were calculated. Then the frequency distribution of players’ positive and negative emotions was calculated for each bug type. These distributions were analyzed using a chi-squared test to determine if bug categories showed non-random variation in the reporting of emotions. Results with a p value of <.05 were considered signifcant. 3 RESULTS AND DISCUSSION The full range of bug types identifed in the sample, as well as the frequency of each type is shown in Table 1 below. Two bug types dominated QA player feedback: game object bugs and visual defects. Since Subnautica is an exploration/survival game where players commonly craft new items, these two types of errors were probably apparent during regular gameplay. However, the prevalence of level design reports also suggested that some