Design of a Question and Answer Approach to Crowd Debugging Christian M. Adriano University of California Irvine adrianoc@uci.edu Namrata Puri University of California Irvine nspuri@uci.edu ABSTRACT Software debugging comprises most of the software maintenance time and is notorious for requiring high-level skills and application specific knowledge. Crowdsourcing software debugging could lower those barriers by having each programmer perform small, self-contained and parallelizable tasks, hence accommodating different levels of availability and expertise. Therefore, such new approach might enable society to tackle massive software development efforts, as for instance, setting a task force of hundreds of programmers to debug and adapt the existing software to be used in an emergency response to a natural catastrophe. This type of effort is unimaginable nowadays due to the high latency in mobilizing the right programmers and organizing their work. Crowdsourcing assists in overcoming these challenges due to the availability of a large base of contributors working towards a common goal. Debugging process is not a sequential task and this leads to the primary issue of dividing the debugging task into microtasks and asking the appropriate questions based on the microtasks for analysis of the software by the crowd. Our paper attempts to provide the solution of dividing the main task into several microtasks by leveraging the structure of the task followed by associating template questions with each of the microtasks. This can assist in reducing the overhead of the individual developer during the debugging process and make crowd debugging a reality. Categories and Subject Descriptors D.2 SOFTWARE ENGINEERING General Terms Design, Experimentation Keywords Debugging, Crowdsourcing, Fault localization, Questions and Answers, Templates. 1. INTRODUCTION Crowdsourcing enables non-professionals to participate in labor intensive activities and produce useful results. For instance, the game Foldit [1] enabled a crowd of people to collectively contribute to find the most stable structure of folded proteins. This brings forth the concept of applying the knowledge of the crowd to contribute to the software development effort. Besides being labor intensive, software development has a large base of contributors to possibly draw on (an estimated 13 million people in US self-report as “doing programs” [2]). Crowdsourcing software development aims at enabling thousands of volunteer programmers to make micro contributions. These micro contributions are further aggregated to accomplish a large software product. For instance, imagine that a resource allocation system (e.g., for food, water, personnel) requires the adaptation of existing software as part of the emergency response for a natural catastrophe. How could we quickly benefit from the thousands of volunteers eager to contribute to adapt the software? How to fast uncover faults in failing functionalities that are subject to unanticipated uses by the emergency response team? Therefore, adaptation boils down to a debugging process that encompasses identifying each failure, relating it to faults in the code (fault localization) and fixing it, but in record times. Crowdsourcing software debugging is challenging because crowdsourcing usually requires a discrete sequence of steps, whereas software debugging is very hard to break down into discrete steps for many reasons. First, debugging consists of iterations of trial and error, so people don’t follow predictable sequences of steps. Second, debugging the same fault with multiple people is even more difficult, because debugging as an individual activity relies on tacit knowledge that is acquired as the programmer searches for the fault, understands it and experiments with possible fixes. Therefore, the design solution for crowd debugging either contemplates a way to transfer the tacit knowledge among programmers or the solution completely precludes the need of it. The research approach comprises a mechanism of question and answers. A lot of developers rely on questions during debugging and maintenance tasks. The solution we propose is that if the questions presented to the developers are answered within a suspicion range then is possible to identify the fault within a range of certainty. Thus, the approach primarily constructs on encapsulating questions in microtasks, the results of which are combined to either suggest a new microtask to be worked on or to reach the final stage of fault identification. Developers need to provide a positive or negative answer to the questions generated by the microtasks. We hypothesize that a continuous chain of answer ultimately produces a root-case that links the failure to the fault. The challenge of the approach is effectively dividing the debugging task into microtasks and designing questions that can be answered by considering the context of only one microtask. Our insight consists of developing questions for which positive answers imply code behaving correctly and negative answers for code behaving suspiciously. The former positive answers indicate Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.