Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WETSoM’10, May 4, 2010, Cape Town, South Africa, Copyright 2010 978-1-60558-976-3/10/05…$10.00. Improving Developer Activity Metrics with Issue Tracking Annotations Andrew Meneely Mackenzie Corcoran Laurie Williams North Carolina State University {apmeneel, mhcorcor, lawilli3}@ncsu.edu ABSTRACT Understanding and measuring how groups of developers collaborate on software projects can provide valuable insight into software quality and the software development process. Current practices of measuring developer collaboration (e.g. with social network analysis) usually employ metrics based on version control change log data to determine who is working on which part of the system. Version control change logs, however, do not tell the whole story. Information about the collaborative problem- solving process is also documented in the issue tracking systems that record solutions to failures, feature requests, or other development tasks. To enrich the data gained from version control change logs, we propose two annotations to be used in issue tracking systems: solution originator and solution approver. We examined the online discussions of 602 issues from the OpenMRS healthcare web application, annotating which developers were the originators of the solution to the issue, or were the approvers of the solution. We used these annotations to augment the version control change logs and found 47 more contributors to the OpenMRS project than the original 40 found in the version control change logs. Applying social network analysis to the data, we found that central developers in a developer network have a high likelihood of being approvers. These results indicate that using our two issue tracking annotations identify project collaborators that version control change logs miss. However, in the absence of our annotations, developer network centrality can be used as an estimate of the project’s solution approvers. This improvement in developer activity metrics provides a valuable connection between what we can measure in the project development artifacts and the team's problem-solving process. Categories and Subject Descriptors D.2.8 [Software Engineering]: Metrics – process metrics, product metrics General Terms Measurement, Human Factors Keywords Developer, collaboration, metric, network analysis 1. INTRODUCTION Understanding and measuring how groups of developers collaborate on software projects can provide valuable insight into software quality and the software development process. One class of metrics, called developer activity metrics, analyze the structure of development in terms of how developers collaborate [5]. Already, several applications have been found for developer activity metrics, including predicting failures [6, 8], predicting vulnerabilities [5], and studying individual projects [1, 2, 4]. Many of these developer activity metrics employ data obtained from version control change logs to determine who is working on which part of the system. Version control change logs, however, do not tell the whole story. While a version control system records the final solution to an issue, the elements of how that solution came to be are captured in other artifacts, such as the issue tracking system. In issue tracking systems, developers can take a known issue (e.g. a failure or a feature request) and assign it a “ticket”. Developers can then link pertinent artifacts to that ticket, such as patches, change sets in the version control system, error logs, or screenshots. Also on the ticket is an online discussion of how to resolve the issue. Consider the following scenario 1 . A large open source project has an open ticket that needs resolving. A user named Frank then submits a patch to fix the problem by linking his patch to the ticket. An online discussion amongst the system’s developers and users then takes place, which ends in developer Ben deciding that Frank’s solution is correct. Ben then applies the changes from Frank’s patch, linking the version control change set to the ticket. The version control change logs would show that only Ben has fixed the problem when, in fact, Frank originated the solution, while Ben approved the solution. That the two developers collaborated on a solution ought to be captured in developer activity metrics. We propose two issue tracking ticket annotations: solution originators and solution approvers. Intended to “give credit where credit is due”, these annotations can be used to augment the logs from version control systems to provide more accurate information about who contributed to which parts of the system. The objective of this research is to improve the information gained by measurements of developer collaboration by introducing and analyzing by two issue tracking annotations: solution originator and solution approver. We examined the 1 Taken from http://dev.openmrs.org/ticket/1171