Scheduling Tasks for Software Crowdsourcing Platforms to Reduce Task Failure Jordan Urbaczek 1 , Razieh Saremi 1 , Mostaan Lotfalian Saremi 1 , and Julian Togelius 2 1 Stevens Institute of Technology, 2 New York University 1 {jurbacze, rsaremi, mlotfali}@stevens.edu 2 julian.togelius@nyu.edu Abstract—Context: Highly dynamic and competitive crowd- sourcing software development (CSD) marketplaces may experi- ence task failure due to unforeseen reasons, such as increased competition over shared supplier resources, or uncertainty asso- ciated with a dynamic worker supply. Existing analysis reveals an average task failure ratio of 15.7% in software crowdsourcing markets. Goal: The objective of this study is to provide a task scheduling recommendation model for software crowdsourcing platforms in order to improve the success and efficiency of software crowdsourcing. Method: We propose a task scheduling method based on neural networks, and develop an system that can predict and analyze task failure probability upon arrival. More specifically, the model uses a range of input variables, including the number of open tasks in the platform, the average task similarity between arriving tasks and open tasks, the winner’s monetary prize, and task duration, to predict the probability of task failure on the planned arrival date and two surplus days. This prediction will offer the recommended the day associated with lowest task failure probability to post the task. The proposed model is based on the workflow and data of Topcoder, one of the primary software crowdsourcing platforms. Results: We present a model that suggests the best recom- mended arrival dates for any task in the project with surplus of two days per task in the project. The model on average provided 4% lower failure ratio per project. Conclusions: The proposed model empowers crowdsourcing managers to explore potential crowdsourcing outcomes with respect to different task arrival strategies. Index Terms—Crowdsourcing, Task Scheduling, Task Similar- ity, Task Failure, Neural Network, TopCoder I. I NTRODUCTION Crowdsourced Software Development (CSD) has been used increasingly to develop software applications [1]. Crowdsourc- ing mini software development tasks leads to lower accelerated development [2]. In order for a CSD platform to function efficiently, it must address both the needs of task providers as demands and crowd workers as suppliers. Any kind of skew in addressing these needs leads to task failure in the CSD platform. Generally, planning for CSD tasks that are complex, independent, and require a significant amount of time, effort, and expertise [1] is challenging. For the task provider, request- ing a crowdsourcing service is even more challenging due to the uncertainty of the similarity among available tasks in the platform and the arrival of new tasks [3] [4]. The availability of crowd workers skill sets and consistency of performance history is also uncertain [5] [6]. These factors raise the issue of receiving qualified submissions, since crowd workers may be interested in multiple tasks from different task providers based on their individual utility factors [7]. It has been reported that crowd workers are more interested in working on tasks with similar concepts, monetary prize, technologies, complexities, priorities, and duration [7] [8] [4] [9]. However, attracting workers to a large group of similar tasks may cause zero registration, zero submissions, or unqualified submissions for some tasks due to lack of availability from workers [10] [11]. Moreover, lower level of task similarity in the platform leads to higher chance of task success and workers elasticity [12]. For example, in Topcoder 1 , a well-known Crowdsourcing Software platform, an average of 13 tasks arrive daily and are added to an average list of 200 existing tasks. There is an average of 137 active workers to take the tasks at that period, which leads to an average of 25 failed tasks each day. According to this example, there will be a long queue of tasks waiting to be taken. Considering the fixed submission date, such a waiting line may result in failed tasks. These challenges have traditionally been addressed with task scheduling methods. The objective of this study is to provide a task schedule recommendation framework for a software crowdsourcing platform in order to improve the success and efficiency of software crowdsourcing. In this study, we first present a motivational example to explain the current task status in a software crowdsourcing platform. Then we propose a task scheduling architecture using a neural network strategy to reduce probability of task failure in the platform. More specifically, the system uses a range of input variables, including the number of open tasks in the platform, the average task similarity between arriving tasks and open tasks, the winner’s monetary prize, and task duration, to predict the probability of task failure on the planned arrival date and two surplus days. This prediction will offer the recommended the day associated with lowest task failure probability to post the task. The proposed system represents a task scheduling method for competitive crowdsourcing platforms based on the workflow of Topcoder, one of the primary software crowd- sourcing platforms. The evaluation results provided on average 4% lower task failure probability. 1 https://www.topcoder.com/ arXiv:2006.01048v2 [cs.DC] 20 Jul 2020