SCPOR: An Elastic Workflow Scheduling Algorithm for Services Computing Cui Lin Department of Computer Science California State University, Fresno Fresno, CA, USA Email: clin@csufresno.edu Shiyong Lu Department of Computer Science Wayne State University Detroit, MI, USA Email: shiyong@wayne.edu Abstract—Due to the complexity of scientific processes, computing and storage resources in a scientific workflow are often needed on an uneven basis, thus, the demand of resources is elastically changing during a run of a workflow. Most existing workflow scheduling algorithms only consider a computing environment in which the number of compute resources is bounded. Resources assigned to a workflow cannot be automatically determined on demand of the size of the workflow and are not released to the environment until an execution of the workflow completes. The salient features of service-oriented computing have brought a new opportunity to schedule workflows on resources with elastically changing demand, as they allow resources to scale on demand as usage changes through dynamic provisioning. To address this issue, we firstly formalize a model of a service-oriented computing environment and a workflow graph representation for the environment. Then, we propose SCPOR, a scientific workflow scheduling algorithm that is able to schedule workflows in need of elastically changing compute resources. Our extensive experiments and comparisons for compute-intensive and data- intensive workflows have shown that SCPOR not only outper- forms several representative workflow scheduling algorithms in optimizing workflow execution time, but also enables resources to scale elastically during workflow execution. Keywords-Workflow scheduling; elastic scaling; service- oriented computing; heterogeneous environment. I. I NTRODUCTION Due to the complexity of scientific processes, scientific workflows have become increasingly compute and data intensive [1]. The resources for computation and data storage in a scientific workflow are often needed on an uneven basis [2]. As shown in Figure 1.(a), only one compute resource is required at stages t 1 and t 4 ; while 6 resources are needed for stage t 2 and 4 resources are needed for stage t 3 to maximize task parallelism. The type of compute resources may also significantly affect the computation time of each task. In Figure 1.(b), the sizes of data consumed by each computation vary during stages t 1 - t 4 . The input data could either distribute across 6 compute resources at t 2 to maximize parallel computing; or partially centralize into 3 compute resources to minimize the data communication for next step of task computation at t 3 . For example, if both task T 2 and T 4 are run on the same compute resource, the output datasets of T 2 and T 4 can be directly consumed by task T 8 , denoted by D [2,4],8 , without the delay of data transferring from one resource to another; sometimes, all input data should be centralized into one compute resource to compute a task, e.g. D [8,9,10,11],12 . Therefore, it is difficult to determine resources for scheduling the workflow, given the elastic resource usage, the heterogeneity of compute resources, and data communication between these resources. T 1 T 2 T 3 T 4 T 5 T 8 T 9 T 6 T 7 D ( 1 , 3 ) D ( 1 , 4 ) D ( 1 , 5 ) D ( 1 , 6 ) D ( 1 , 7 ) D (1, 2) D( [ 2 ,4 ] ,8 ) D( [ 3 ,5 ] ,9 ) D( [ 4 ,6 ] ,1 0 ) D( [ 5 ,7 ] ,1 1 ) D ([8 ,9 , 1 0 ,1 1 ],1 2 ) t 2 t 3 t 4 t 5 t 1 t 2 t 3 t 4 ( a) ( b ) T 1 0 T 1 2 T 1 1 D (1 2 ,0 ) D (0 ,1 ) t 1 Figure 1. Elastically changing demand on workflow (a) compute resources and (b) data storage resources. The emergence of service-oriented computing environ- ments has brought a new opportunity to solve this scheduling problem. A service-oriented computing (SOC) environment provides scientific workflows with salient features that are distinct from other computing environments: (1) compute resources in an SOC environment are exposed as services to allow the access over the network; (2) the number and type of compute resources assigned to a workflow can be determined by service requests; (3) the number of resources assigned to a workflow can be dynamically changed at runtime: additional resources can be assigned due to under- provisioning, while unnecessary resources can be released back to the environment in case of over-provisioning. Work- flow compute resources can be elastically scaled on demand; (4) not all requested compute resources need to be assigned at the beginning of a workflow execution. Resources can be assigned only when they are needed. To run a workflow in an SOC environment, as shown in Figure 2, a user firstly makes a request to a Workflow Scheduling Service (WSS). The request includes the type of compute resources for an execution of the workflow, and the idle time threshold to release a long idle resource back to