An architecture including network QoS in scientific workflows Zhiming Zhao Paola Grosso Ralph Koning Jeroen van der Ham Cees de Laat System and Network Engineering research group Informatics Institute, University of Amsterdam Science Park 104, 1098XG, Amsterdam, the Netherlands {Z.Zhao|P.Grosso|R.Koning|vdHam|C.T.A.M.Delaat}@uva.nl Abstract—The quality of the network services has so far rarely been considered in composing and executing scientific workflows. Currently, scientific applications tune the execution quality of workflows neglecting network resources, and by selecting only optimal software services and computing resources. One reason is that IP-based networks provide few possibilities for workflow systems to manage the service quality, and limit or prevent band- width reservation or network paths selection. We see nonetheless a strong need from scientific applications, and network operators, to include the network quality management in the workflow systems. In this position paper, we discuss our vision on this issue and propose an agent based solution to include network resources in the loop of workflow composition, scheduling and execution when advanced network services are available. Our approach is conducted in the context of the CineGrid project. Key words: Quality of Service, Advanced Network, Scien- tific Workflow, Grid I. I NTRODUCTION The quality of the network connections and services has rarely been taken into account by current scientific workflow management systems, because 1) traditional IP-based networks provide limited reservation capability for workflow engines; 2) the existing e-Science applications assume available network connections as non-changeable services, and seek customized solutions at software level to optimize computing processes and data storage; and 3) the existing applications mainly consider the functionality of the e-Science services, and limited support has been provided for including (network) quality requirements for the services in the composition, enactment and execution of workflows. Still advanced net- work services are essential for many scientific applications, where data transmission delays become a bottleneck of the global performance of the application [1]. Without the quality guarantee at the network level, the global workflow quality requirements cannot be assured. Several strategies have been tried to improve the workflow performance in these cases, such as caching data in a closer location to the computing element [2], or reducing the load of computing tasks by reusing the previous computed results [3]. However, for applications that require data streams from remote data sensors, such as in [4], those solutions will not be sufficient, and advanced network services are crucial to accelerate large data transfer between the distributed application components. The recent emergence of advanced network infrastructures for e-Science enables tuning of network performance at the application level. For instance, the hybrid network architecture [5] can provide connections on different layers based on the same physical fibre; this allows applications to request dedicated circuits for transferring very large quantity data. By including network resources in the scheduling loop, the high level application gets an extra opportunity to optimize execution and improve performance. These new opportunities come with some chal- lenges. Not all network infrastructures provide the network services for reserving specific connections or allocating net- work bandwidth; the service invocation in different network domains is often proprietary and not easily extensible, and makes request for network service provisioning across sites difficult [6]; scheduling network resources requires knowledge on the current state of the network, which implies the ex- istence of a sophisticated monitoring system. In this paper, we discuss the challenges in including network resources in scientific workflows, and propose an agent based architecture to accomplish this goal and realize application level quality tuning of network resources. We apply this in the context of an ongoing project, CineGrid [7]. The paper is organized as follows; first, we will review the state of the art in this field, and then present our solution and the first prototype of our system. II. QOS AND WORKFLOWS: THE STATE OF THE ART From the life-cycle perspective of a scientific workflow, QoS is relevant to four aspects: workflow composition, service selection, execution control and provenance. In the following sections we briefly review the existing work on each issue. A. QoS aware workflow composition A workflow composition process that is QoS-aware must: 1) compose a service of the highest quality and 2) determine the quality of the composition process itself. The first goal is achieved by computing the global quality starting from the QoS attributes of constituting services [8]. Graph reduction is a widely used approach [9]; a pre-defined set of logic patterns define certain reduction rules which can be used to simplify the logical dependencies among constituting services. From the reduction rules, the quality parameters are computed; for