Agent Technology and Scientific Workflow Management in an e-Science Environment Zhiming Zhao Adam Belloum Peter Sloot Bob Hertzberger Informatics Institute, University of Amsterdam Kruislaan 403, 1098SJ, Amsterdam, the Netherlands {zhiming|adam|sloot|bob}@science.uva.nl Abstract In e-Science environments, scientific workflow manage- ment systems (SWMS) hide the integration details among Grid resources and allow scientists to prototype an exper- imental computing system at a high level of abstraction. However, the development of an effective SWMS requires profound knowledge on both application domains and the network programming, and is often time consuming. Agent technologies provide suitable solutions to decompose the control intelligence of flow execution and to encapsulate distributed e-Science resources. The work presented in this paper is conducted in the context of the Dutch Virtual Labo- ratory for e-Science (VL-e) project. Agent technologies are proposed to realise generic workflow support. 1 Introduction An e-Science environment organises Grid services and software components, and allows a scientist to utilise re- mote resources in his domain specific research at an abstract level. Generic Grid middlewares, e.g., Globus toolkit [2] and UNICORE [20], realise services for discovering, ac- cessing and utilising remote resources, and form the basic infrastructure of an e-Science environment. On top of this infrastructure, a Scientific Workflow Management System (SWMS) automates the experiment routines, and glues dif- ferent levels of issues: experiment planning, resources de- ployments and the runtime execution control of the experi- ment. During the past decade, SWMSs have been applied in different domains, e.g., in bio informatics [11, 21], in high energy physics [4], and in astronomical observations [1]. The development of SWMSs is complex and highly in- terdisciplinary: not only the modelling of application pro- cesses requires deep understanding of domain specific ex- periments, but also the coupling of workflow resources in- volves details of different layers of middleware. More im- portantly, the dynamic issues in a runtime e-Science infras- tructure, e.g., availability of resources, demand a SWMS sophisticated control intelligence [25]. The development of an effective SWMS is thus time consuming. A number of focuses can be enumerated from the effort for facilitating the SWMS development. The first one is on developing a new system by extending existing mature workflow models and engines, e.g., Pegasus is on top of DAGMan [10] and Kepler is based on Ptolemy [3]. Another focus is on choos- ing proper middleware to couple SWMS resources; it aims at improve the efficiency for developing high level con- trol intelligence instead of detailed resource binding, e.g., Taverna uses web services as its basic resources [17]. Fi- nally, using the state of art software engineering technolo- gies, e.g., components and agents oriented methodologies, to construct SWMSs is yet another important focus. The work presented in this paper belongs to the latter focus; we discuss how agent technologies are used in SWMS in the context of a Dutch e-Science project: Virtual Laboratory for e-Science (VL-e) [22]. In this paper, we discuss the feasibility and challenges for employing agent technologies to realise scientific work- flow support as generic e-Science services. This paper is organised as follows. First, we analyse the basic issues in realising a SWMS and briefly describe the research context of the Dutch VL-e project. After that, we discuss the short- comings of the current implementation of the generic VL-e framework, and propose an agent based solution to improve the situation. 2 Scientific workflow in an e-Science frame- work From different perspectives, a SWMS can be viewed dif- ferently. As a meta programming environment, a SWMS models the dependencies between experiment processes and allows a scientist to prototype an experimental com- puting system by assembling resources at an abstract level Proceedings of the 17th IEEE International Conference on Tools with Artificial Intelligence (ICTAI’05) 1082-3409/05 $20.00 © 2005 IEEE