An Agent-Based Infrastructure for Parallel Java on Heterogeneous Clusters Jameela Al-Jaroodi, Nader Mohamed, Hong Jiang, and David Swanson Department of Computer Science and Engineering University of Nebraska – Lincoln Lincoln, NE 68588-0115 [jaljaroo, nmohamed, jiang, dswanson@cse.unl.edu] Abstract In this paper, we introduce an agent-based infrastructure that provides software services and functions for developing and deploying high performance programming models and applications on clusters. A Java-based prototype, based on this architecture, has been developed. Since this system is written completely in Java, it is portable and allows executing programs in parallel across multiple heterogeneous platforms. With the agent-based infrastructure, users need not deal with the mechanisms of deploying and loading user classes on the heterogeneous cluster. Moreover, details of scheduling, controlling, monitoring, and executing user jobs are hidden. In addition, the management of system resources is made transparent to the user. Such uniform services, when rendered available in a ubiquitous manner, are essential for facilitating the development and deployment of scalable high performance Java applications on clusters. An initial deployment over a heterogeneous, distributed cluster results in significantly enhanced performance; absolute performance compared to C (MPI) improves with increased granularity of the algorithms. Key Words: Parallel Java, distributed Java, cluster, heterogeneous systems, software agents, middleware. 1. Introduction An agent-based infrastructure is developed to provide services for high performance-programming environments and applications that can be used on clusters or heterogeneous systems. Software agents enhance expandability, allowing the number of machines involved to grow easily, by providing services that include job distribution, monitoring, and controlling for the system. Agent technology has been used in many systems to enhance the performance and quality of service provided. One example is in communications systems where software agents provide powerful means to let systems inter-operate and manage large heterogeneous networks [6]. In this study, we utilize software agents for a parallel and distributed Java programming environment. This provides flexibility and ease of managing the different resources available. A new choice of architecture for high performance computing has emerged to be clusters of PCs/workstations [3] and heterogeneous networked systems. Such systems provide processing power comparable to special purpose high-end multi-processor systems for a fraction of the cost. However, these systems require system and software services that can support the distributed architectures and provide transparent and efficient utilization of the multiple machines available. Moreover, as Java becomes more popular and available on almost any platform, there has been an increasing interest in using Java for high performance computing and distributed applications on clusters. This has generated intensive research in the area of parallel, distributed and multithreaded programming environments and tools for Java on distributed systems. In the rest of this paper, we first discuss parallel Java programming models and identify common infrastructure service requirements on clusters and heterogeneous systems in section 2. Then, in section 3, we describe the architecture and features of the proposed infrastructure. Section 4 discusses an example, the Java Object-Passing Interface (JOPI), that utilizes the services provided by the agents, and presents the results of experiments conducted to evaluate the performance. In section 5, the advantages of the agent-based infrastructure are discussed. Finally, Section 6 concludes the paper with remarks about the current and future work. 2. Parallel Java on Clusters Java’s popularity among developers is increasing steadily and many research groups have started exploring the possibilities of using Java on multi-processor systems. Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER’02) 0-7695-1745-5/02 $17.00 © 2002 IEEE