Cluster-based Implementation of Resource Brokering Strategy for Parallel Training of Neural Networks Volodymyr Turchenko 1 , Taras Puhol 1 , Anatoly Sachenko 1 and Lucio Grandinetti 2 1 Research Institute of Intelligent Computer Systems, Ternopil National Economic University, 3 Peremoga Square, Ternopil, 46004, UKRAINE, {vtu,tpu,as}@theu.edu.ua 2 Department of Electronics, Informatics and Systems, University of Calabria, via P. Bucci 41C, 87036, Rende (CS), ITALY, lugran@unical.it Abstract—The implementation issues of a cluster-based resource brokering strategy intended for efficient parallelization of neural networks training are presented in this paper. We describe a strategy of resource brokering based on the prediction of execution time and parallelization efficiency of algorithms using a BSP computation model and Pareto optimality with a weight coefficients approach for choosing optimal solution. Our results show a reasonable adaptation of the resource brokering strategy to the environment of a real computational cluster providing the minimal total time to delivery of the parallel application. Keywords—computational cluster, resource broker, Pareto-optimality, neural networks. I. INTRODUCTION 1 The task of a Grid resource broker and scheduler is to dynamically identify and characterize the available resources and to select and allocate the most appropriate resources for a given job [1]. Resource brokering and scheduling problems are the most studied in Grid research community [7-6]. Resource brokering can give some advantages for the parallelized job, for example shortest total time to delivery, execution time, response time, shortest makespan or other execution criteria [1]. Recent works show an importance of economic criteria to analyze the trade-off between an execution time and a cost that can provide efficient resource brokering strategies to execute parallel jobs most economically in the minimum time [6]. The analysis of papers [1-6] shows that the existing solutions usually use two criteria, an execution time and a cost, as the most important optimization criteria for the development of efficient strategies of resource brokering. Recently we have developed a resource brokering strategy based on three criteria, a cost of a parallel system, a predicted execution time and a parallelization efficiency of the algorithm [7]. This strategy has been developed for the efficient parallelization of neural networks training, in particular it is tested on the example of the batch pattern 1 This research is funded by EU FP7 Marie Curie PIIF-GA-2008- 221524 and PIIFR-GA-2008-908524 grants. back propagation training algorithm of a multilayer perceptron [8-9]. The developed strategy uses a BSP- based computational cost model [10] of the parallel algorithm for the prediction of its execution time and parallelization efficiency. The strategy is based on Pareto- optimality with the weight coefficients approach for choosing optimal solution. Our results show [7] that the developed resource brokering strategy has good conformity with the desired scheduling policy of minimization the execution time of the algorithm with maximization of its parallelization efficiency in the most economic way. The resource brokering strategy is developed within the creation of a parallel Grid-aware library for neural networks training PaGaLiNNet [11]. The library consists of software modules implementing several parallel algorithms, e.g. parallel batch pattern back propagation training algorithms of multi-layer perceptron (MLP) [5-6] and recurrent neural network [12] and the resource broker module. The implementation issues of this library are testing now on the Ukrainian Academic Grid [13], which is based on NorduGrid middleware. The authors of [1] provided general description of algorithms, methods and software for a Grid resource manager that performs resource brokering and job scheduling. It selects computational resources based on actual job requirements, job characteristics and information provided by the recourses. They described general job submission and resource brokering algorithm based on the shortest predicted TTD. The main difference of our approach is in the implementation of our resource brokering strategy [7] which selects computational resources based not only on the predicted values of job’s execution time (it is a main part of TTD – total time to delivery), but also using the price of the computational resource and the predicted values of parallelization efficiency of the algorithm. The goal of this paper is to describe the first step of the implementation of Grid-based strategy of resource brokering. This first stage of our implementation we have developed in the environment of a computational cluster since from the organizational point of view both cluster-