Dynamic Load Balancing Experiments in a Grid Vrije Universiteit Amsterdam, The Netherlands CWI Amsterdam, The Presented by 張肇烜

Outline Introduction Introduction Experimental Setup Experimental Setup Implementation of Dynamic Load Balancing Implementation of Dynamic Load Balancing Experimental results Experimental results Conclusions Conclusions

Introduction A gird environment is extremely unpredictable. A gird environment is extremely unpredictable. Exponential Smoothing (ES) was shown to be a good predictor for processing power. Exponential Smoothing (ES) was shown to be a good predictor for processing power. The experiments were performed with the classical Successive Over Relaxation (SOR) application. The experiments were performed with the classical Successive Over Relaxation (SOR) application.

Experimental Setup To carry out experiments with parallel applications in a realistic setting, the test bed must have the following key characteristics of a grid environment: To carry out experiments with parallel applications in a realistic setting, the test bed must have the following key characteristics of a grid environment: Processor capacities often differ. Processor capacities often differ. Processor loads change over time. Processor loads change over time. Processors are geographically distributed. Processors are geographically distributed. Network conditions are highly unpredictable. Network conditions are highly unpredictable.

Experimental Setup (cont.) We have chosen to conduct our experiments with four sites. We have chosen to conduct our experiments with four sites.

Experimental Setup (cont.) Our implementation of the load balancing step is as follows: Our implementation of the load balancing step is as follows: At the end of each iteration the processor predict their processing speed for the next iteration. At the end of each iteration the processor predict their processing speed for the next iteration. After every N iterations the processors send their prediction to processor 0, the DLB scheduler. After every N iterations the processors send their prediction to processor 0, the DLB scheduler. The processor calculates the “ optimal ” load distribution given those prediction and sends relevant information to each processor. The processor calculates the “ optimal ” load distribution given those prediction and sends relevant information to each processor.

Experimental Setup (cont.) Equal Load Balancing (ELB) : Equal Load Balancing (ELB) : ELB assumes no prior knowledge of processor speeds of the nodes, and consequently balances the load equally among the different nodes. ELB assumes no prior knowledge of processor speeds of the nodes, and consequently balances the load equally among the different nodes.

Implementation of Dynamic Load Balancing We use the Exponential Smoothing (ES) technique to obtain these predictions. We use the Exponential Smoothing (ES) technique to obtain these predictions. ES appears to be a simple and usable method in load balancing strategies. ES appears to be a simple and usable method in load balancing strategies.

Experimental results Stochastic behavior of the calculation times: Stochastic behavior of the calculation times:

Experimental results (cont.) Stochastic behavior of the calculation times: Stochastic behavior of the calculation times:

Experimental results (cont.) Stochastic behavior of the calculation times: Stochastic behavior of the calculation times:

Experimental results (cont.) Experiments with DLB and ELB: Experiments with DLB and ELB: Running time for SOR based on DLB compared to ELB: Running time for SOR based on DLB compared to ELB:

Experimental results (cont.) Experiments with DLB and ELB: Experiments with DLB and ELB: Cumulative running time as a function of the iteration number: Cumulative running time as a function of the iteration number:

Experimental results (cont.) Experiments with DLB and ELB: Experiments with DLB and ELB: Running times as a function of the number of rows: Running times as a function of the number of rows:

Conclusions Extensive experimentation in the testbed environment PlanetLab have led to the following conclusions: Extensive experimentation in the testbed environment PlanetLab have led to the following conclusions: A significant speedup factor of on average 1.8 can be consistently achieved by implementing DLB. A significant speedup factor of on average 1.8 can be consistently achieved by implementing DLB. The relation between the running time and the problem size is approximately linear. The relation between the running time and the problem size is approximately linear.

Conclusions (cont.) We address a number of challenges for further research: We address a number of challenges for further research: In depth-analysis of parallel applications with a non-linear structure. In depth-analysis of parallel applications with a non-linear structure. Develop optimal load balancing schemes advanced and accurate predictions of the calculation times are needed. Develop optimal load balancing schemes advanced and accurate predictions of the calculation times are needed.