1. Load Balancing and Termination Detection Load balance : - statically before the execution of any processes - dynamic during the execution of the processes

2. Load Balancing and Termination Detection Potential static load balancing techniques Mapping problem and scheduling algorithm. Round robin algorithm. Randomized algorithm. Recursive algorithm. Simulated annealing. Genetic algorithm

3. Load Balancing and Termination Detection bin packing algorithm

4. Load Balancing and Termination Detection Communicated processes should be executed on processors with direct communication paths to reduce the communication delays. The mapping of processes to processors is a NP complete problem. NP stands for non-deterministic polynomial and means that there is No polynomial time algorithm for solving the problem.

5. Load Balancing and Termination Detection There are several problems with static load balancing. Difficult to estimate accurately the execution times of various parts of a program without actually executing the parts. Communication delays may vary under different circumstances. Some problems have indeterminate number of steps to reach their solution We need to turn to dynamic load balancing

6. Load Balancing and Termination Detection Dynamic load balancing - centralized (master slave) - decentralized (peer-to-peer)

7. Load Balancing and Termination Detection Centralized work pool (processor farm) divide-and-conquer problems the problems in which tasks are different and of different size. the number of tasks change during execution such searching algorithm FIFO Priority

8. Load Balancing and Termination Detection Termination: Stopping the computation when the solution has been reached is Called termination. -The task queue is empty -Every process has made a request for another task without any tasks being generated (local termination information,such search algorithm) specific local termination information, synchronous iterations)

9. Load Balancing and Termination Detection The disadvantage of centralized load balancing -potential exists for a bottleneck when there are many slave process that could be making simultaneously requests. -centralized work pool will be satisfactory if there are few slaves and the tasks are intensive.

10. Load Balancing and Termination Detection Decentralized dynamic load balancing One approach is to distribute the work pool

11. Load Balancing and Termination Detection Four policies of load balance. Transfer policy. Location policy. Selection policy. Information policy

12. Load Balancing and Termination Detection Location policy - receiver initiated : high system load - sender initiated : light system load - hybrid initiated ping-pong effect : keep passing on the same task that is received

13. Load Balancing and Termination Detection Process Selection - round-robin algorithm - random polling algorithm

14. Load Balancing and Termination Detection Load balancing using a line structure

15. Load Balancing and Termination Detection - for left and right communication - for the current task

16. Load Balancing and Termination Detection

17. Distributed Termination Detection Algorithm. Application-specific local termination condition exist throughout the collection of processes, at time t, (each process can send a message to the master when its local termination conditions are satisfied). There are no messages in transit between processes at time t. (it is hard to recognize)

18. Load Balancing and Termination Detection Using Acknowledge Messages. Each process is in one of two states - inactive - active

19. Load Balancing and Termination Detection. Its local termination condition exists. It has transmitted all its acknowledgements for tasks it has received.. It has received all its acknowledgements for tasks it has send out.

20. Load Balancing and Termination Detection Ring termination algorithm 1. When P0 has terminated, it generates a token that it passed to P1. 2. When Pi(1<=I<=n) receive the token and has already terminated, it passed the token onward to Pi+1. Otherwise, it waits for its local termination condition and then passes the token onward. Pn-1 passes the token to P0. 3. When P0 receives a token, it knows that all processes in the ring have terminated. A messages can then be sent to all processes informing them of global termination, if necessary.

21. Load Balancing and Termination Detection

23. Tree algorithm

24. Load Balancing and Termination Detection Shortest path 1. The shortest distance between two towns or other points on a map, where the weights represent distance 2. The quickest route to travel, where the weights represent time 3. The least expensive way to travel by air, where the weighs represent the cost of the flights between cities 4. The best way to climb a mountain given a terrain map with contours 5. The best rout through a computer network for minimum message delay 6. The most efficient manufacturing system, where the weights represent hours of work.

25. Load Balancing and Termination Detection Graph representation -adjacency matrix -adjacency list Space:O(n 2 ) Time : O(1)

26. Load Balancing and Termination Detection Space:O(nv max ) Time : O(v)

27. Load Balancing and Termination Detection Searching a graph - Moor single-source shortest path algorithm - Dijkstra ’ s single-source shortest path algorithm

28. Load Balancing and Termination Detection