Simple Load Balancing CS550 Operating Systems

Announcements Project will be posted – TBA This project will use the client-server model and will require implementation of load balancing We will review square matrix multiplication as an example of load balancing

Parallel decomposition and load balancing Parallel decomposition - how do we divide up work and do work in parallel. Load balancing - how do we distribute this work so that it is best divided between resources (CPUs/nodes for cluster computing) For addition of square matrices A and B, we could divide the work by rows and try to allocate the same number of rows to each thread or process.

Example Assume we have two N by N matrices, A and B Assume we have K threads or processes where K < N. We can divide up the work as follows: – Allocate N / K rows to each process or thread and perform addition. – Note that N / K is an integer division.

Example Given two 16*16 matrices A and B, divide the work between 4 threads So, each thread should calculate 16 / 4 = 4 rows of addition.

What if things don't divide up evenly? Given two 1024 * 1024 matrices, divide the work between 5 threads. Using the previous formula, we would allocate 1024 / 5 = 204 rows to each thread to perform addition. But 204 * 5 = What about the other 4 rows? Notice that 1024 % 5 gives the remainder. We must allocate the rest of the work to at least one of our existing threads. We could try to balance the work again if necessary.

Other Load Balancing Problems Finding pi Numerical Integration Particle Physics Simulations Computational Chemistry Weather Air Traffic Control Sometimes dividing work (load balancing) can be very difficult – see IEEE and ACM journals.