1. Parallel Processing
LAB NO 1

2. Parallel Processing
Ability to carry out multiple operations/tasks at the same time using more than one processors or processor cores
Differs from multitasking or multiprogramming in which a single CPU executes several programs at the same time

3. Distributed Computing
Multiple autonomous computers that communicate through a computer network to achieve a common goal
In parallel computing, all processors have access to shared memory, which can be used to exchange information between processors
In distributed computing, each processor has its own private memory (distributed memory); info can be exchanged by passing messages between processors

4. Cluster Computing
Computer cluster is a set of loosely connected computers that work together so that they can be viewed as a single, highly available system
Used for either load balancing or high availability

5. Grid Computing
Applying resources of many computers (from multiple administrative domains) in a network to a single problem at the same time
Although a grid can be dedicated to a specialized application, it is more common to use a single grid for a variety of different purposes

6. Cluster vs Grid Homogeneous system vs heterogeneous system
Tight coupling vs loose coupling
Grids are inherently distributed and spread over LANs, Man or WAN
Single system image vs independent, autonoumous nodes
In a cluster, entire system behaves like a single system with resources managed by a centralized resource manager
In a grid, every node is autonomous, has its own resource manager and behaves like an independent entity.

7. Parallel Computing Memory Architectures
Shared Memory
Can be simultaneously accessed by multiple processes
Programs may run on single or multiple processors
In multi-processor systems, large block of RAM can be accessed by several CPUs
Distributed Memory
Each processor in a multi-processor computer system has its own private memory
MPI or socket programming are main tools used for distributed memory environment

8. MPI- Message Passing Interface
A library of functions or an API
Allows communication between processors in a distributed memory architecture
Allows portability
Uses either a C or a FORTRAN compiler

9. MPI Data Types MPI datatype handle C datatype MPI_INT int MPI_SHORT
MPI_LONG
long
MPI_FLOAT
float
MPI_DOUBLE
double
MPI_CHAR
char
MPI_BYTE
unsigned char

10. MPI Program # include <stdio.h> # include <mpi.h>
main(int argc, char **argv) {
//coding
MPI_Init(&argc, &argv);
MPI_Finalize(); }

11. MPI Init O 4 1
Network 3 2

12. MPI Finalize O 4 1
Network 3 2

13. Compiling and running
mpicc –o <executable file name> <source file name>
mpirun <executable file name>
mpirun –np <no> <executable file name>