Introduction MSCS 6060 – Parallel and Distributed Systems

Parallel System A parallel system consists of multiple processors that communicate with each other using shared memory 2

Distributed System A distributed system consists of multiple processors connected by a communication network 3

4 Parallel vs. Distributed System Parallel SystemsDistributed Systems MemoryTightly coupled shared memory UMA, NUMA Distributed memory Message passing, RPC, and/or used of distributed shared memory ControlGlobal clock controlNo global clock control Synchronization algorithms needed Processor interconnection Order of Tbps Bus, mesh, tree, mesh of tree, and hypercube (-related) network Order of Gbps Ethernet(bus), token ring and SCI (ring), myrinet(switching network) Main focusPerformance Scientific computing Performance(cost and scalability) Reliability/availability Information/resource sharing

5 Software and hardware service layers in distributed systems

6 Client Server Model

A Service Provided by Multiple Servers 7

Proxy Servers and Caches 8

Peer to Peer Computing 9

Mobile Agents 10

Parallel Computing Tutorial 11

12 Advantages of Distributed Computing Distributed applications  airline reservation, banking system Data Sharing  distributed databases Resource sharing  sharing expensive processor Low cost  Availability of high bandwidth networks and low cost workstations Reliability  Failure of single system does not affect the whole Scalability  In parallel systems, shared memory becomes a bottleneck Modularity and Heterogeneity  Flexible to add a new processor

References (Parallel Computing Tutorial) Distributed Systems: Concepts and Design; 4th EdDistributed Systems: Concepts and Design; 4th Ed by G. Coulouris, J. Dollimore, T. Kindberg, Addison Wesley (ISBN ) Concurrent and Distributed Computing in JavaConcurrent and Distributed Computing in Java by Vijay K. Garg 13