Distributed Computing Software based solutions to Parallel Computing

Distributed Computing b Cluster based computing b Network of Workstations Duality of standalone workstation Vs. parallel computing environment.Duality of standalone workstation Vs. parallel computing environment. b Distributed Applications RC5 ClientsRC5 Clients SETI ClientsSETI Clients

Issues of Distributed Computing b Homogeneous networks Networks of similar machines. Requires only one version of executables.Networks of similar machines. Requires only one version of executables. b Heterogeneous networks Different Architectures of machines. (Alpha,x86]. Each architecture requires its own version of executables.Different Architectures of machines. (Alpha,x86]. Each architecture requires its own version of executables. b Issues of data representation, system level interface differences, and suitability.

Support Model b Application support is critcal. b Three basic models Application is built with full awareness and supportApplication is built with full awareness and support User land facilities are provided to facilitate awareness and supportUser land facilities are provided to facilitate awareness and support Kernel level supportKernel level support

Application Support b Application modeled as self-contained entity capable of client-server, peer-peer cooperation models. RC5 clientRC5 client SETI clientSETI client b Support can be streamlined b Loss of transparency, load balancing,lower QoS.

User Level Support b Provide basic parallel mechanisms Message/Data passingMessage/Data passing SynchronizationSynchronization Process HandlingProcess Handling –Rough control b Provides greater transparency Less dependence on architecture, transparency supports heterogeneous solutionsLess dependence on architecture, transparency supports heterogeneous solutions

Kernel Support b Provides system level support SchedulingScheduling –Load Balancing –Process Migration Resource AllocationResource Allocation b Doesn’t work well in heterogeneous configurations. b Process Migration increases communication costs

Three Solutions b PVM - Parallel Virtual Machine User Land Parallel SolutionUser Land Parallel Solution Message PassingMessage Passing b MPI - Message Passing Interface StandardStandard Several ImplementationsSeveral Implementations User Land SupportUser Land Support

Three Solutions - continued b MOSIX - Linux Kernel Extensions Provide dynamic schedulingProvide dynamic scheduling process migrationprocess migration system level integrationsystem level integration

PVM - Parallel Virtual Machine b Message Passing Paradigm b Heterogeneous Computing Deals effectively with data representation issuesDeals effectively with data representation issues b Supports variety of systems including MPP’s, SMP’s, and vector machines b Uses a daemon to provide parallel facilities

PVM - continued b Tracks tasks using a unique system assigned ID called a TID b Supports grouping, and group level activities.

MPI or LAM b Message Passing Interface b MPI is the standard, several implementations exist. One is LAM and is maintained by Notre Dame. b Similar to PVM HeterogeneousHeterogeneous Uses a daemon, also includes a peer-peer mode.Uses a daemon, also includes a peer-peer mode.

MPI - LAM - continued b LAM environment must be started explicitly b Provides compiler shells to handle program compilation b Nodes are dynamic b Suite of utilities to maintain the message passing virtual machine. b The parallelism is explicitly programmed

MPI - LAM - continued b MPI addresses resource limitations. Uses a property called Guaranteed Envelope Resources to maintain integrity of processes.

MOSIX and Linux b Linux, not much more to say. b Mosix is developed to extend several unix operating systems b Homogeneous b Transparent and Preemptive process migration b Dynamic process reassignment

Mosix - continued b Global resource assignment b Ideal for cluster based computing b Offers memory ushering prevents thrashingprevents thrashing b Maintains home system b Requires that the nodes of the cluster be networked well

Mosix - Continued b Mosix maintains a given process’ association with a home node. Unique home nodeUnique home node b Implements a bi-level approach to maintain this association through migration User Context (remote) - system independent contextsUser Context (remote) - system independent contexts

Mosix - continued Deputy - node dependent interface. Remains when process migrates from home node.Deputy - node dependent interface. Remains when process migrates from home node. b All system dependent calls are routed through remote to deputy at home system gettimeofdaygettimeofday b Initial assignment needs to be done by PVM or MPI. Mosix doesn’t handle this aspect.

MOSIX - Migrated Process

Bad Mosix b Overhead is increased Delayed system callsDelayed system calls I/O sockets (file, network) don’t route wellI/O sockets (file, network) don’t route well Migration time adds overheadMigration time adds overhead b Needs further work in migratable sockets and files

You’re a good audience b Are you asleep yet. b But I tried! b Thank you, b Any Questions?