Paradigms for Process Interaction ECEN5053 Software Engineering of Distributed Systems University of Colorado, Boulder

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Paradigms  Ways to combine the three basic process interaction paradigms  All were developed as graph theory problems  Now that distributed computing exists, – we can use them – recognize problems as variants of these paradigms – apply combinations to obtain solutions – some have grown up to be patterns

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder A limitation  There are entire books on fault tolerant systems  Necessary where high availability and reliability are essential  Outside the scope of this course  Therefore, the algorithms will assume the processors do not fail  The Patterns take this into account to some extent  The more you know, the more you know you don’t know

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Overview  Manager/Workers, aka “Distributed Bag of Tasks” – Master-Slave Architectural Design Pattern  Heartbeat algorithms  Pipeline algorithms – Pipes and Filters Architectural Design Pattern  Probe/Echo algorithms  Broadcast algorithms  Token-passing algorithms – Asynchronous Completion Token Pattern  Replicated servers  Microkernel Architectural Design Pattern  Broker Pattern

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Overview (continued)  Communication patterns – Forwarder-Receiver – Client-Dispatcher-Server – Publisher-Subscriber

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Manager/Workers  Several worker processes share a bag containing independent tasks  Each worker repeatedly removes a task from the “bag”, executes it, possibly generates one or more new tasks that it puts into the “bag”.  This is a variant of which primary paradigm?  Useful for: – Problems with a fixed number of independent tasks – Problems resulting from divide-and-conquer approach

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Manager/Worker (cont. 1)  Benefits – Easy to vary the number of workers – Easy to ensure that each does about the same amount of work  Implementation ManagerWorker implement the bagget tasks dole tasksdeposit results by collect results comm with mgr detect termination

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Master-Slave Pattern (POSA 1)  A way to organize work  Supports fault tolerance, parallel computation, and computational accuracy.  A master component distributes work to identical slave components and computes a final result from the results these slaves return  Applies the divide and conquer principle  Partition work into several subtasks processed independently  A special case is called “triple modular redundancy” – result is considered valid if at least 2 provide the same result.

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Master-Slave Pattern 3 application areas  Fault tolerance: – Execution of a service is delegated to several replicated implementations. Failure of service executions can be detected and handled.  Parallel computing – most common: – A complex task is divided into a fixed number of identical sub-tasks executed in parallel.  Computational accuracy: – Execution of a service is delegated to several different implementations. Inaccurate results can be detected and handled.

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Heartbeat  Reason for the name – worker actions – expand – contract – process – repeat  Useful – Grid computations – image processing – Cellular automata – simulations; e.g. forest fires, biological growth – What do these have in common?

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Heartbeat Implementation  Recall meaning of a “barrier” – Ensures every worker finishes one iteration before starting the next – Heartbeat implements a “fuzzy” barrier process Worker [ = 1 to numberOfWorkers] { declarations of local variables; initializations of local variables; while (not done) { send values to neighbors; receive values from neighbors; update local values – process information; }

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipelines  A filter process – receives data from an input port, – processes it, and – sends results to an output port  What primary paradigm category is this?  A pipeline is a linear collection of filter processes  You see this concept in Unix pipes in a single CPU environment.

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipelines (cont.)  Useful – A way to circulate values among processes – A sorting network – Parallel computations  Three basic structures for a pipeline – Open: Input source and output destination are not specified. Can “drop down” anywhere. – Closed: Open pipeline connected to a coordinator as input source and output consumer – Circular: Pipeline whose ends connect

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipeline example  Already looked at 2 distributed implementations of matrix multiplication a * b = c where n x n matrices are dense. 1.n workers; 1 worker per row of a but each worker stored all of b 2.n workers but each stored only one column of b  n workers, each produces 1 row of c  workers have none of a or b to start  connected in a closed pipeline thru which they acquire the data items and pass the results

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipeline example (cont)  Coordinator – sends every row of a and every column of b down the pipeline to the first worker – eventually receives every row of c from the last worker  Each worker – receives rows of a, – keeps the first it receives – passes the subsequent rows down – receives columns of b, passes them and computing one inner product – sends its row of c to next, then receives and passes on rows of c from previous workers in the pipeline – last worker sends its row of c and others it receives to coordinator

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipes and Filters Pattern  POSA 1 – under Architectural Patterns  Provides a structure for systems that process a stream of data.  Each processing step is encapsulated in a filter component.  Data is passed through pipes between adjacent filters.  Recombining filters allows you to build families of related systems.

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipes and Filters -- forces  Be able to enhance system by exchanging processing steps, recombining steps  Small processing steps are easier to reuse in different contexts  Non-adjacent processing steps do not share information  Different sources of input data exist  Possible to present or store final results in various ways  Explicit storage of intermediate results in files clutters directories and is error-prone  May not want to rule out multi-processing steps

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipes and Filters - solution  Divides the task into several sequential processing steps – Connected by a data flow; output of a step is input to the next – Each step is implemented by a filter component – A filter consumes and delivers data incrementally to achieve low latency and enable real parallel processing – Input is a data source; Output is a data sink. – Input, filters, output connected by pipes which implement the data flow between the steps

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipes and Filters - Scenarios

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipes and Filters – Scenarios

10/6/2004ECEN5053 SW Eng of Dist. Sys, Univ of Colorado, Boulder Pipes and Filters – Benefits & Liabilities  Benefits – No intermediate files necessary but possible – Flexibility by filter exchange – Flexibility by recombination – Reuse of filter components – Rapid prototyping of pipelines – Efficiency by parallel processing  Liabilities – Sharing state information is expensive or inflexible – Efficiency gain by parallel processing may be illusion – Data transformation overhead – Error handling – Achilles heel