Programming Distributed Systems with High Level Abstractions Douglas Thain University of Notre Dame 23 October 2008

Distributed Systems Scale: 2 – 100s – 1000s – millions Domains:Single or Multi Users: 1 – 10 – 100 – 1000 – Naming:Direct, Virtual Scheduling:Timesharing / Space Sharing Interface:Allocate CPU / Execute Job Security:None / IP / PKI / KRB … Storage: Embedded / External

Cloud Computing? Scale: 2 – 100s – 1000s – 10000s Domains:Single or Multi Users: 1 – 10 – 100 – 1000 – Naming:Direct, Virtual Scheduling:Timesharing / Spacesharing Interface:Allocate CPU / Execute Job Security:None / IP / PKI / KRB … Storage: Embedded / External

Grid Computing? Scale: 2 – 100s – 1000s – 10000s Domains:Single or Multi Users: 1 – 10 – 100 – 1000 – Naming:Direct, Virtual Scheduling:Timesharing / Spacesharing Interface:Allocate CPU / Execute Job Security:None / IP / PKI / KRB … Storage: Embedded / External

An Assembly Language of Distributed Computing Fundamental Operations –TransferFile( source, destination ) –ExecuteJob( host, exe, input, output ) –AllocateVM( cpu, mem, disk, opsys ) Semantics of Assembly are Subtle: –When do instructions commit? –Delay slots before control transfers? –What exceptions are valid for each opcode? –Precise or imprecise exceptions? –What is the cost of each instruction?

Programming in Assembly Stinks You know the problems: –Stack management. –Garbage collection. –Type checking. –Co-location of data and computation. –Query optimizations. –Function shipping or data shipping? –How many nodes should I harness?

Abstractions for Distributed Computing Abstraction: a declarative specification of the computation and data of a workload. A restricted pattern, not meant to be a general purpose programming language. Avoid the really terrible cases. Provide users with a bright path. Data structures instead of file systems.

All-Pairs Abstraction AllPairs( set A, set B, function F ) returns matrix M where M[i][j] = F( A[i], B[j] ) for all i,j B1 B2 B3 A1A2A3 FFF A1 An B1 Bn F AllPairs(A,B,F) F FF FF F Moretti, Bulosan, Flynn, Thain, AllPairs: An Abstraction… IPDPS 2008

Example Application Goal: Design robust face comparison function. F 0.05 F 0.97

Similarity Matrix Construction F Current Workload: 4000 images 256 KB each 10s per F (five days) Future Workload: images 1MB each 1s per F (three months)

Non-Expert User Using 500 CPUs Try 1: Each F is a batch job. Failure: Dispatch latency >> F runtime. HN CPU FFFF F Try 2: Each row is a batch job. Failure: Too many small ops on FS. HN CPU FFFF F F F F F F F F F F F F F F F F Try 3: Bundle all files into one package. Failure: Everyone loads 1GB at once. HN CPU FFFF F F F F F F F F F F F F F F F F Try 4: User gives up and attempts to solve an easier or smaller problem.

All-Pairs Abstraction AllPairs( set A, set B, function F ) returns matrix M where M[i][j] = F( A[i], B[j] ) for all i,j B1 B2 B3 A1A2A3 FFF A1 An B1 Bn F AllPairs(A,B,F) F FF FF F

What is the right metric? Speedup? –Seq Runtime / Parallel Runtime Parallel Efficiency? –Speedup / N CPUs? Neither works, because the number of CPUs varies over time and between runs. Cost Efficiency –Work Completed / Resources Consumed –Person-Miles / Gallon –Results / CPU-hours –Results / $$$

All-Pairs Abstraction

T2 Classify Abstraction Classify( T, R, N, P, F ) T = testing setR = training set N = # of partitionsF = classifier P T1 T3 F F F T R V1 V2 V3 CV Moretti, Steinhauser, Thain, Chawla, Scaling up Classifiers to Cloud Computers, ICDM 2008.

BXGrid Abstractions B1 B2 B3 A1A2A3 FFF F FF FF F Lbrown Lblue Rbrown R S1 S2 S3 eyecolor F F F ROC Curve S = Select( color=“brown” ) B = Transform( S,F ) M = AllPairs( A, B, F ) Bui, Thomas, Kelly, Lyon, Flynn, Thain BXGrid: A Repository and Experimental Abstraction… in review 2008.

Implementing Abstractions S = Select( color=“brown” ) B = Transform( S,F ) M = AllPairs( A, B, F ) DBMS Relational Database (2x) Active Storage Cluster (16x) CPU Relational Database CPU Condor Pool (500x)

Compatibility of Abstractions? Assembly Language Map-ReduceAll-PairsClassify

Compatibility of Abstractions? Assembly Language Map-Reduce All-Pairs Classify ??? Mismatch: MR relies on data partition. AP relies on data re-use. Mismatch: Classify partitions logically. MR partitions physically.

Compatibility of Abstractions? Assembly Language Map-ReduceAll-PairsClassify SwiftDryad More General, Less Optimized?

From Clouds to Multicore Next Step: AP Implementation that runs well on Single CPU, Multicore, Cloud, or Cloud of Multicores. Assembly Language Map-ReduceAll-PairsClassify DryadSwift CPU Assembly Language Map-ReduceAll-PairsClassify DryadSwift CPU $$$ RAM

Acknowledgments Cooperative Computing Lab – Grad Students: –Chris Moretti –Hoang Bui –Michael Albrecht –Li Yu NSF Grants CCF , CNS Undergraduate Students –Mike Kelly –Rory Carmichael –Mark Pasquier –Christopher Lyon –Jared Bulosan