Summary Background Introduction in algorithms and applications Why do we need parallel processing? Moore’s law. Applications. Introduction in algorithms and applications Methodology to develop efficient parallel (distributed-memory) algorithms  Understand various forms of overhead (communication, load imbalance, search overhead, synchronization) Understand various distributions (blockwise, cyclic) Understand various load balancing strategies (static, dynamic master/worker model) Understand correctness problems (e.g. message ordering)

Summary Parallel machines and architectures Processor organizations, topologies, criteria Types of parallel machines arrays/vectors, shared-memory, distributed memory Routing Flynn’s taxonomy What are cluster computers? What networks do real machines (like the Blue Gene) use? Speedup, efficiency (+ their implications), Amdahl’s law

Summary Programming methods, languages, and environments Different forms of message passing naming, explicit/implicit receive, synchronous/asynchronous sending Select statement SR primitives (not syntax) MPI: message passing primitives, collective communication   Java parallel programming model and primitives HPF: problems with automatic parallelization; division of work between programmer and HPF compiler; alignment/distribution primitives; performance implications

Summary Applications Grid computing N-body problems: load balancing and communication (locality) optimizations, costzones, performance comparison Search algorithm (TDS): use asynchronous communication + clever (transposition-driven) scheduling Grid computing What are grids? Why do we need them? Application-level tools for grid programming and execution Why is parallel computing on grids feasible? Ibis: goals, applications, design, programming vs. deployment, communication primitives, divide&conquer parallelism, performance behavior on a grid

Summary Multimedia content analysis on Grids What is Multimedia Content Analysis (MMCA) ? Imaging applications, feature vectors, low-level patterns Why parallel computing in MMCA - and how? Need for speed and transparency Software Platform: Parallel-Horus. Task/data parallelism, Separable Recursive Filtering, Lazy Parallelization Example – Parallel Image Processing on Clusters Grids and their specific problems. Promise and problems of the grid Towards a Software Platform for MMCA on Grids. Problems with wide- area servers Large-scale MMCA applications on Grids (TRECVID, object recognition)

Summary Multi/many-cores processors Variety of platforms (core types & counts, memory architecture & sizes, parallelism layers & types, scheduling) Performance evaluation (arithmetic intensity, Roofline model) Programming models for multi/many-cores Deal with performance, portability, programmability The correlator application Programming models for Cell, GPUs, Generic models