1. Multi-core

2.  What is parallel programming ?  Classification of parallel architectures  Dimension of instruction  Dimension of data  Memory models for parallel programming  Distributed memory model  Share memory model  Multi-core architectures  Steps in parallelization Jaruloj Chongstitvatana2Parallel Programming: Introduction

3. CONCURRENT  Two or more actions progress at the same time. PARALLEL  Two or more actions execute simultaneously. Jaruloj ChongstitvatanaParallel Programming: Introduction3 How to write a program that can execute correctly on two proccesors or more.

4.  Design considerations  Correctness  Efficiency  Simplicity  Scalability Jaruloj ChongstitvatanaParallel Programming: Introduction4

5.  Single Instruction Single Data: SISD  Single Instruction Multiple Data: SIMD  Multiple Instruction Single Data: MISD  Multiple Instruction Multiple Data: MIMD Jaruloj ChongstitvatanaParallel Programming: Introduction5

6.  No parallelism Jaruloj ChongstitvatanaParallel Programming: Introduction6 instructiondata

7.  Vector machine  Graphic processors Jaruloj ChongstitvatanaParallel Programming: Introduction7 instructiondata1data2data3

8.  No application (at this time) Jaruloj ChongstitvatanaParallel Programming: Introduction8 instruction2data1 instruction1instruction3

9.  Multi-core architectures Jaruloj ChongstitvatanaParallel Programming: Introduction9 instruction2data1data2data3 instruction1instruction3

10. SHARED MEMORY MODEL  Multiple processors connect to one shared memory.  All processors can access same memory locations. DISTRIBUTED MEMORY MODEL  Each processor connects to its private memory. Jaruloj ChongstitvatanaParallel Programming: Introduction10 memory bus processor bus processor memory processor memory processor memory

11. SHARED MEMORY  Bottleneck for memory access  No data transfer  Easy for share data DISTRIBUTED MEMORY  Better memory access  Data transfer from non- local memory  Easy for private data Jaruloj ChongstitvatanaParallel Programming: Introduction11

12.  AMD Multicore Opteron  Sun UltraSparc T1  IBM Cell Broadband Engine (CBE)  Intel Core2 Duo Jaruloj ChongstitvatanaParallel Programming: Introduction12

13. Jaruloj ChongstitvatanaParallel Programming: Introduction13 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

14. Jaruloj ChongstitvatanaParallel Programming: Introduction14 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

15. Jaruloj ChongstitvatanaParallel Programming: Introduction15 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

16. Jaruloj ChongstitvatanaParallel Programming: Introduction16 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

17. Jaruloj ChongstitvatanaParallel Programming: Introduction17 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

18. Starting from sequential code  Identify possible concurrency  Design and implement  Test for correctness  Tuning for performance  May effect correctness  If tuning is impossible, consider redesign Jaruloj ChongstitvatanaParallel Programming: Introduction18

19.  Data race  Deadlock Notes:  Results might be slightly different for each run due to round-off error for different sequence of operations. Jaruloj ChongstitvatanaParallel Programming: Introduction19