Multi-core
What is parallel programming ? Classification of parallel architectures Dimension of instruction Dimension of data Memory models for parallel programming Distributed memory model Share memory model Multi-core architectures Steps in parallelization Jaruloj Chongstitvatana2Parallel Programming: Introduction
CONCURRENT Two or more actions progress at the same time. PARALLEL Two or more actions execute simultaneously. Jaruloj ChongstitvatanaParallel Programming: Introduction3 How to write a program that can execute correctly on two proccesors or more.
Design considerations Correctness Efficiency Simplicity Scalability Jaruloj ChongstitvatanaParallel Programming: Introduction4
Single Instruction Single Data: SISD Single Instruction Multiple Data: SIMD Multiple Instruction Single Data: MISD Multiple Instruction Multiple Data: MIMD Jaruloj ChongstitvatanaParallel Programming: Introduction5
No parallelism Jaruloj ChongstitvatanaParallel Programming: Introduction6 instructiondata
Vector machine Graphic processors Jaruloj ChongstitvatanaParallel Programming: Introduction7 instructiondata1data2data3
No application (at this time) Jaruloj ChongstitvatanaParallel Programming: Introduction8 instruction2data1 instruction1instruction3
Multi-core architectures Jaruloj ChongstitvatanaParallel Programming: Introduction9 instruction2data1data2data3 instruction1instruction3
SHARED MEMORY MODEL Multiple processors connect to one shared memory. All processors can access same memory locations. DISTRIBUTED MEMORY MODEL Each processor connects to its private memory. Jaruloj ChongstitvatanaParallel Programming: Introduction10 memory bus processor bus processor memory processor memory processor memory
SHARED MEMORY Bottleneck for memory access No data transfer Easy for share data DISTRIBUTED MEMORY Better memory access Data transfer from non- local memory Easy for private data Jaruloj ChongstitvatanaParallel Programming: Introduction11
AMD Multicore Opteron Sun UltraSparc T1 IBM Cell Broadband Engine (CBE) Intel Core2 Duo Jaruloj ChongstitvatanaParallel Programming: Introduction12
Jaruloj ChongstitvatanaParallel Programming: Introduction13 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Jaruloj ChongstitvatanaParallel Programming: Introduction14 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Jaruloj ChongstitvatanaParallel Programming: Introduction15 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Jaruloj ChongstitvatanaParallel Programming: Introduction16 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Jaruloj ChongstitvatanaParallel Programming: Introduction17 Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.
Starting from sequential code Identify possible concurrency Design and implement Test for correctness Tuning for performance May effect correctness If tuning is impossible, consider redesign Jaruloj ChongstitvatanaParallel Programming: Introduction18
Data race Deadlock Notes: Results might be slightly different for each run due to round-off error for different sequence of operations. Jaruloj ChongstitvatanaParallel Programming: Introduction19