E-mail : ra.ramadan@uoh.edu.sa Web: http://www.rabieramadan.org Parallel Computing Rabie A. Ramadan , PhD E-mail : ra.ramadan@uoh.edu.sa Web: http://www.rabieramadan.org 1 1 1

About the Course Course Title: Parallel Computing. Course Number: COE 420 Credit Hours: 3 Time of Lectures: 8---9:40 . Room: 11002.

Description of Course This course will provide An introduction to parallel computing. Different models in parallelism. Different measurements in parallel computing. Different models of parallel programming. Different methods of paralyzing the problems

Objectives of Course Be familiar with the importance of parallelism. Be familiar with a collection of parallel models (architectures). Be able to evaluate the efficiency of parallel programs and solutions by using different measurements. Be familiar with the different parallel programming models. Be able to use an efficient technique to solve the problem by parallelism.

Contents of Course Introduction to parallel computing (chapter 1). Parallel Models (chapter 2). Performance Measurements(chapters 3). Parallel Programming Models (chapter 4). Parallel Techniques (chapter 5). Problem Solving (chapter 6). 6

References Parallel Processing and Parallel Algorithms Seyed H. Roosta Springer, 2000 Lecture Notes by instructor.

Evaluation Total Marks: 100 Exam 1 (Major 1 ) 20% Final Exam 45 % Quiz (3), Reports, Homework, Attendance 15% 8

Homework Assignments The deadline of the home work is ONE week after finishing the chapter. The score of the homework will be decrease if you submit the home work after one week. Can discuss in groups, but each student must turn in his/her own written solution.

How to Succeed in this Course Be there in class. Participate in class. Ask the instructor if there is any misunderstand in the class. Didn’t play in mobile or talk with your friends. Review the lecture after each class or by the end of the week. Be formal and precise on all problem sets, home work, in-class exams.

Accelerated Processing Units (APUs)

Part 1: Introduction Concept of Parallel Computing. Goals of Parallel Computing. Parallel Computer. Applications of Parallelism.

Concept of Parallelism First-View What is the main idea for designing the computer? simulate the person data processing memory mind information/data result/action/.. input data output output devices CPU input devices

sequence of instructions Objectives of this simulation Sequential Computer sequence of instructions sequence of data input unit control unit processor memory output unit Objectives of this simulation do the computation fast running in long time without suspend/stop does not effected by external affecting ….

required many specializations can you transfer these concepts to me. In real life, the persons are cooperate to what: dangerous fast big required many specializations unsolved …. can you transfer these concepts to me.

CPU CPU CPU

required many specialization dangerous fast big required many specialization unsolved ….

solve the problem on the computer Second-View computer Problem solve the problem on the computer Stages O.S. architecture language, compiler output problem algorithm Is the problem solved in an efficient time or not? If yes then thanks How If no then try to solve the problem fast.

increase the no. of processors O.S. language, compiler architecture problem algorithm new/modify optimization technology reducing the time. Until !!!!! Lower bound no improvement solution

First-View Second-View Parallel Computers

http://rabieramadan.org/classes/2015-2016/COE420/

Why parallel computing? (1)solve the large problem. Example: Web search engines/databases processing millions of transactions per second often bigger is more important than faster (2) solve the problem fast. Example: sorting problem takes O(n log n) . In parallelism we can sort n elements in O(log n) time

Why parallel computing? (3) cost-effectiveness (4) fault-tolerance Fault tolerance is the property that enables a system to continue operating properly in the event of the failure of (or one or more faults within) some of its components. 

Why parallel computing? (5) TAKE ADVANTAGE OF NON-LOCAL RESOURCES Using compute resources on a wide area network, or even the Internet when local compute resources are scarce or insufficient.

Key enabling factors (1) Advances in microprocessor

Key enabling factors (2) Advances in interconnect technology.

Key enabling factors (3) Advances in software technology. Operating system Programming languages (Compiler)

parallel computer A parallel computer is a collection of processing elements that can cooperate and communicate to solve large problem fast. a collection of processing elements How many? tens/hundred/thousand How powerful is each? powerful/weak Homogenous - heterogeneous Few very powerful or many weak ones

Shared memory / message passing that can communicate… how do PEs communicate? Shared memory / message passing Interconnection network p1 p2 p0 p11 p10 p9 p7 p6 p8 p3 p4 p5 p0 p1 pi pi+1 pn-1 memory access unit Shared memory Interconnection network architecture? bus / crossbar / multistage,… Cost / latency / scalability / fault tolerance Latency is the time it takes one message to travel from source to destination.. An architecture is scalable if it continues to yield the same performance per processor as the number of PEs increases.

General purpose machine Special purpose machine and cooperate… synchronization autonomy to solve large problem General purpose machine Special purpose machine What applications are amenable to parallel processing: Highly parallel applications Monte-Carlo, Numerical computation Medium parallel applications VLSI-CAD Not-so-parallel applications Editors, compilers

Science Global climate modeling Astrophysical modeling Biology: genomics; protein folding; drug design Computational Chemistry Computational Material Sciences and Nanosciences Engineering Crash simulation Semiconductor design Earthquake and structural modeling Computation fluid dynamics (airplane design) Combustion (engine design) Business Financial and economic modeling Transaction processing, web services and search engines Defense Nuclear weapons -- test by simulations Cryptography

When the problem is Complex, Large / small, Expensive, Dangerous