1. E-mail : ra.ramadan@uoh.edu.sa Web: http://www.rabieramadan.org
Parallel Computing
Rabie A. Ramadan , PhD
Web: 1 1 1

2. About the Course Course Title: Parallel Computing.
Course Number: COE 420
Credit Hours: 3
Time of Lectures:
8---9:40 .
Room:

3. Description of Course This course will provide
An introduction to parallel computing.
Different models in parallelism.
Different measurements in parallel computing.
Different models of parallel programming.
Different methods of paralyzing the problems

4. Objectives of Course Be familiar with the importance of parallelism.
Be familiar with a collection of parallel models (architectures).
Be able to evaluate the efficiency of parallel programs and solutions by using different measurements.
Be familiar with the different parallel programming models.
Be able to use an efficient technique to solve the problem by parallelism.

6. Contents of Course Introduction to parallel computing (chapter 1).
Parallel Models (chapter 2).
Performance Measurements(chapters 3).
Parallel Programming Models (chapter 4).
Parallel Techniques (chapter 5).
Problem Solving (chapter 6). 6

7. References Parallel Processing and Parallel Algorithms Seyed H. Roosta
Springer, 2000
Lecture Notes by instructor.

8. Evaluation Total Marks: 100 Exam 1 (Major 1 ) 20%
Final Exam %
Quiz (3), Reports,
Homework, Attendance % 8

9. Homework Assignments
The deadline of the home work is ONE week after finishing the chapter.
The score of the homework will be decrease if you submit the home work after one week.
Can discuss in groups, but each student must turn in his/her own written solution.

10. How to Succeed in this Course
Be there in class.
Participate in class.
Ask the instructor if there is any misunderstand in the class.
Didn’t play in mobile or talk with your friends.
Review the lecture after each class or by the end of the week.
Be formal and precise on all problem sets, home work, in-class exams.

11. Accelerated Processing Units (APUs)

12. Part 1: Introduction Concept of Parallel Computing.
Goals of Parallel Computing.
Parallel Computer.
Applications of Parallelism.

13. Concept of Parallelism
First-View
What is the main idea for designing the computer?
simulate the person
data
processing
memory
mind
information/data
result/action/..
input data
output
output devices
CPU
input devices

14. sequence of instructions Objectives of this simulation
Sequential Computer
sequence of instructions
sequence of data
input unit
control unit
processor
memory
output unit
Objectives of this simulation
do the computation fast
running in long time without suspend/stop
does not effected by external affecting ….

15. required many specializations can you transfer these concepts to me.
In real life, the persons are cooperate to what:
dangerous
fast
big
required many specializations
unsolved ….
can you transfer these concepts to me.

16. CPU
CPU
CPU

17. required many specialization
dangerous
fast
big
required many specialization
unsolved ….

18. solve the problem on the computer
Second-View
computer
Problem
solve the problem on the computer
Stages
O.S.
architecture
language,
compiler
output
problem
algorithm
Is the problem solved in an efficient time or not?
If yes then thanks
How
If no then try to solve the problem fast.

19. increase the no. of processors O.S. language, compiler architecture
problem
algorithm
new/modify
optimization
technology
reducing the time.
Until !!!!!
Lower bound
no improvement
solution

20. First-View
Second-View
Parallel Computers

23. Why parallel computing?
(1)solve the large problem.
Example: Web search engines/databases processing millions of transactions per second
often bigger is more important than faster
(2) solve the problem fast.
Example: sorting problem takes O(n log n) . In parallelism we can sort n elements in O(log n) time

24. Why parallel computing?
(3) cost-effectiveness
(4) fault-tolerance
Fault tolerance is the property that enables a system to continue operating properly in the event of the failure of (or one or more faults within) some of its components. 

25. Why parallel computing?
(5) TAKE ADVANTAGE OF NON-LOCAL RESOURCES
Using compute resources on a wide area network, or even the Internet when local compute resources are scarce or insufficient.

26. Key enabling factors
(1) Advances in microprocessor

27. Key enabling factors
(2) Advances in interconnect technology.

28. Key enabling factors
(3) Advances in software technology.
Operating system
Programming languages (Compiler)

30. parallel computer
A parallel computer is a collection of processing elements that can
cooperate and communicate to solve large problem fast.
a collection of processing elements
How many?
tens/hundred/thousand
How powerful is each?
powerful/weak
Homogenous - heterogeneous
Few very powerful or many weak ones

31. Shared memory / message passing
that can communicate…
how do PEs communicate?
Shared memory / message passing
Interconnection
network p1 p2 p0
p11
p10 p9 p7 p6 p8 p3 p4 p5 p0 p1 pi
pi+1
pn-1
memory access unit
Shared memory
Interconnection network architecture?
bus / crossbar / multistage,…
Cost / latency / scalability / fault tolerance
Latency is the time it takes one message to travel from source to destination..
An architecture is scalable if it continues to yield the same performance per processor as the number of PEs increases.

32. General purpose machine Special purpose machine
and cooperate…
synchronization
autonomy
to solve large problem
General purpose machine
Special purpose machine
What applications are amenable to parallel processing:
Highly parallel applications
Monte-Carlo, Numerical computation
Medium parallel applications
VLSI-CAD
Not-so-parallel applications
Editors, compilers

33. Science
Global climate modeling
Astrophysical modeling
Biology: genomics; protein folding; drug design
Computational Chemistry
Computational Material Sciences and Nanosciences
Engineering
Crash simulation
Semiconductor design
Earthquake and structural modeling
Computation fluid dynamics (airplane design)
Combustion (engine design)
Business
Financial and economic modeling
Transaction processing, web services and search engines
Defense
Nuclear weapons -- test by simulations
Cryptography

36. When the problem is Complex, Large / small, Expensive, Dangerous