2015/10/14Part-I1 Introduction to Parallel Processing.

Slides:



Advertisements
Similar presentations
Prepared 7/28/2011 by T. O’Neil for 3460:677, Fall 2011, The University of Akron.
Advertisements

SE-292 High Performance Computing
Khaled A. Al-Utaibi  Computers are Every Where  What is Computer Engineering?  Design Levels  Computer Engineering Fields  What.
Parallel Programming Yang Xianchun Department of Computer Science and Technology Nanjing University Introduction.
Parallel Computers Chapter 1
Introduction to MIMD architectures
1 COMP 206: Computer Architecture and Implementation Montek Singh Mon, Dec 5, 2005 Topic: Intro to Multiprocessors and Thread-Level Parallelism.
Room: E-3-31 Phone: Dr Masri Ayob TK 2123 COMPUTER ORGANISATION & ARCHITECTURE Lecture 4: Computer Performance.
Multiprocessors ELEC 6200: Computer Architecture and Design Instructor : Agrawal Name: Nam.

Introduction What is Parallel Algorithms? Why Parallel Algorithms? Evolution and Convergence of Parallel Algorithms Fundamental Design Issues.
Multiprocessors CSE 471 Aut 011 Multiprocessors - Flynn’s Taxonomy (1966) Single Instruction stream, Single Data stream (SISD) –Conventional uniprocessor.
1 CSE SUNY New Paltz Chapter Nine Multiprocessors.
 Parallel Computer Architecture Taylor Hearn, Fabrice Bokanya, Beenish Zafar, Mathew Simon, Tong Chen.
CPE 731 Advanced Computer Architecture Multiprocessor Introduction
1 Computer Science, University of Warwick Architecture Classifications A taxonomy of parallel architectures: in 1972, Flynn categorised HPC architectures.
Introduction to Parallel Processing Ch. 12, Pg
CPU Performance Assessment As-Bahiya Abu-Samra *Moore’s Law *Clock Speed *Instruction Execution Rate - MIPS - MFLOPS *SPEC Speed Metric *Amdahl’s.
Advanced Computer Architectures
Computer performance.
1 Parallel computing and its recent topics. 2 Outline 1. Introduction of parallel processing (1)What is parallel processing (2)Classification of parallel.
High-Performance Computing 12.1: Concurrent Processing.
Computer Science and Engineering Copyright by Hesham El-Rewini Advanced Computer Architecture.
1 Chapter 1 Parallel Machines and Computations (Fundamentals of Parallel Processing) Dr. Ranette Halverson.
Previously Fetch execute cycle Pipelining and others forms of parallelism Basic architecture This week we going to consider further some of the principles.
Department of Computer Science University of the West Indies.
Chapter 2 Parallel Architecture. Moore’s Law The number of transistors on a chip doubles every years. – Has been valid for over 40 years – Can’t.
Computer Science and Engineering Parallel and Distributed Processing CSE 8380 February Session 6.
Parallel Computing.
CS591x -Cluster Computing and Parallel Programming
CPS 4150 Computer Organization Fall 2006 Ching-Song Don Wei.
Pipelining and Parallelism Mark Staveley
Winter 2014Parallel Processing, Fundamental ConceptsSlide 1 Part I Fundamental Concepts.
Data Structures and Algorithms in Parallel Computing Lecture 1.
Outline Why this subject? What is High Performance Computing?
Lecture 3: Computer Architectures
3/12/2013Computer Engg, IIT(BHU)1 INTRODUCTION-1.
3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 3.
An Overview of Parallel Processing
LECTURE #1 INTRODUCTON TO PARALLEL COMPUTING. 1.What is parallel computing? 2.Why we need parallel computing? 3.Why parallel computing is more difficult?
CDA-5155 Computer Architecture Principles Fall 2000 Multiprocessor Architectures.
Computer Science and Engineering Parallel and Distributed Processing CSE 8380 April 28, 2005 Session 29.
CS203 – Advanced Computer Architecture Performance Evaluation.
Classification of parallel computers Limitations of parallel processing.
Lecture 13 Parallel Processing. 2 What is Parallel Computing? Traditionally software has been written for serial computation. Parallel computing is the.
Computer Architecture Furkan Rabee
These slides are based on the book:
CS203 – Advanced Computer Architecture
Auburn University COMP8330/7330/7336 Advanced Parallel and Distributed Computing Parallel Hardware Dr. Xiao Qin Auburn.
Introduction to Parallel Processing
Flynn’s Taxonomy Many attempts have been made to come up with a way to categorize computer architectures. Flynn’s Taxonomy has been the most enduring of.
Introduction to Parallel Computing: MPI, OpenMP and Hybrid Programming
Advanced Architectures
CMSC 611: Advanced Computer Architecture
buses, crossing switch, multistage network.
What Exactly is Parallel Processing?
CS 147 – Parallel Processing
Flynn’s Classification Of Computer Architectures
Architecture & Organization 1
What is Parallel and Distributed computing?
MIMD Multiple instruction, multiple data
Architecture & Organization 1
Chapter 17 Parallel Processing
Multiprocessors - Flynn’s taxonomy (1966)
Course Description: Parallel Computer Architecture
buses, crossing switch, multistage network.
Overview Parallel Processing Pipelining
AN INTRODUCTION ON PARALLEL PROCESSING
Computer Evolution and Performance
Chapter 4 Multiprocessors
Presentation transcript:

2015/10/14Part-I1 Introduction to Parallel Processing

2015/10/14Part-I2 Preface It has allowed hardware performance to continue its exponential growth. This trend is expected to continue in the near future. It has led to unprecedented hardware complexity and almost intolerable development costs. In computer designers' quest for user-friendliness, compactness, simplicity, high per­formance, low cost, and low power, parallel processing plays a key role. –High-performance uniprocessors are becoming increasingly complex, expensive, and power-hungry.

2015/10/14Part-I3 Introduction to Parallelism WHY PARALLEL PROCESSING? –In the past two decades, the performance of microprocessors has enjoyed an exponential growth. (a factor of 2 every 18 months, Moore's law) Increase in complexity of VLSI chips Introduction of, and improvements in, architectural features –Moore's law seems to hold regardless of how one measures processor performance: counting the number of executed instructions per second (IPS), counting the number of floating-point operations per second (FLOPS), or using sophisticated benchmark suites that attempt to measure the processor's performance on real applications.

2015/10/14Part-I4

2015/10/14Part-I5 WHY PARALLEL PROCESSING? (cont’d) physical laws –The most easily understood physical limit is that imposed by the finite speed of signal propagation along a wire. This is sometimes referred to as the speed-of-light argument. –pipelining and memory-latency-hiding techniques. –The speed-of-light argument suggests that once the above limit has been reached, the only path to improved performance is the use of multiple processors. (the same argument can be invoked to conclude that any parallel processor)

2015/10/14Part-I6

2015/10/14Part-I7

2015/10/14Part-I8 WHY PARALLEL PROCESSING? (cont’d) Who needs supercomputers with TFLOPS or PFLOPS performance? The motivations for parallel processing can be summarized as follows: –Higher speed, or solving problems faster. –Higher throughput, or solving more instances of given problems. –Higher computational power, or solving larger problems. speed-up factor This book focuses on the interplay of architectural and algorithmic speed-up techniques.

2015/10/14Part-I9

2015/10/14Part-I10 A MOTIVATING EXAMPLE A major issue in devising a parallel algorithm for a given problem is the way in which the computational load is divided between the multiple processors. Problem: Prime number finding

2015/10/14Part-I11 Prime number finding Single Processor Multiprocessors (a possible solution, share memory)

2015/10/14Part-I12

2015/10/14Part-I13 Prime number finding (cont’d.) Multiprocessors (data parallel approach, distributed memory)

2015/10/14Part-I14

2015/10/14Part-I15 PARALLEL PROCESSING UPS AND DOWNS Parallel processing, in the literal sense of the term, is used in virtually every modern computer. –overlap between instruction preparation and execution in a pipelined processor. –multiple functional units –multitasking –very-long-instruction-word (VLIW) computers In this book, the term parallel processing is used in a restricted sense of having multiple (usually identical) processors for the main computation and not for the I/O or other peripheral activities.

2015/10/14Part-I16 The history of parallel processing The history of parallel processing has had its ups and downs with what appears to be a 20-year cycle. –commercial

2015/10/14Part-I17 TYPES OF PARALLELISM: A TAXONOMY Parallel computers can be divided into two main categories of control flow and data flow. –Control-flow parallel computers are essentially based on the same principles as the sequential or von Neumann computer. –Data-flow parallel computers, sometimes referred to as "non-von Neumann“ (DNA computer) In 1966, M. 1. Flynn proposed a four-way classification of computer systems based on the notions of instruction streams and data streams.

2015/10/14Part-I18

2015/10/14Part-I19

2015/10/14Part-I20 Flynn classification The MIMD category includes a wide class of computers. For this reason, in 1988, E. E. Johnson proposed a further classification of such machines based on their –memory structure (global or distributed) and –the mechanism used for communication/synchronization (shared variables or message passing). SPMD and MPMD CISC, NUMA, PRAM, RISC, and VLIW.

2015/10/14Part-I21 Uniform memory access Cache only Share memory architecture

2015/10/14Part-I22 Distrusted memory architecture

2015/10/14Part-I23

2015/10/14Part-I24

2015/10/14Part-I25

2015/10/14Part-I26

2015/10/14Part-I27

2015/10/14Part-I28

2015/10/14Part-I29

2015/10/14Part-I30 Multistage interconnection network

2015/10/14Part-I31

2015/10/14Part-I32 ROADBLOCKS TO PARALLEL PROCESSING Grosch's law (computing power is proportional to the square of cost) Minsky's conjecture (speed-up is proportional to the logarithm of the number p of processors) The tyranny of Ie technology (uniprocessors will be just as fast) The tyranny of vector supercomputers (why bother with parallel processors?) The software inertia (billions of dollars worth of existing software) Amdahl's law

2015/10/14Part-I33

2015/10/14Part-I34 EFFECTIVENESS Of PARALLEL PROCESSING

2015/10/14Part-I35