Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction to Parallel Processing

Similar presentations


Presentation on theme: "Introduction to Parallel Processing"— Presentation transcript:

1 Introduction to Parallel Processing
CS 147 November 12, 2004 Johnny Lai

2 Multi-Processor Computing System
Computing Elements Applications Programming paradigms P . Microkernel Multi-Processor Computing System Threads Interface Operating System Hardware Process Processor Thread P

3 Two Eras of Computing Architectures System Software/Compiler
Applications P.S.Es System Software Sequential Era Parallel Era Commercialization R & D Commodity

4 History of Parallel Processing
PP can be traced to a tablet dated around 100 BC. Tablet has 3 calculating positions. Infer that multiple positions: Reliability/ Speed

5 Why Parallel Processing?
Computation requirements are ever increasing -- visualization, distributed databases, simulations, scientific prediction (earthquake), etc. Sequential architectures reaching physical limitation (speed of light, thermodynamics)

6 Human Architecture! Growth Performance
Vertical Horizontal Growth Age

7 Computational Power Improvement
Multiprocessor C.P.I. Uniprocessor No. of Processors

8 Why Parallel Processing?
The Tech. of PP is mature and can be exploited commercially; significant R & D work on development of tools & environment. Significant development in Networking technology is paving a way for heterogeneous computing.

9 Why Parallel Processing?
Hardware improvements like Pipelining, Superscalar, etc., are non-scalable and requires sophisticated Compiler Technology. Vector Processing works well for certain kind of problems.

10 Parallel Program has & needs ...
Multiple “processes” active simultaneously solving a given problem, general multiple processors. Communication and synchronization of its processes (forms the core of parallel programming efforts).

11 Processing Elements Architecture

12 Processing Elements Simple classification by Flynn:
(No. of instruction and data streams) SISD - conventional SIMD - data parallel, vector computing MISD - systolic arrays MIMD - very general, multiple approaches. Current focus is on MIMD model, using general purpose processors. (No shared memory) There are many different ways to organize the processors and memory within a multiprocessor system, and different ways to classify these system

13 SISD : A Conventional Computer
Processor Data Input Data Output Instructions Speed is limited by the rate at which computer can transfer information internally. Ex:PC, Macintosh, Workstations

14 The MISD Architecture Data Input Stream Output Processor A B C Instruction Stream A Stream B Instruction Stream C More of an intellectual exercise than a practicle configuration. Few built, but commercially not available

15 SIMD Architecture Instruction Stream Processor A B C Data Input stream A stream B stream C Data Output Ci<= Ai * Bi Ex: CRAY machine vector processing, Thinking machine cm* Intel MMX (multimedia support)

16 MIMD Architecture Instruction Stream A Instruction Stream B Instruction Stream C Data Output stream A Data Input stream A Processor A Data Output stream B Data Input stream B Processor B Data Output stream C Processor C Data Input stream C Unlike SISD, MISD, MIMD computer works asynchronously. Shared memory (tightly coupled) MIMD Distributed memory (loosely coupled) MIMD

17 Shared Memory MIMD machine
Processor A Processor B Processor C MEMORY BUS MEMORY BUS MEMORY BUS Global Memory System Comm: Source PE writes data to GM & destination retrieves it Easy to build, conventional OSes of SISD can be easily be ported Limitation : reliability & expandibility. A memory component or any processor failure affects the whole system. Increase of processors leads to memory contention. Ex. : Silicon graphics supercomputers....

18 Distributed Memory MIMD
IPC channel IPC channel Processor A Processor B Processor C MEMORY BUS MEMORY BUS MEMORY BUS Memory System A System B System C Communication : IPC on High Speed Network. Network can be configured to ... Tree, Mesh, Cube, etc. Unlike Shared MIMD easily/ readily expandable Highly reliable (any CPU failure does not affect the whole system)

19 Laws of caution..... Speed of computers is proportional to the square of their cost. i.e. cost = Speed Speedup by a parallel computer increases as the logarithm of the number of processors. Speedup = log2(no. of processors) C S (speed = cost2) S P log2P

20 Caution.... Very fast development in PP and related area have blurred concept boundaries, causing lot of terminological confusion : concurrent computing/ programming, parallel computing/ processing, multiprocessing, distributed computing, etc.

21 It’s hard to imagine a field that changes as rapidly as computing.

22 Caution.... Even well-defined distinctions like shared memory and distributed memory are merging due to new advances in technolgy. Good environments for developments and debugging are yet to emerge.

23 Caution.... There is no strict delimiters for contributors to the area of parallel processing : CA,OS, HLLs, databases, computer networks, all have a role to play. This makes it a Hot Topic of Research

24 Types of Parallel Systems
Shared Memory Parallel Smallest extension to existing systems Program conversion is incremental Distributed Memory Parallel Completely new systems Programs must be reconstructed Clusters Slow communication form of Distributed

25 Operating Systems for PP
MPP systems having thousands of processors requires OS radically different fromcurrent ones. Every CPU needs OS : to manage its resources to hide its details Traditional systems are heavy, complex and not suitable for MPP

26 Operating System Models
Frame work that unifies features, services and tasks performed Three approaches to building OS.... Monolithic OS Layered OS Microkernel based OS Client server OS Suitable for MPP systems Simplicity, flexibility and high performance are crucial for OS.

27 Monolithic Operating System
Application Programs System Services Hardware User Mode Kernel Mode Better application Performance Difficult to extend Ex: MS-DOS

28 Memory & I/O Device Mgmt
Layered OS Application Programs Application Programs User Mode Kernel Mode System Services Memory & I/O Device Mgmt Process Schedule Hardware Easier to enhance Each layer of code access lower level interface Low-application performance Ex : UNIX

29 Traditional OS OS User Mode Kernel Mode OS Designer Application
Programs User Mode Kernel Mode OS Hardware OS Designer

30 New trend in OS design Servers Microkernel User Mode Kernel Mode
Application Programs Application Programs User Mode Kernel Mode Microkernel Hardware

31 Microkernel/Client Server OS (for MPP Systems)
Application Thread lib. File Server Network Server Display Server User Kernel Microkernel Send Reply Hardware Tiny OS kernel providing basic primitive (process, memory, IPC) Traditional services becomes subsystems Monolithic Application Perf. Competence OS = Microkernel + User Subsystems Ex: Mach, PARAS, Chorus, etc.

32 Few Popular Microkernel Systems
MACH, CMU PARAS, C-DAC Chorus QNX, (Windows)

33 Reference http://www.cs.mu.oz.au http://www.whatis.com
Computer System Organization & Architecture John D. Carpinelli (^_^)


Download ppt "Introduction to Parallel Processing"

Similar presentations


Ads by Google