Multi-Processing in High Performance Computer Architecture:

Slides:



Advertisements
Similar presentations
Distributed Systems CS
Advertisements

SE-292 High Performance Computing
Multiprocessors— Large vs. Small Scale Multiprocessors— Large vs. Small Scale.
WHAT IS AN OPERATING SYSTEM? An interface between users and hardware - an environment "architecture ” Allows convenient usage; hides the tedious stuff.
1 Distributed Computing Algorithms CSCI Distributed Computing: everything not centralized many processors.
1: Operating Systems Overview
OPERATING SYSTEM OVERVIEW
Operating Systems CS208. What is Operating System? It is a program. It is the first piece of software to run after the system boots. It coordinates the.
Lecture 37: Chapter 7: Multiprocessors Today’s topic –Introduction to multiprocessors –Parallelism in software –Memory organization –Cache coherence 1.
CS 8625 High Performance and Parallel, Dr. Hoganson Copyright © 2001, 2004, 2005, 2006, 2008, Dr. Ken Hoganson CS8625-June-2-08 Class Will Start Momentarily…
LIGO-G Z 8 June 2001L.S.Finn/LDAS Camp1 How to think about parallel programming.
LOGO OPERATING SYSTEM Dalia AL-Dabbagh
Operating System Review September 10, 2012Introduction to Computer Security ©2004 Matt Bishop Slide #1-1.
Multiple Processor Systems. Multiprocessor Systems Continuous need for faster and powerful computers –shared memory model ( access nsec) –message passing.
Chapter 2 Parallel Architecture. Moore’s Law The number of transistors on a chip doubles every years. – Has been valid for over 40 years – Can’t.
Multiprocessing. Going Multi-core Helps Energy Efficiency William Holt, HOT Chips 2005 Adapted from UC Berkeley "The Beauty and Joy of Computing"
Operating Systems. Definition An operating system is a collection of programs that manage the resources of the system, and provides a interface between.
Data Management for Decision Support Session-4 Prof. Bharat Bhasker.
McGraw-Hill©The McGraw-Hill Companies, Inc., 2000 OS 1.
3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 1.
Background Computer System Architectures Computer System Software.
Introduction Goal: connecting multiple computers to get higher performance – Multiprocessors – Scalability, availability, power efficiency Job-level (process-level)
1.3 Operating system services An operating system provide services to programs and to the users of the program. It provides an environment for the execution.
Multi-Core CPUs Matt Kuehn. Roadmap ► Intel vs AMD ► Early multi-core processors ► Threads vs Physical Cores ► Multithreading and Multi-core processing.
INTRODUCTION TO HIGH PERFORMANCE COMPUTING AND TERMINOLOGY.
These slides are based on the book:
Auburn University COMP8330/7330/7336 Advanced Parallel and Distributed Computing Parallel Hardware Dr. Xiao Qin Auburn.
Modeling Big Data Execution speed limited by: Model complexity
A Level Computing – a2 Component 2 1A, 1B, 1C, 1D, 1E.
Introduction to Parallel Computing: MPI, OpenMP and Hybrid Programming
Multiprocessing.
18-447: Computer Architecture Lecture 30B: Multiprocessors
Lecture 5 Approaches to Concurrency: The Multiprocessor
Introduction to parallel programming
CS5102 High Performance Computer Systems Thread-Level Parallelism
Distributed Processors
For Massively Parallel Computation The Chaotic State of the Art
Introduction
Mobile Operating System
CS 147 – Parallel Processing
Introduction to Parallelism.
Operating Systems (CS 340 D)
Multi-Processing in High Performance Computer Architecture:
CMSC 611: Advanced Computer Architecture
Parallel and Multiprocessor Architectures – Shared Memory
Chapter 4: Threads.
Intro to Architecture & Organization
Chapter 17 Parallel Processing
ECE/CS 757: Advanced Computer Architecture II
Coe818 Advanced Computer Architecture
Characteristics of Reconfigurable Hardware
Introduction to Multiprocessors
What is Concurrent Programming?
AN INTRODUCTION ON PARALLEL PROCESSING
Distributed Systems CS
Distributed Systems CS
Hybrid Programming with OpenMP and MPI
COMP60621 Fundamentals of Parallel and Distributed Systems
Multithreaded Programming
What is Concurrent Programming?
Operating Systems (CS 340 D)
Distributed Computing:
Operating System Introduction.
The Challenge of Cross - Language Interoperability
Chapter 4 Multiprocessors
Introduction to Operating Systems
The University of Adelaide, School of Computer Science
Operating System Overview
COMP60611 Fundamentals of Parallel and Distributed Systems
Types of Parallel Computers
Presentation transcript:

Multi-Processing in High Performance Computer Architecture:

What is Multiprocessing? Enables several programs to run concurrently Coordinated processing of Programs by more than one processor Use of 2 or more CPUs within a single computer system Ability of a system to support more than one processor and to allocate tasks between them

Idealism (Target for Processor Performance):

Memory Hierarchies:

Memory in Modern Processor (L1 Cache?):

Why Not Uni-Processors: Making a wider Processor can efficiently run parallel programs but not programs that have dependencies In Uni-Processor, the instructions a = b + c; d = e + f can be run in parallel because non of the results depend on other calculations The instructions b = e + f; a = b + c, might not run in parallel due to data dependencies. There are several more complex data dependencies. Parallel part : Fast, One at a time(Stalls) : Slow

Why Multi-Processors:

Multi-Processor needs Parallel Programs(Years to Develop):

Centralized Shared Memory: (UMA) Distance from memory to each core is approximately same(UMA) Replicate Core and Caches to build a Multi Dimensional System(SMP)

Issues in Centralized Main Memory:

Distributed Memory: Each Core has it’s own local memory and cache forming a single core computer system. A network interface card, connected to a network of interconnections Cache miss goes directly to the local processor’s memory and if needed it can only access the other processor’s memory through the network message passing. To communicate, data is sent EXPLICITLY to the core. Programmer is forced to be aware of communication between cores and try to minimize it. Think of it as a set of machines communicating over a network.

Distributed (Multicomputer) Memory System(NUMA):

Shared Memory vs Message Passing (Hardware vs Software):

Simple Implementation of Summing Array in Parallel on 4 cores (Message Passing):

A Simple Shared Memory Implementation:

Performance Metrics (Message Passing vs Shared Memory):

Multithreading as Shared Memory Hardware:

Analyzing Multithreading Performance:

Summary: Multithreaded Categories