Message Passing Computing 1 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing Lecture 1 High Performance Computing Helvi Hartmann FIAS Inverted CERN.

Slides:



Advertisements
Similar presentations
Gordon Bell Bay Area Research Center Microsoft Corporation
Advertisements

Prepared 7/28/2011 by T. O’Neil for 3460:677, Fall 2011, The University of Akron.
Distributed Systems CS
SE-292 High Performance Computing
Multiprocessors— Large vs. Small Scale Multiprocessors— Large vs. Small Scale.
CA 714CA Midterm Review. C5 Cache Optimization Reduce miss penalty –Hardware and software Reduce miss rate –Hardware and software Reduce hit time –Hardware.
Today’s topics Single processors and the Memory Hierarchy
Beowulf Supercomputer System Lee, Jung won CS843.
Types of Parallel Computers
Chapter 1 Parallel Computers.
Parallel Computers Chapter 1
CSCI-455/522 Introduction to High Performance Computing Lecture 2.
Multiprocessors ELEC 6200: Computer Architecture and Design Instructor : Agrawal Name: Nam.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd Edition, by B. Wilkinson & M. Allen, ©
Multiprocessors CSE 471 Aut 011 Multiprocessors - Flynn’s Taxonomy (1966) Single Instruction stream, Single Data stream (SISD) –Conventional uniprocessor.
1 CSE SUNY New Paltz Chapter Nine Multiprocessors.
IBM RS/6000 SP POWER3 SMP Jari Jokinen Pekka Laurila.
CPE 731 Advanced Computer Architecture Multiprocessor Introduction
Cluster Computing Slides by: Kale Law. Cluster Computing Definition Uses Advantages Design Types of Clusters Connection Types Physical Cluster Interconnects.
Lecture 1: Introduction to High Performance Computing.
1 Computer Science, University of Warwick Architecture Classifications A taxonomy of parallel architectures: in 1972, Flynn categorised HPC architectures.
Lecture 37: Chapter 7: Multiprocessors Today’s topic –Introduction to multiprocessors –Parallelism in software –Memory organization –Cache coherence 1.
Fundamental Issues in Parallel and Distributed Computing Assaf Schuster, Computer Science, Technion.
1b.1 Types of Parallel Computers Two principal approaches: Shared memory multiprocessor Distributed memory multicomputer ITCS 4/5145 Parallel Programming,
CS 470/570:Introduction to Parallel and Distributed Computing.
Computer System Architectures Computer System Software
High-Performance Computing 12.1: Concurrent Processing.
1b.1 Types of Parallel Computers Two principal approaches: Shared memory multiprocessor Distributed memory multicomputer ITCS 4/5145 Parallel Programming,
Parallel and Distributed Systems Instructor: Xin Yuan Department of Computer Science Florida State University.
Multi-core Programming Introduction Topics. Topics General Ideas Moore’s Law Amdahl's Law Processes and Threads Concurrency vs. Parallelism.
Data Warehousing 1 Lecture-24 Need for Speed: Parallelism Virtual University of Pakistan Ahsan Abdullah Assoc. Prof. & Head Center for Agro-Informatics.
1 BİL 542 Parallel Computing. 2 Parallel Programming Chapter 1.
Cluster Workstations. Recently the distinction between parallel and distributed computers has become blurred with the advent of the network of workstations.
Beowulf Cluster Jon Green Jay Hutchinson Scott Hussey Mentor: Hongchi Shi.
2015/10/14Part-I1 Introduction to Parallel Processing.
Chapter 2 Parallel Architecture. Moore’s Law The number of transistors on a chip doubles every years. – Has been valid for over 40 years – Can’t.
Multiprocessing. Going Multi-core Helps Energy Efficiency William Holt, HOT Chips 2005 Adapted from UC Berkeley "The Beauty and Joy of Computing"
Computer Science and Engineering Parallel and Distributed Processing CSE 8380 February Session 6.
Parallel Processing Steve Terpe CS 147. Overview What is Parallel Processing What is Parallel Processing Parallel Processing in Nature Parallel Processing.
1 CMPE 511 HIGH PERFORMANCE COMPUTING CLUSTERS Dilek Demirel İşçi.
Copyright © 2011 Curt Hill MIMD Multiple Instructions Multiple Data.
MODERN OPERATING SYSTEMS Third Edition ANDREW S. TANENBAUM Chapter 8 Multiple Processor Systems Tanenbaum, Modern Operating Systems 3 e, (c) 2008 Prentice-Hall,
M U N - February 17, Phil Bording1 Computer Engineering of Wave Machines for Seismic Modeling and Seismic Migration R. Phillip Bording February.
CS591x -Cluster Computing and Parallel Programming
Data Structures and Algorithms in Parallel Computing Lecture 1.
By Chi-Chang Chen.  Cluster computing is a technique of linking two or more computers into a network (usually through a local area network) in order.
Outline Why this subject? What is High Performance Computing?
COMP381 by M. Hamdi 1 Clusters: Networks of WS/PC.
3/12/2013Computer Engg, IIT(BHU)1 INTRODUCTION-2.
Message Passing Computing 1 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing Lecture 2 Message Passing Helvi Hartmann FIAS Inverted CERN School.
Computer Architecture Lecture 24 Parallel Processing Ralph Grishman November 2015 NYU.
CDA-5155 Computer Architecture Principles Fall 2000 Multiprocessor Architectures.
Introduction Goal: connecting multiple computers to get higher performance – Multiprocessors – Scalability, availability, power efficiency Job-level (process-level)
Constructing a system with multiple computers or processors 1 ITCS 4/5145 Parallel Programming, UNC-Charlotte, B. Wilkinson. Jan 13, 2016.
These slides are based on the book:
Introduction to Parallel Processing
Lynn Choi School of Electrical Engineering
Lecture 5 Approaches to Concurrency: The Multiprocessor
CS5102 High Performance Computer Systems Thread-Level Parallelism
Constructing a system with multiple computers or processors
CS775: Computer Architecture
What is Parallel and Distributed computing?
Introduction to Multiprocessors
Constructing a system with multiple computers or processors
Constructing a system with multiple computers or processors
Constructing a system with multiple computers or processors
By Brandon, Ben, and Lee Parallel Computing.
Chapter 4 Multiprocessors
Types of Parallel Computers
Husky Energy Chair in Oil and Gas Research
Presentation transcript:

Message Passing Computing 1 iCSC2015,Helvi Hartmann, FIAS Message Passing Computing Lecture 1 High Performance Computing Helvi Hartmann FIAS Inverted CERN School of Computing, February 2015

Message Passing Computing 2 iCSC2015,Helvi Hartmann, FIAS What is High Performance Computing?

Message Passing Computing 3 iCSC2015,Helvi Hartmann, FIAS Cluster Computing parallel computing on systems with distributed memory

Message Passing Computing 4 iCSC2015,Helvi Hartmann, FIAS Why do we need it?

Message Passing Computing 5 iCSC2015,Helvi Hartmann, FIAS Example: Weather Forecast

Message Passing Computing 6 iCSC2015,Helvi Hartmann, FIAS Grand Challenges

Message Passing Computing 7 iCSC2015,Helvi Hartmann, FIAS Grand Challenges

Message Passing Computing 8 iCSC2015,Helvi Hartmann, FIAS Big Data

Message Passing Computing 9 iCSC2015,Helvi Hartmann, FIAS Why not make a single fast computer? processing operation/s instead of 10 9 operation/s

Message Passing Computing 10 iCSC2015,Helvi Hartmann, FIAS Memory Wall 1 instruction in 4ns memory 27cycles + 40ns = 148ns

Message Passing Computing 11 iCSC2015,Helvi Hartmann, FIAS Memory Wall 1 instruction in 4ns L1 Cache 4 cycles = 16ns L2 Cache 12 cycles = 48ns L3 Cache 27 cycles = 108ns memory = 148ns Caches —> located directly next to the CPU to reduce access time - small space available - expensive in power usage

Message Passing Computing 12 iCSC2015,Helvi Hartmann, FIAS Moore‘s Law Amount of transistor on a chip will double every second year —> „computerspeed doubles every second year“ Use of multiprocessors

Message Passing Computing 13 iCSC2015,Helvi Hartmann, FIAS Microarchitecture PipeliningMultithreadingPrefetching CachingOut-of-order execution Vectorization Only small gains as compared to if Moore‘s law remained valid

Message Passing Computing 14 iCSC2015,Helvi Hartmann, FIAS Shared Memory

Message Passing Computing 15 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall

Message Passing Computing 16 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall Cache Coherency

Message Passing Computing 17 iCSC2015,Helvi Hartmann, FIAS

Message Passing Computing 18 iCSC2015,Helvi Hartmann, FIAS Shared Memory Parallel Processing Memory Wall Cache Coherency Expensive!!!

Message Passing Computing 19 iCSC2015,Helvi Hartmann, FIAS What is possible except improving the Microarchitecture?

Message Passing Computing 20 iCSC2015,Helvi Hartmann, FIAS Cluster Computing

Message Passing Computing 21 iCSC2015,Helvi Hartmann, FIAS Distributed Memory

Message Passing Computing 22 iCSC2015,Helvi Hartmann, FIAS Shared Memory Cheap Standard processor very powerful Scalable High Latencies

Message Passing Computing 23 iCSC2015,Helvi Hartmann, FIAS High Latencies? Detour: Network Topologies

Message Passing Computing 24 iCSC2015,Helvi Hartmann, FIAS

Message Passing Computing 25 iCSC2015,Helvi Hartmann, FIAS Does it work any easier?

Message Passing Computing 26 iCSC2015,Helvi Hartmann, FIAS switches

Message Passing Computing 27 iCSC2015,Helvi Hartmann, FIAS Parallelization

Message Passing Computing 28 iCSC2015,Helvi Hartmann, FIAS Amdahl‘s Law sequential part limits speedup significantly

Message Passing Computing 29 iCSC2015,Helvi Hartmann, FIAS Parallelization While data points increase by a factor 1000 communication grows by 10

Message Passing Computing 30 iCSC2015,Helvi Hartmann, FIAS Parallelization

Message Passing Computing 31 iCSC2015,Helvi Hartmann, FIAS History of parallel computing

Message Passing Computing 32 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing 1994 NASA: Goddard Space Flight Center calculation of 2D streaming properties 16 Linux PC’s Beowulf Cluster

Message Passing Computing 33 iCSC2015,Helvi Hartmann, FIAS

Message Passing Computing 34 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing Linux free and open software stability and portability PVM free and open standard

Message Passing Computing 35 iCSC2015,Helvi Hartmann, FIAS History of Parallel Computing search for extraterrestrial intelligence analyse data of radioteleskopes for extraterrestrial signals million people shared computing time April TFlops, fastest computer 70,7TFlops, second fastest 51,8TFlops

Message Passing Computing 36 iCSC2015,Helvi Hartmann, FIAS CBM - FLES CBM untriggered Experiment Datarate of 1TB/s 1000 input nodes

Message Passing Computing 37 iCSC2015,Helvi Hartmann, FIAS Conclusion

Message Passing Computing 38 iCSC2015,Helvi Hartmann, FIAS : far-far-more-advanced-than-any-software-weve-ever-created-bill-gates/ Llano html itter_globus_ html