Parallel Programming Henri Bal Vrije Universiteit Faculty of Sciences Amsterdam.

Slides:

Advertisements

Similar presentations

Big Data: Big Challenges for Computer Science Henri Bal Vrije Universiteit Amsterdam.

Advertisements

Lecture 38: Chapter 7: Multiprocessors Today’s topic –Vector processors –GPUs –An example 1.

Nis, March 4th, 2005Public Reporting on Project Results Review of Activities of Working Group for Computer Architecture Mile Stojcev Oliver Vojinovic Faculty.

Parallel Programming Yang Xianchun Department of Computer Science and Technology Nanjing University Introduction.

ICS 556 Parallel Algorithms Ebrahim Malalla Office: Bldg 22, Room

Introduction to Parallel Computing

Course Outline Introduction in algorithms and applications Parallel machines and architectures Overview of parallel machines, trends in top-500 Cluster.

Lecture 2: Modern Trends 1. 2 Microprocessor Performance Only 7% improvement in memory performance every year! 50% improvement in microprocessor performance.

Chapter1 Fundamental of Computer Design Dr. Bernard Chen Ph.D. University of Central Arkansas.

More information Examination (date: 19 december) Written exam based on: - Reader (without paper 12 by Marsland and Campbell) - Awari paper (available from.

Summary Background –Why do we need parallel processing? Applications Introduction in algorithms and applications –Methodology to develop efficient parallel.

Parallel Programming Henri Bal Rob van Nieuwpoort Vrije Universiteit Amsterdam Faculty of Sciences.

Team Members: Tyler Drake Robert Wrisley Kyle Von Koepping Justin Walsh Faculty Advisors: Computer Science – Prof. Sanjay Rajopadhye Electrical & Computer.

Chapter Chapter Goals Describe the layers of a computer system Describe the concept of abstraction and its relationship to computing Describe.

Chapter 1. Introduction This course is all about how computers work But what do we mean by a computer? –Different types: desktop, servers, embedded devices.

Introduction What is Parallel Algorithms? Why Parallel Algorithms? Evolution and Convergence of Parallel Algorithms Fundamental Design Issues.

Chapter 1 An Overview of Personal Computers

EET 4250: Chapter 1 Performance Measurement, Instruction Count & CPI Acknowledgements: Some slides and lecture notes for this course adapted from Prof.

CIS 314 : Computer Organization Lecture 1 – Introduction.

Parallel Programming Henri Bal Vrije Universiteit Faculty of Sciences Amsterdam.

Course Outline Introduction in algorithms and applications Parallel machines and architectures Overview of parallel machines, trends in top-500 Cluster.

CS 732: Advance Machine Learning Usman Roshan Department of Computer Science NJIT.

1 Lecture 1: CS/ECE 3810 Introduction Today’s topics:  logistics  why computer organization is important  modern trends.

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display. Slides prepared by Walid A. Najjar & Brian J. Linard, University.

Parallel Programming Henri Bal Vrije Universiteit Amsterdam Faculty of Sciences.

Chapter 1 Sections 1.1 – 1.3 Dr. Iyad F. Jafar Introduction.

Chapter 01 Nell Dale & John Lewis.

Chapter1 Fundamental of Computer Design Dr. Bernard Chen Ph.D. University of Central Arkansas Fall 2010.

Course Outline Introduction in software and applications. Parallel machines and architectures –Overview of parallel machines –Cluster computers (Myrinet)

Multi Core Processor Submitted by: Lizolen Pradhan

BY: ALI AJORIAN ISFAHAN UNIVERSITY OF TECHNOLOGY 2012 GPU Architecture 1.

By Arun Bhandari Course: HPC Date: 01/28/12. GPU (Graphics Processing Unit) High performance many core processors Only used to accelerate certain parts.

Chapter 1 The Big Picture.

EET 4250: Chapter 1 Computer Abstractions and Technology Acknowledgements: Some slides and lecture notes for this course adapted from Prof. Mary Jane Irwin.

Cluster Computers. Introduction Cluster computing –Standard PCs or workstations connected by a fast network –Good price/performance ratio –Exploit existing.

Outline Course Administration Parallel Archtectures –Overview –Details Applications Special Approaches Our Class Computer Four Bad Parallel Algorithms.

MS Thesis Defense “IMPROVING GPU PERFORMANCE BY REGROUPING CPU-MEMORY DATA” by Deepthi Gummadi CoE EECS Department April 21, 2014.

1 Recap (from Previous Lecture). 2 Computer Architecture Computer Architecture involves 3 inter- related components – Instruction set architecture (ISA):

Programming Concepts in GPU Computing Dušan Gajić, University of Niš Programming Concepts in GPU Computing Dušan B. Gajić CIITLab, Dept. of Computer Science.

Multiprocessing. Going Multi-core Helps Energy Efficiency William Holt, HOT Chips 2005 Adapted from UC Berkeley "The Beauty and Joy of Computing"

Reminder Lab 0 Xilinx ISE tutorial Research Send me an if interested Looking for those interested in RC with skills in compilers/languages/synthesis,

Course Outline Introduction in algorithms and applications Parallel machines and architectures Overview of parallel machines, trends in top-500, clusters,

Chapter 1 — Computer Abstractions and Technology — 1 The Computer Revolution Progress in computer technology – Underpinned by Moore’s Law Makes novel applications.

Chapter 1 Computer Abstractions and Technology. Chapter 1 — Computer Abstractions and Technology — 2 The Computer Revolution Progress in computer technology.

Summary Background –Why do we need parallel processing? Moore’s law. Applications. Introduction in algorithms and applications –Methodology to develop.

Computer performance issues* Pipelines, Parallelism. Process and Threads.

DR. SIMING LIU SPRING 2016 COMPUTER SCIENCE AND ENGINEERING UNIVERSITY OF NEVADA, RENO CS 219 Computer Organization.

CSci6702 Parallel Computing Andrew Rau-Chaplin

CS 732: Advance Machine Learning

Parallel Programming Henri Bal Vrije Universiteit Faculty of Sciences Amsterdam.

3/12/2013Computer Engg, IIT(BHU)1 PARALLEL COMPUTERS- 3.

Computer Architecture Lecture 24 Parallel Processing Ralph Grishman November 2015 NYU.

Parallel Programming Henri Bal Vrije Universiteit Faculty of Sciences Amsterdam.

Hardware Trends CSE451 Andrew Whitaker. Motivation Hardware moves quickly OS code tends to stick around for a while “System building” extends way beyond.

Introduction Goal: connecting multiple computers to get higher performance – Multiprocessors – Scalability, availability, power efficiency Job-level (process-level)

Hardware Trends CSE451 Andrew Whitaker. Motivation Hardware moves quickly OS code tends to stick around for a while “System building” extends way beyond.

VU-Advanced Computer Architecture Lecture 1-Introduction 1 Advanced Computer Architecture CS 704 Advanced Computer Architecture Lecture 1.

Fermi National Accelerator Laboratory & Thomas Jefferson National Accelerator Facility SciDAC LQCD Software The Department of Energy (DOE) Office of Science.

BLUE GENE Sunitha M. Jenarius. What is Blue Gene A massively parallel supercomputer using tens of thousands of embedded PowerPC processors supporting.

INTRODUCTION TO HIGH PERFORMANCE COMPUTING AND TERMINOLOGY.

Conclusions on CS3014 David Gregg Department of Computer Science

Web: Parallel Computing Rabie A. Ramadan , PhD Web:

Course Outline Introduction in algorithms and applications

Introduction to Reconfigurable Computing

CS775: Computer Architecture

Summary Background Introduction in algorithms and applications

Course Outline Introduction in algorithms and applications

Chapter 1 Introduction.

Parallel Computing Demand for High Performance

Vrije Universiteit Amsterdam

Presentation transcript:

Parallel Programming Henri Bal Vrije Universiteit Faculty of Sciences Amsterdam

Overview What is parallel programming? Why do we need parallel programming? Organization of this course Practicum Parallel Programming

ICWall A large display suitable for: – Education (classroom) – Modern scientific applications that need to visualize large data-sets – 3D applications

ICWall: an "off-the-shelf" video wall High-end graphics is very expensive Use tiled display with many projectors Use standard (inexpensive) components – 9 PCs, graphic and network cards, 16 projectors Similar idea as cluster computing, but applied to graphics Current ICWall is strongly simplified version –1 PC with 2 projectors

Parallel Programming Sequential programming Single thread of control Parallel programming Multiple threads of control Why parallel programming? Eases programming? Not really. Performance? Yes!

Famous quote “Parallel programming may do something to revive the pioneering spirit in programming, which seems to be degenerating into a rather dull and routine occupation’’ S. Gill, Computer Journal, 1958

Why do we need parallel processing? Many applications need much faster machines Sequential machines are reaching their speed limits –Moore’s Law (1975): Circuit complexity doubles every 18 months –Using many transistors effectively to increase performance becomes critical problem Memory becomes a bottleneck –DRAM access times improve only 10% per year –Caches more and more important

Parallel processing Use multiple processors to solve large problems fast –Also gives more cache memory and aggregate memory bandwidth Microprocessors are getting cheaper and cheaper Cheap multiprocessors and multicore CPUs bring parallel processing to the desktop!

History 1950s: first ideas (see Gill’s quote) 1967 first parallel computer (ILLIAC IV) 1970s programming methods, experimental machines 1980s: parallel languages (SR, Linda, Orca), commercial supercomputers 1990s: software standardization (MPI), clusters, large-scale machines (Blue Gene) 2000s: grid computing: combining resources world-wide (Globus) Now: much research on multi-cores, Graphical Processor Units (GPUs), IBM Cell, ….

Large-scale parallel machines Many parallel machines exist See Current #1: RoadRunner Cell cluster with cores #2 is IBM BlueGene/L with cores

Challenging Applications Modeling ozone layer, climate, ocean Quantum chemistry Protein folding General: computational science Aircraft modeling Handling use volumes of data from scientific instruments –Lofar (astronomy) –LHC (CERN, high-energy physics) Computer chess Analyzing multimedia content Generating movies

Application areas Engineering and design Scientific applications Commercial applications (e.g., transactions, databases) Embedded systems (e.g., cars) This class focuses on scientific applications

About this Course Goal: Study how to write programs that run in parallel on a large number of machines. Focus on programming methods, languages, applications Focus on distributed-memory (message passing) machines, not (yet) on multi-cores, special purpose (GPU, Cell) machines Prerequisites: Some knowledge about sequential languages Little knowledge about networking and operating systems

Aspects of Parallel Computing Algorithms and applications Programming methods, languages, and environments Parallel machines and architectures

Course Outline Introduction in algorithms and applications Parallel machines and architectures Overview of parallel machines, trends in top-500 Cluster computers, BlueGene Programming methods, languages, and environments Message passing (SR, MPI, Java) Higher-level language: HPF Applications N-body problems, search algorithms, bioinformatics Grid computing Multimedia content analysis on Grids (guest lecture Frank Seinstra)

Course Information Examination Written exam based on: - Reader: available electronically from Blackboard - Lectures More information (and slides):

Practicum Parallel Programming Separate practicum (6 ECTS) Implement ASP + SOR algorithms in C/MPI Implement Capture Go search algorithm in Java/RMI Test and measure the programs on our DAS cluster

More information Register by to (Niels Drost) now Starts after this course (mid October) See for a complete description