Presentation is loading. Please wait.

Presentation is loading. Please wait.

Computer Science and Engineering Copyright by Hesham El-Rewini Advanced Computer Architecture CSE 8383 February 21 2006 Session 11.

Published byBritton Tate Modified over 8 years ago

Similar presentations

Presentation on theme: "Computer Science and Engineering Copyright by Hesham El-Rewini Advanced Computer Architecture CSE 8383 February 21 2006 Session 11."— Presentation transcript:

1 Computer Science and Engineering Copyright by Hesham El-Rewini Advanced Computer Architecture CSE 8383 February 21 2006 Session 11

2 Computer Science and Engineering Copyright by Hesham El-Rewini Contents Dynamic Networks (Cont.) Static Networks (Revisited) Performance Analysis

3 Computer Science and Engineering Copyright by Hesham El-Rewini Multistage Interconnection Networks ISC1 ISC2 ISCn switches ISC  Inter-stage Connection Patterns

4 Computer Science and Engineering Copyright by Hesham El-Rewini Perfect-Shuffle Routing Function Given x = {a n, a n-1, …, a 2, a 1 } P(x) = {a n-1, …, a 2, a 1, a n } X = 110001 P(x) = 100011

5 Computer Science and Engineering Copyright by Hesham El-Rewini Perfect Shuffle Example 000  000 001  010 010  100 011  110 100  001 101  011 110  101 111  111

6 Computer Science and Engineering Copyright by Hesham El-Rewini Perfect-Shuffle 000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111

7 Computer Science and Engineering Copyright by Hesham El-Rewini Exchange Routing Function Given x = {a n, a n-1, …, a 2, a 1 } E i (x) = {a n, a n-1, …, a i, …, a 2, a 1 } X = 0000000 E 3 (x) = 0000100

8 Computer Science and Engineering Copyright by Hesham El-Rewini Exchange E 1 000  001 001  000 010  011 011  010 100  101 101  100 110  111 111  110

9 Computer Science and Engineering Copyright by Hesham El-Rewini Exchange E 1 000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111

10 Computer Science and Engineering Copyright by Hesham El-Rewini Butterfly Routing Function Given x = {a n, a n-1, …, a 2, a 1 } B(x) = {a 1, a n-1, …, a 2, a n } X = 010001 P(x) = 110000

11 Computer Science and Engineering Copyright by Hesham El-Rewini Butterfly Example 000  000 001  100 010  010 011  110 100  001 101  101 110  011 111  111

12 Computer Science and Engineering Copyright by Hesham El-Rewini Butterfly 000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111

13 Computer Science and Engineering Copyright by Hesham El-Rewini Multi-stage network 000 001 010 011 100 101 110 111 000 001 010 011 100 101 110 111

14 Computer Science and Engineering Copyright by Hesham El-Rewini MIN (cont.) An 8X8 Banyan network

15 Computer Science and Engineering Copyright by Hesham El-Rewini Min Implementation Control (X) Source (S) Destination (D) X = f(S,D)

16 Computer Science and Engineering Copyright by Hesham El-Rewini Example X = 0 X = 1 ( crossed ) (straight ) A B C D A B C D

17 Computer Science and Engineering Copyright by Hesham El-Rewini Consider this MIN S1 S2 S3 S4 S5 S6 S7 S8 D1 D2 D3 D4 D5 D6 D7 D8 stage 1 stage 2 stage 3

18 Computer Science and Engineering Copyright by Hesham El-Rewini Example (Cont.) Let control variable be X 1, X 2, X 3 Find the values of X 1, X 2, X 3 to connect: S1  D6 S7  D5 S4  D1

19 Computer Science and Engineering Copyright by Hesham El-Rewini The 3 connections S1 S2 S3 S4 S5 S6 S7 S8 D1 D2 D3 D4 D5 D6 D7 D8 stage 1 stage 2 stage 3

20 Computer Science and Engineering Copyright by Hesham El-Rewini Boolean Functions X = x 1, x 2, x 3 S = s 2, s 2, s 3 D = d 1, d 2, d 3 Find X = f(S,D)

21 Computer Science and Engineering Copyright by Hesham El-Rewini Crossbar Switch M1 M2 M3 M4 M5 M6 M7 M8 P1 P2 P3 P4 P5 P6 P7 P8

22 Computer Science and Engineering Copyright by Hesham El-Rewini Analysis and performance metrics dynamic networks NetworksDelayCostBlockingDegree of FT BusO(N)O(1)Yes0 Multiple-busO(mN)O(m)Yes(m-1) MINO(logN)O(NlogN)Yes0 CrossbarO(1)O(N 2 )No0

23 Computer Science and Engineering Copyright by Hesham El-Rewini Static Network Analysis (Revisited) Graph Representation Parameters Cost Degree Diameter Fault tolerance

24 Computer Science and Engineering Copyright by Hesham El-Rewini Graph Review G = (V,E) -- V: nodes, E: edges Directed vs. Undirected Weighted Graphs Path, path length, shortest path Cycles, cyclic vs. acyclic Connectivity: connected, weakly connected, strongly connected, fully connected

25 Computer Science and Engineering Copyright by Hesham El-Rewini Linear Array N nodes, N-1 edges Node Degree: Diameter: Cost: Fault Tolerance:

26 Computer Science and Engineering Copyright by Hesham El-Rewini Ring N nodes, N edges Node Degree: Diameter: Cost: Fault Tolerance:

27 Computer Science and Engineering Copyright by Hesham El-Rewini Chordal Ring N nodes, N edges Node Degree: Diameter: Cost: Fault Tolerance:

28 Computer Science and Engineering Copyright by Hesham El-Rewini Barrel Shifter Number of nodes N = 2 n Start with a ring Add extra edges from each node to those nodes having power of 2 distance i & j are connected if |j-i| = 2 r, r = 0, 1, 2, …, n-1

29 Computer Science and Engineering Copyright by Hesham El-Rewini Mesh and Torus Node Degree: Internal  4 Other  3, 2 Diameter: 2(n-1) N = n*n Node Degree: 4 Diameter: 2* floor(n/2)

30 Computer Science and Engineering Copyright by Hesham El-Rewini Hypercubes N = 2 d d dimensions (d = log N) A cube with d dimensions is made out of 2 cubes of dimension d-1 Symmetric Degree, Diameter, Cost, Fault tolerance Node labeling – number of bits

31 Computer Science and Engineering Copyright by Hesham El-Rewini Hypercubes d = 0d = 1d = 2d = 3 0 1 01 00 11 10 000 001 100110 111 011 101 010

32 Computer Science and Engineering Copyright by Hesham El-Rewini Hypercubes 1110 1111 1010 1011 0110 0111 0010 0011 1101 1010 1000 1001 0100 0101 0010 0000 0001 S d = 4

33 Computer Science and Engineering Copyright by Hesham El-Rewini Hypercube of dimension d N = 2 d d = log n Node degree = d Number of bits to label a node = d Diameter = d Number of edges = n*d/2 Hamming distance! Routing

34 Computer Science and Engineering Copyright by Hesham El-Rewini Subcubes and Cube Fragmentation What is a subcube? Shared Environment Fragmentation Problem Is it Similar to something you know?

35 Computer Science and Engineering Copyright by Hesham El-Rewini Cube Connected Cycles (CCC) k-cube  2 k nodes k-CCC from k-cube, replace each vertex of the k cube with a ring of k nodes K-CCC  k* 2 k nodes Degree, diameter  3, 2k Try it for 3-cube

36 Computer Science and Engineering Copyright by Hesham El-Rewini K-ary n-Cube d = cube dimension K = # nodes along each dimension N = k d Wraparound Hupercube  binary d-cube Tours  k-ary 2-cube

37 Computer Science and Engineering Copyright by Hesham El-Rewini Grosch’s Law Moore’s Law Von Neumann’s Bottlneck Parallelism Speedup Amdahl’s Law The Gustafson-Barsis Law Benchmarks Performance Evaluation

38 Computer Science and Engineering Copyright by Hesham El-Rewini Grosch’s Law (1960s) “To sell a computer for twice as much, it must be four times as fast” Vendors skip small speed improvements in favor of waiting for large ones Buyers of expensive machines would wait for a twofold improvement in performance for the same price.

39 Computer Science and Engineering Copyright by Hesham El-Rewini Moore’s Law Gordon Moore (cofounder of Intel) Processor performance would double every 18 months This prediction has held for several decades Unlikely that single-processor performance continues to increase indefinitely

40 Computer Science and Engineering Copyright by Hesham El-Rewini Von Neumann’s bottleneck Great mathematician of the 1940s and 1950s Single control unit connecting a memory to a processing unit Instructions and data are fetched one at a time from memory and fed to processing unit Speed is limited by the rate at which instructions and data are transferred from memory to the processing unit.

41 Computer Science and Engineering Copyright by Hesham El-Rewini Problem Assume that a switching component such as a transistor can switch in zero time. We propose to construct a disk- shaped computer chip with such a component. The only limitation is the time it takes to send electronic signals from one edge of the chip to the other. Make the simplifying assumption that electronic signals travel 300,000 kilometers per second. What must be the diameter of a round chip so that it can switch 10 9 times per second? What would the diameter be if the switching requirements were 10 12 time per second?

42 Computer Science and Engineering Copyright by Hesham El-Rewini Parallelism Multiple CPUs Within the CPU One Pipeline Multiple pipelines

43 Computer Science and Engineering Copyright by Hesham El-Rewini Superscalar Parallelism Scheduling

44 Computer Science and Engineering Copyright by Hesham El-Rewini Past Trends in Parallel Architecture (inside the box) Completely custom designed components (processors, memory, interconnects, I/O) Longer R&D time (2-3 years) Expensive systems Quickly becoming outdated Bankrupt companies!!

45 Computer Science and Engineering Copyright by Hesham El-Rewini New Trends in Parallel Architecture (outside the box) Advances in commodity processors and network technology Network of PCs and workstations connected via LAN or WAN forms a Parallel System Network Computing Compete favorably (cost/performance) Utilize unused cycles of systems sitting idle

46 Computer Science and Engineering Copyright by Hesham El-Rewini Speedup S = Speed(new) / Speed(old) S = Work/time(new) / Work/time(old) S = time(old) / time(new) S = time(before improvement) / time(after improvement)

47 Computer Science and Engineering Copyright by Hesham El-Rewini Speedup Time (one CPU): T(1) Time (n CPUs): T(n) Speedup: S S = T(1)/T(n)

48 Computer Science and Engineering Copyright by Hesham El-Rewini Amdahl’s Law The performance improvement to be gained from using some faster mode of execution is limited by the fraction of the time the faster mode can be used

49 Computer Science and Engineering Copyright by Hesham El-Rewini 20 hours 200 miles A B Walk 4 miles /hour Bike 10 miles / hour Car-1 50 miles / hour Car-2 120 miles / hour Car-3 600 miles /hour must walk Example

50 Computer Science and Engineering Copyright by Hesham El-Rewini 20 hours 200 miles A B Walk 4 miles /hour  50 + 20 = 70 hours S = 1 Bike 10 miles / hour  20 + 20 = 40 hours S = 1.8 Car-1 50 miles / hour  4 + 20 = 24 hours S = 2.9 Car-2 120 miles / hour  1.67 + 20 = 21.67 hours S = 3.2 Car-3 600 miles /hour  0.33 + 20 = 20.33 hours S = 3.4 must walk Example

51 Computer Science and Engineering Copyright by Hesham El-Rewini Amdahl’s Law (1967)  : The fraction of the program that is naturally serial (1-  ): The fraction of the program that is naturally parallel

52 Computer Science and Engineering Copyright by Hesham El-Rewini S = T(1)/T(N) T(N) = T(1)  + T(1)(1-  ) N S = 1  + (1-  ) N = N  N + (1-  )

53 Computer Science and Engineering Copyright by Hesham El-Rewini Amdahl’s Law

54 Computer Science and Engineering Copyright by Hesham El-Rewini Gustafson-Barsis Law N &  are not independent from each other T(N) = 1 T(1) =  + (1-  ) N S = N – (N-1)   : The fraction of the program that is naturally serial

55 Computer Science and Engineering Copyright by Hesham El-Rewini Gustafson-Barsis Law

56 Computer Science and Engineering Copyright by Hesham El-Rewini

57 Computer Science and Engineering Copyright by Hesham El-Rewini Distributed Computing Performance Single Program Performance Multiple Program Performance

58 Computer Science and Engineering Copyright by Hesham El-Rewini

59 Computer Science and Engineering Copyright by Hesham El-Rewini Benchmark Performance Serial Benchmarks Parallel Benchmarks PERFECT Benchmarks NAS Kernel The SLALOM The Golden Bell Prize WebSTONE for the Web Performance Comparisons

Download ppt "Computer Science and Engineering Copyright by Hesham El-Rewini Advanced Computer Architecture CSE 8383 February 21 2006 Session 11."

Similar presentations

Comparison Of Network On Chip Topologies Ahmet Salih BÜYÜKKAYHAN 2007706435 - 2009 Fall.

Comparison Of Network On Chip Topologies Ahmet Salih BÜYÜKKAYHAN Fall.
SE-292 High Performance Computing

SE-292 High Performance Computing
Classification of Distributed Systems Properties of Distributed Systems n motivation: advantages of distributed systems n classification l architecture.

Classification of Distributed Systems Properties of Distributed Systems n motivation: advantages of distributed systems n classification l architecture.
Parallel Architectures: Topologies Heiko Schröder, 2003.

Parallel Architectures: Topologies Heiko Schröder, 2003.
Parallel Architectures: Topologies Heiko Schröder, 2003.

Parallel Architectures: Topologies Heiko Schröder, 2003.
1 Interconnection Networks Direct Indirect Shared Memory Distributed Memory (Message passing)

1 Interconnection Networks Direct Indirect Shared Memory Distributed Memory (Message passing)
CSCI 8150 Advanced Computer Architecture Hwang, Chapter 2 Program and Network Properties 2.4 System Interconnect Architectures.

CSCI 8150 Advanced Computer Architecture Hwang, Chapter 2 Program and Network Properties 2.4 System Interconnect Architectures.
Advanced Topics in Algorithms and Data Structures An overview of the lecture 2 Models of parallel computation Characteristics of SIMD models Design issue.

Advanced Topics in Algorithms and Data Structures An overview of the lecture 2 Models of parallel computation Characteristics of SIMD models Design issue.
1 CSE 591-S04 (lect 14) Interconnection Networks (notes by Ken Ryu of Arizona State) l Measure –How quickly it can deliver how much of what’s needed to.

1 CSE 591-S04 (lect 14) Interconnection Networks (notes by Ken Ryu of Arizona State) l Measure –How quickly it can deliver how much of what’s needed to.
Models of Parallel Computation Advanced Algorithms & Data Structures Lecture Theme 12 Prof. Dr. Th. Ottmann Summer Semester 2006.

Models of Parallel Computation Advanced Algorithms & Data Structures Lecture Theme 12 Prof. Dr. Th. Ottmann Summer Semester 2006.
DCS/2003/1 CENG 532 - Distributed Computing Systems Measures of Performance.

DCS/2003/1 CENG Distributed Computing Systems Measures of Performance.
1 Static Interconnection Networks CEG 4131 Computer Architecture III Miodrag Bolic.

1 Static Interconnection Networks CEG 4131 Computer Architecture III Miodrag Bolic.
Introduction to Parallel Processing Ch. 12, Pg

Introduction to Parallel Processing Ch. 12, Pg
Chapter 5 Array Processors. Introduction  Major characteristics of SIMD architectures –A single processor(CP) –Synchronous array processors(PEs) –Data-parallel.

Chapter 5 Array Processors. Introduction  Major characteristics of SIMD architectures –A single processor(CP) –Synchronous array processors(PEs) –Data-parallel.
CPU Performance Assessment As-Bahiya Abu-Samra 4-3-2014 - *Moore’s Law *Clock Speed *Instruction Execution Rate - MIPS - MFLOPS *SPEC Speed Metric *Amdahl’s.

CPU Performance Assessment As-Bahiya Abu-Samra *Moore’s Law *Clock Speed *Instruction Execution Rate - MIPS - MFLOPS *SPEC Speed Metric *Amdahl’s.
Interconnect Network Topologies

Interconnect Network Topologies
Interconnection Networks. Applications of Interconnection Nets Interconnection networks are used everywhere! ◦ Supercomputers – connecting the processors.

Interconnection Networks. Applications of Interconnection Nets Interconnection networks are used everywhere! ◦ Supercomputers – connecting the processors.
Computer Science Department

Computer Science Department
Interconnect Networks

Interconnect Networks
Network Topologies Topology – how nodes are connected – where there is a wire between 2 nodes. Routing – the path a message takes to get from one node.

Network Topologies Topology – how nodes are connected – where there is a wire between 2 nodes. Routing – the path a message takes to get from one node.

Similar presentations

Ads by Google