Presentation is loading. Please wait.

Presentation is loading. Please wait.

Tony Nowatzki Vinay Gangadhar Karthikeyan Sankaralingam

Published byἸοῦστος Φωτόπουλος Modified over 6 years ago

Similar presentations

Presentation on theme: "Tony Nowatzki Vinay Gangadhar Karthikeyan Sankaralingam"— Presentation transcript:

1 Exploring the Potential of Heterogeneous Von Neumann / Dataflow Execution Models
Tony Nowatzki Vinay Gangadhar Karthikeyan Sankaralingam University of Wisconsin - Madison

2 Very high/non-local parallelism Best Suited Architecture:
Time Program Execution Low ILP High ILP Low ILP High ILP Reason for high ILP: Due to Speculation Very high/non-local parallelism Helpful Arch. Features: Branch Prediction, speculative scheduling, fast recovery Very-high issue width, Very-large instruction window Best Suited Architecture: Von Neumann Explicit-Dataflow Von Neumann Explicit- Dataflow Our Proposal: Highly Speculative, Modest ILP High ILP, Little Speculation

3 Dual Issue OOO + 4 Others:
Related Work Our Proposal Von Neumann Von Neumann Von Neumann Explicit Dataflow Dual Issue OOO + 4 Others: (Beret, CCores, bigLITTLE, In-place Loop) 5% Speedup, 20% Energy Efficiency Dual Issue OOO + SEED - 30% Speedup, 70% Energy Efficiency

4 Outline Control Memory (General Purpose) Describing why VonNeumann can complement Dataflow architectures Von- Neumann Explicit Dataflow Leveraging program properties for efficient heterogeneous design Nested Loops Designing SEED: Specialization Engine for Explicit-Dataflow (Offload Engine) Speedup Energy Performing a Design-Space Exploration Across (Small, Medium, Big) VN Cores

5 Von Neumann (Out-of-Order)
Explicit-Dataflow Instruction-by-instruction execution of Control Flow Graph Dependence-graph execution, Control deps becomes data deps Instruction Stream Instruction Window Dependence Graph +Speculation +Local/Non Local ILP +Lower Overheads No Fetch, Decode, Rename – No dynamic dependence graph construction

6 Explicit-Dataflow Wins
Loop 1: Data Dependent Control Loop 2: High ILP, Non-Critical Control ld + × ld × ld × ld + > + + + + + + > br - - - br Von Neumann Wins Explicit-Dataflow Wins (Speculation) (Higher ILP)

7 Unpredictable (Dataflow?) Memory Latency Bound (Dataflow?)
Higher ILP Data-Parallel (SIMD/GPU) High ILP (Data- flow?) Unpredictable (Dataflow?) Memory Regularity General Code (Out-of-Order) Memory Latency Bound (Dataflow?) Control Regularity

8 24 Irregular Workloads from SpecINT/MediaBench
Outline Control Memory Describing why VonNeumann can complement Dataflow architectures. Von- Neumann Explicit Dataflow Leveraging program properties for efficient heterogeneous design Nested Loops 24 Irregular Workloads from SpecINT/MediaBench Designing SEED: Specialization Engine for Explicit-Dataflow Speedup Energy Performing a Design-Space Exploration Across (Small, Medium, Big) VN Cores

9 Property 1: Affinity Phase Behavior
Architecture affinity over Time App. 1 Hundreds to Millions of Instructions App. 2 App. 3 VonNeumann Data-Parallel Dataflow Out-of-Order SIMD Explicit- Dataflow Cache Hierarchy fast-switching Workload Architecture Affinity 13% 25% 63%

10 Property 2: Benefits of Restricted Scope
Arbitrary Code Inner Loops Traces (call) (must support arbitrary procedure calls, recursion, instruction misses) Area/Power: High Low Low Coverage: (any duration) 61% 41% 100% Coverage: (long duration) 46% 27% 100% Fine Print: Static Region Size 1024 Instructions; Long Duration (lasting Longer than 1000 Cycles)

11 Property 2: Benefits of Restricted Scope
Nested Loops Scope: Arbitrary Code Inner Loops Traces (call) Low Area/Power: High Low Low Coverage: (any duration) 74% 61% 41% 100% Coverage: (long duration) 67% 46% 27% 100% Fine Print: Static Region Size 1024 Instructions; Long Duration (lasting Longer than 1000 Instructions)

12 Outline Control Memory Describing why VonNeumann can complement Dataflow architectures. Von- Neumann Explicit Dataflow Leveraging program properties for efficient heterogeneous design Nested Loops Fine-Grain Switching => Phased Dataflow Affinity Simplify Dataflow Arch. => Nested-Loop Scope Designing SEED: Specialization Engine for Explicit-Dataflow System Overview Architecture Inspiration Architecture Overview Speedup Energy Performing a Design-Space Exploration Across (Small, Medium, Big) VN Cores

13 System Overview Compiler finds and inlines profitable nested loop regions, and generates the dataflow representation for SEED. Invoking SEED: SEED_CONFIG: Begins streaming configuration ( cycles) SEED_BEGIN: Transfers execution, powers down non-stateful host core components Resuming OOO: Core is powered on Live values transferred to OOO core registers. Binary OOO Core SEED L1 Cache Nested Loop Compiler Runs on Time Program System Architecture SEED_CONFIG SEED_BEGIN Resume OOO Time Program Execution

14 Leveraging Decades of Dataflow Research
Design Decisions Scope Integration to Host Control Flow Control Speculation Dataflow Firing Execution Units Design Decisions TRIPS WaveScalar DySER BERET Scope Whole Program Inner Loop Compute Hot Loop Trace Integration to Host Behind L1 Cache Control Flow VN/Predicat. Switch Insts. Predication Trace-only Control Speculation Block-based None Dataflow Firing Position-based Tag-based Static Not applicable Execution Units Homog. FU Heterog. FU Compound FU Criteria: 1. Low Area/Power 2. High Performance 3. Complement Capabilities of VonNeumann

15 SEED Architecture … L1 Cache IMU IMU CFU1 CFU8 ODU ODU OOO CPU
Seed Unit 1 Seed Unit 8 ICache OOO CPU DCache IMU (Instruction Mgmt. Unit) IMU Config & Init. Bus Arbiter Entering & Exiting SEED Regions SEED IMU: Keeps instruction definitions and unrolled operand storage (32 comp. insts). Issues one instruction per cycle to CFU. ODU+Bus: Multi-bus distribution network for communicating between SEED Units. CFU8 … CFU1 CFU: SEED Units are organized around a set of compound functional units (CFUs) Instructions each. Compile-time scheduler assigns instructions to compound FUs and SEED units, using an integer linear program [see Paper] CPU XFER Store Buffer ODU ODU Store Buffer: Interface to memory system.

16 Capability Comparison
L1 Cache OOO CPU Config & Init. XFER Store Buffer IMU ODU Bus Arbiter Seed Unit 1 Seed Unit 8 (Instruction Mgmt. Unit) CFU8 … CFU1 ICache DCache Quad-Issue OOO SEED Max. Effective IPC 3-3.5 (4) 6-7 (16) Max. Instruction Window 48 (48) (1024) Speculative Control Yes No Speculative Scheduling (parens: theoretical max)

17 Outline Control Memory Describing why VonNeumann can complement Dataflow architectures. Von- Neumann Explicit Dataflow Leveraging program properties for efficient heterogeneous design Nested Loops Designing SEED: Specialization Engine for Explicit-Dataflow Speedup Energy Performing a Design-Space Exploration Across (Small, Medium, Big) VN Cores

18 Experimental Setup Methodology
Simulator: Modified Gem5 + McPAT/Cacti Power: Core + L1 + L2 + Static and 22nm) Comparison to State-of-the-art VonNeuman-Based Heterogeneous Execution Models: “Accelerators”: BERET, Conservation-Cores Micro-arch: bigLITTLE, In-place Loop Execution Oracle scheduler: choose best arch. per-region (others have demonstrated solutions [Padmanabha et al., MICRO 2013]) Design Space Exploration: IO2: Dual-Issue Inorder OOO2: Dual-Issue, 32-entry IW OOO4: Quad-Issue, 48-entry IW

19 Architecture Comparison
State-of-the-art 1.14× Perf. 1.54× En. This Way Better 1.3× Perf. 1.7× En. ThisWork

20 Dataflow Heterogeneity Analysis
Slowdown Regions Vectorizable on OOO Core Control on Critical Path Short Regions Similar Regions No speculative scheduling Fewer Branch-Mispredicts Less Stack Spilling Speedup Regions High Instruction Parallelism High Memory Parallelism

21 Dataflow Heterogeneity Analysis
Slowdown Regions Vectorizable on OOO Core Control on Critical Path Short Regions Similar Regions No speculative scheduling Fewer Branch-Mispredicts Less Stack Spilling Speedup Regions High Instruction Parallelism High Memory Parallelism

22 Dataflow Heterogeneity Analysis
Slowdown Regions Vectorizable on OOO Core Control on Critical Path Short Regions Similar Regions No speculative scheduling Fewer Branch-Mispredicts Less Stack Spilling Speedup Regions High Instruction Parallelism High Memory Parallelism

23 Dataflow Heterogeneity Analysis
Slowdown Regions Vectorizable on OOO Core Control on Critical Path Short Regions Similar Regions No speculative scheduling Fewer Branch-Mispredicts Less Stack Spilling Speedup Regions High Instruction Parallelism High Memory Parallelism

24 Dataflow Heterogeneity Analysis
Slowdown Regions Vectorizable on OOO Core Control on Critical Path Short Regions Similar Regions No speculative scheduling Fewer Branch-Mispredicts Less Stack Spilling Speedup Regions High Instruction Parallelism High Memory Parallelism

25 Dataflow Heterogeneity Analysis
Slowdown Regions Vectorizable on OOO Core Control on Critical Path Short Regions Similar Regions No speculative scheduling Fewer Branch-Mispredicts Less Stack Spilling Speedup Regions High Instruction Parallelism High Memory Parallelism

26 Dataflow Heterogeneity Analysis
Slowdown Regions Vectorizable on OOO Core Control on Critical Path Short Regions Similar Regions No speculative scheduling Fewer Branch-Mispredicts Less Stack Spilling Speedup Regions High Instruction Parallelism High Memory Parallelism

27 Heterogeneity Analysis Summary
Slowdown Regions Vectorizable on OOO Core Control on Critical Path Short Regions Similar Regions No speculative scheduling Fewer Branch-Mispredicts Less Stack Spilling Speedup Regions High Instruction Parallelism High Memory Parallelism

28 Conclusions Heterogeneous Von Neumann + Dataflow has a high potential
Especially for modest sized OOO Cores Delay need for application-specific accelerators? Looking Forward Augment dataflow-architecture with data-parallel capabilities? Alternative heterogeneous models (what other program properties to leverage?) Can micro-architecture modifications achieve the same benefits? VN DF DF ? Thank you!

Download ppt "Tony Nowatzki Vinay Gangadhar Karthikeyan Sankaralingam"

Similar presentations

Breaking SIMD Shackles with an Exposed Flexible Microarchitecture and the Access Execute PDG Venkatraman Govindaraju, Tony Nowatzki, Karthikeyan Sankaralingam.

Breaking SIMD Shackles with an Exposed Flexible Microarchitecture and the Access Execute PDG Venkatraman Govindaraju, Tony Nowatzki, Karthikeyan Sankaralingam.
Thread Criticality Predictors for Dynamic Performance, Power, and Resource Management in Chip Multiprocessors Abhishek Bhattacharjee Margaret Martonosi.

Thread Criticality Predictors for Dynamic Performance, Power, and Resource Management in Chip Multiprocessors Abhishek Bhattacharjee Margaret Martonosi.
DBMSs on a Modern Processor: Where Does Time Go? Anastassia Ailamaki Joint work with David DeWitt, Mark Hill, and David Wood at the University of Wisconsin-Madison.

DBMSs on a Modern Processor: Where Does Time Go? Anastassia Ailamaki Joint work with David DeWitt, Mark Hill, and David Wood at the University of Wisconsin-Madison.
Combining Statistical and Symbolic Simulation Mark Oskin Fred Chong and Matthew Farrens Dept. of Computer Science University of California at Davis.

Combining Statistical and Symbolic Simulation Mark Oskin Fred Chong and Matthew Farrens Dept. of Computer Science University of California at Davis.
Scalable Multi-Cache Simulation Using GPUs Michael Moeng Sangyeun Cho Rami Melhem University of Pittsburgh.

Scalable Multi-Cache Simulation Using GPUs Michael Moeng Sangyeun Cho Rami Melhem University of Pittsburgh.
1 Advanced Computer Architecture Limits to ILP Lecture 3.

1 Advanced Computer Architecture Limits to ILP Lecture 3.
Data Marshaling for Multi-Core Architectures M. Aater Suleman Onur Mutlu Jose A. Joao Khubaib Yale N. Patt.

Data Marshaling for Multi-Core Architectures M. Aater Suleman Onur Mutlu Jose A. Joao Khubaib Yale N. Patt.
UPC Microarchitectural Techniques to Exploit Repetitive Computations and Values Carlos Molina Clemente LECTURA DE TESIS, (Barcelona,14 de Diciembre de.

UPC Microarchitectural Techniques to Exploit Repetitive Computations and Values Carlos Molina Clemente LECTURA DE TESIS, (Barcelona,14 de Diciembre de.
Limits on ILP. Achieving Parallelism Techniques – Scoreboarding / Tomasulo’s Algorithm – Pipelining – Speculation – Branch Prediction But how much more.

Limits on ILP. Achieving Parallelism Techniques – Scoreboarding / Tomasulo’s Algorithm – Pipelining – Speculation – Branch Prediction But how much more.
Computer Architecture 2011 – Out-Of-Order Execution 1 Computer Architecture Out-Of-Order Execution Lihu Rappoport and Adi Yoaz.

Computer Architecture 2011 – Out-Of-Order Execution 1 Computer Architecture Out-Of-Order Execution Lihu Rappoport and Adi Yoaz.
June 20 th 2004University of Utah1 Microarchitectural Techniques to Reduce Interconnect Power in Clustered Processors Karthik Ramani Naveen Muralimanohar.

June 20 th 2004University of Utah1 Microarchitectural Techniques to Reduce Interconnect Power in Clustered Processors Karthik Ramani Naveen Muralimanohar.
Defining Wakeup Width for Efficient Dynamic Scheduling A. Aggarwal, O. Ergin – Binghamton University M. Franklin – University of Maryland Presented by:

Defining Wakeup Width for Efficient Dynamic Scheduling A. Aggarwal, O. Ergin – Binghamton University M. Franklin – University of Maryland Presented by:
Multiscalar processors

Multiscalar processors
Computer Architecture 2010 – Out-Of-Order Execution 1 Computer Architecture Out-Of-Order Execution Lihu Rappoport and Adi Yoaz.

Computer Architecture 2010 – Out-Of-Order Execution 1 Computer Architecture Out-Of-Order Execution Lihu Rappoport and Adi Yoaz.
Distributed Microarchitectural Protocols in the TRIPS Prototype Processor Sankaralingam et al. Presented by Cynthia Sturton CS 258 3/3/08.

Distributed Microarchitectural Protocols in the TRIPS Prototype Processor Sankaralingam et al. Presented by Cynthia Sturton CS 258 3/3/08.
Korea Univ B-Fetch: Branch Prediction Directed Prefetching for In-Order Processors 컴퓨터 · 전파통신공학과 2015020802 최병준 1 Computer Engineering and Systems Group.

Korea Univ B-Fetch: Branch Prediction Directed Prefetching for In-Order Processors 컴퓨터 · 전파통신공학과 최병준 1 Computer Engineering and Systems Group.
University of Michigan Electrical Engineering and Computer Science 1 Extending Multicore Architectures to Exploit Hybrid Parallelism in Single-Thread Applications.

University of Michigan Electrical Engineering and Computer Science 1 Extending Multicore Architectures to Exploit Hybrid Parallelism in Single-Thread Applications.
COMPUTER SCIENCE &ENGINEERING Compiled code acceleration on FPGAs W. Najjar, B.Buyukkurt, Z.Guo, J. Villareal, J. Cortes, A. Mitra Computer Science & Engineering.

COMPUTER SCIENCE &ENGINEERING Compiled code acceleration on FPGAs W. Najjar, B.Buyukkurt, Z.Guo, J. Villareal, J. Cortes, A. Mitra Computer Science & Engineering.
Hybrid-Scheduling: A Compile-Time Approach for Energy–Efficient Superscalar Processors Madhavi Valluri and Lizy John Laboratory for Computer Architecture.

Hybrid-Scheduling: A Compile-Time Approach for Energy–Efficient Superscalar Processors Madhavi Valluri and Lizy John Laboratory for Computer Architecture.
Is Out-Of-Order Out Of Date ? IA-64’s parallel architecture will improve processor performance William S. Worley Jr., HP Labs Jerry Huck, IA-64 Architecture.

Is Out-Of-Order Out Of Date ? IA-64’s parallel architecture will improve processor performance William S. Worley Jr., HP Labs Jerry Huck, IA-64 Architecture.

Similar presentations

Ads by Google