Center for Embedded Computer Systems Dynamic Conditional Branch Balancing during the High-Level Synthesis of Control-Intensive Designs Supported by Semiconductor Research Corporation 1 School of Information and Computer Science University of California, Irvine 2 Department of Computer Science And Engineering University of California, San Diego Sumit Gupta 1 Nikil Dutt 1 Rajesh Gupta 2 Alex Nicolau 1

2 High Level Synthesis: From Behavior to Hardware M e m o r y ALU Control Data path d = e - fg = h + i If Node TF c x = a + b c = a < b j = d x g l = e + x x = a + b; c = a < b; if (c) then d = e – f; else g = h + i; j = d x g; l = e + x;  Our approach targets descriptions with nested conditionals and loops

3 Synthesizing Control-Intensive Designs Programming style and control constructs have tremendous impact on the quality of HLS results Programming style and control constructs have tremendous impact on the quality of HLS results Operation placement is for programming convenience: not optimized for synthesis Operation placement is for programming convenience: not optimized for synthesis Restructure and duplicate code using Parallelizing Compiler Transformations: Speculative Code Motions Restructure and duplicate code using Parallelizing Compiler Transformations: Speculative Code Motions We present heuristics that carefully guide and increase the scope of the speculative code motions – particularly operation duplication (Conditional Speculation ) We present heuristics that carefully guide and increase the scope of the speculative code motions – particularly operation duplication (Conditional Speculation )

4 Toolbox Approach to Scheduling Scheduling Code Motion Dynamic CSE Loop Transformations Percolation/Trailblazing Speculative Code Motions CSE/IVA/Copy Prop Operation Chaining Loop Transformations Heuristics Transformations Toolbox Scheduling Framework Scheduling Heuristics employ Code Transformations from Transformations Toolbox Scheduling Heuristics employ Code Transformations from Transformations Toolbox

5 Scheduling using Speculative Code Motions BB 2BB 3 BB 1 BB 6BB 7 BB 5 BB 4 BB c b d + + a BB 0 BB 9 Speculate Across If Block Speculate BB 2BB 3 BB 1 BB 6BB 7 BB 5 BB 4 BB c b + a BB 0 BB 9 + d + Resource Allocation+

6 BB 2BB 3 BB 1 BB 6BB 7 BB 5 BB 4 BB c b + a BB 0 BB 9 + d Scheduling using Speculative Code Motions BB 2BB 3 BB 1 BB 6BB 7 BB 5 BB 4 BB c b d + + Across If Block Conditional Speculation + a + d BB 0 BB 9 + d + Resource Allocation+   Conditional Speculation   Duplicates Operations into the branches of a Conditional Block

7 Increasing the Scope of Code Motions If Node TF _ e BB 0 BB 2 BB 1 BB 3 BB 4 + a + b _ c _ d S0 S1 S2 S3 + Resource Allocation Original Design If Node TF _ e BB 0 BB 2 BB 1 BB 3 BB 4 + a + b _ c _ d Scheduled Design Unbalanced Conditional Longest Path A is a set of concurrent operations inside a basic block A scheduling step is a set of concurrent operations inside a basic block A basic block is a sequence of scheduling steps with no control branches/merges between them A basic block is a sequence of scheduling steps with no control branches/merges between them

8 Insert New Scheduling Step in Shorter Branch If Node TF _ e BB 0 BB 2 BB 1 BB 3 BB 4 + a + b _ c _ d If Node TF _ e BB 0 BB 2 BB 1 BB 3 BB 4 + a + b _ c _ d S0 S1 S2 S3 + Resource Allocation Original DesignScheduled Design

9 Insert New Scheduling Step in Shorter Branch If Node TF BB 0 BB 2 BB 1 BB 3 BB 4 + a + b _ c _ d If Node TF _ e BB 0 BB 2 BB 1 BB 3 BB 4 + a + b _ c _ d S0 S1 S2 S3 + Resource Allocation e __ e Original DesignScheduled Design Insert scheduling steps into shorter conditional branch Insert scheduling steps into shorter conditional branch Enables further code compaction Enables further code compaction

10 Organization of Scheduling Heuristics Scheduling Heuristic Candidate Mover Candidate Provider IR Walker Traverses Design to find next basic block to schedule Traverses Design to find Candidate Operations to schedule Chooses one of the Candidate Operations to Schedule Moves, duplicates and schedules chosen Operation Scheduler

11 Organization of Scheduling Heuristics Scheduling Heuristic Candidate Mover Candidate Provider IR Walker Scheduler BBDT Branch Balancing During Traversal BBDCM Branch Balancing During Code Motion Check if BBDCM will Enable Code Motion

12 BBDT: Get Next Step to Schedule Schedule Design starting from first basic block in Design Schedule Design starting from first basic block in Design On each call, returns next step in current basic block On each call, returns next step in current basic block If last step in basic block is reached If last step in basic block is reached If current BB is a Branch of a Conditional C If basic blocks in other branches of C are scheduled and have more scheduling steps, Insert new step in currBB Traverse design and get next basic block Traverse design and get next basic block Return first step from next basic block Return first step from next basic block If Node TF _ f BB 1 BB 3 BB 2 BB 4 BB 5 + b + d _ c BB 0 _ e _ a

13 BBDT: Insert Scheduling Steps while Getting Next Step to Schedule Schedule Design starting from first basic block in Design Schedule Design starting from first basic block in Design On each call, returns next step in current basic block On each call, returns next step in current basic block If last step in basic block is reached If last step in basic block is reached If current BB is a Branch of a Conditional C If basic blocks in other branches of C are scheduled and have more scheduling steps, Insert new step in currBB Traverse design and get next basic block Traverse design and get next basic block Return first step from next basic block Return first step from next basic block If Node TF _ f BB 1 BB 3 BB 2 BB 4 BB 5 + b + d _ c _ a BB 0 _ e Unbalanced Conditional

14 BBDT: Insert Scheduling Steps while Getting Next Step to Schedule If Node TF _ f BB 1 BB 3 BB 2 BB 4 BB 5 + b + d _ c _ a BB 0 _ e Schedule Design starting from first basic block in Design Schedule Design starting from first basic block in Design On each call, returns next step in current basic block On each call, returns next step in current basic block If last step in basic block is reached If last step in basic block is reached If current BB is a Branch of a Conditional C If basic blocks in other branches of C are scheduled and have more scheduling steps, Insert new step in currBB Traverse design and get next basic block Traverse design and get next basic block Return first step from next basic block Return first step from next basic block

15 Scope of BBDT If Node 2 TF f BB 2 BB 4 BB 3 BB 5 BB 7 + c + _ d BB 6 b _ a If Node 1 TF BB 0 BB Resource Constraints Scheduling order: BB1, BB3 and BB4 After scheduling the step in BB4 BBDT adds one new step in BB3 & BB4 since number of steps in BB1 is larger Scheduled Being Scheduled

16 Scope of BBDT If Node 2 TF f BB 2 BB 4 BB 3 BB 5 + c + _ d BB 6 b _ a If Node 1 TF BB 0 BB 1 + Scheduling order: Scheduling order: BB1, BB3 and BB4 BB1, BB3 and BB4 After scheduling the step in BB4 After scheduling the step in BB4 BBDT adds one new step in BB3 & BB4 since number of steps in BB1 is larger BBDT adds one new step in BB3 & BB4 since number of steps in BB1 is larger The new step in BB4 is now scheduled The new step in BB4 is now scheduled Operation f can be duplicated up into BB1, BB3 and BB4 Operation f can be duplicated up into BB1, BB3 and BB4 BB 7 + Resource Constraints Scheduled Being Scheduled

17 Scope of BBDT Inserts steps (Balances Branches) after scheduling last branch of conditional Inserts steps (Balances Branches) after scheduling last branch of conditional Needs all other branches to be scheduled already Needs all other branches to be scheduled already To get an accurate picture of the number of steps and the resource utilization in all the conditional branches To get an accurate picture of the number of steps and the resource utilization in all the conditional branches Continues to schedule the new scheduling step in the last branch Continues to schedule the new scheduling step in the last branch

18 Limitations of BBDT If Node 2 TF f BB 2 BB 4 BB 3 BB 5 + c + _ d BB 6 b _ a If Node 1 TF BB 0 BB 1 + _ e Scheduling order: Scheduling order: BB1, BB3 and BB4 BB1, BB3 and BB4 BBDT can add new steps/balance branches After scheduling last branch in conditional BBDT can add new steps/balance branches After scheduling last branch in conditional BBDT adds new step in BB3 after scheduling BB4, so we cannot do CS of operation f BBDT adds new step in BB3 after scheduling BB4, so we cannot do CS of operation f BB 7 + Resource Constraints Scheduled Being Scheduled

19 Limitations of BBDT If Node 2 TF f BB 2 BB 4 BB 3 BB 5 + c + _ d BB 6 b _ a If Node 1 TF BB 0 BB 1 + _ e Scheduling order: Scheduling order: BB1, BB3 and BB4 BB1, BB3 and BB4 BBDT can add new steps/balance branches After scheduling last branch in conditional BBDT can add new steps/balance branches After scheduling last branch in conditional BBDT adds new step in BB3 after scheduling BB4, so we cannot do CS of operation f BBDT adds new step in BB3 after scheduling BB4, so we cannot do CS of operation f BB 7 + Resource Constraints Scheduled Being Scheduled No backtracking/re-scheduling of basic blocks/branches that have already been scheduled

20 BBDCM: Branch Balancing During Code Motion Operates while checking if an operation can be conditionally speculated into the scheduling step under consideration Operates while checking if an operation can be conditionally speculated into the scheduling step under consideration Checks resource utilization of all the basic blocks that the operation will be duplicated in Checks resource utilization of all the basic blocks that the operation will be duplicated in Tries to Find Idle Resources in all branches of the conditional Tries to Find Idle Resources in all branches of the conditional Inserts new step whenever possible if there is no idle resource: This is the Modification to Balance Branches Inserts new step whenever possible if there is no idle resource: This is the Modification to Balance Branches

21 Idle Resources A resource is said to be Idle in a scheduling step if There is no operation scheduled on it in that step There is no operation scheduled on it in that step For multi-cycle resources For multi-cycle resources if there is no operation scheduled on previous step(s) if there is no operation scheduled on previous step(s) if there is no operation scheduled on next step(s) if there is no operation scheduled on next step(s) If Node TF BB 1 BB 3 BB 2 BB 4 BB 5 BB 0 _ a + d _ e 2 Cycle Multiplier Previous Step Next Step X

22 BBDCM: Allow Conditional Speculation ? If Node 2 TF f BB 2 BB 4 BB 3 BB 5 BB 7 + c + e _ _ d BB 6 b _ a If Node 1 TF BB 0 BB 1 + While considering CS in last conditional BB While considering CS in last conditional BB Find Idle Resources in each basic block to duplicate in Find Idle Resources in each basic block to duplicate in If no idle resource in any BB If no idle resource in any BB If number of steps in BB ≤ number of steps in any other BB If number of steps in BB ≤ number of steps in any other BB Insert New scheduling step Insert New scheduling step + +Allow CS ? BB 7 + Resource Constraints Scheduled Being Scheduled

23 BBDCM: Allow Conditional Speculation ? If Node 2 TF f BB 2 BB 4 BB 3 BB 5 BB 7 + c + e _ _ d BB 6 b _ a If Node 1 TF BB 0 BB 1 + While considering CS in last conditional BB While considering CS in last conditional BB Find Idle Resources in each basic block to duplicate in Find Idle Resources in each basic block to duplicate in If no idle resource in any BB If no idle resource in any BB If number of steps in BB ≤ number of steps in any other BB If number of steps in BB ≤ number of steps in any other BB Insert New scheduling step Insert New scheduling step + ++ BB 7 + Resource Constraints Scheduled Being Scheduled Allow CS ?

24 BBDCM: Allow Conditional Speculation ? If Node 2 TF f BB 2 BB 4 BB 3 BB 5 BB 7 + c + e _ _ d BB 6 b _ a If Node 1 TF BB 0 BB 1 + While considering CS in last conditional BB While considering CS in last conditional BB Find Idle Resources in each basic block to duplicate in Find Idle Resources in each basic block to duplicate in If no idle resource in any BB If no idle resource in any BB If number of steps in BB ≤ number of steps in any other BB If number of steps in BB ≤ number of steps in any other BB Insert New scheduling step Insert New scheduling step ++ f f BB 7 + Resource Constraints Scheduled Being Scheduled BBDCM inserts new scheduling steps while applying code motions  if it enables Conditional Speculation BBDCM inserts new scheduling steps while applying code motions  if it enables Conditional Speculation

25 Implementation: SPARK High Level Synthesis Framework C Input => RTL VHDL Output VHDL => Logic Synthesis Results Customizable Scheduler Modular toolbox of transformations Heuristics select transformations Branch Balancing Algorithms Integrate with Scheduling Heursitics

26 Experiments: Target Applications Design # of Ifs # of Loops # Non-Empty Basic Blocks # of Operations MPEG-1 pred MPEG-1 pred GIMPtiler MPEG-1 Prediction Block MPEG-1 Prediction Block GIMP Image Processing software GIMP Image Processing software

27 All Code Motions except CS + Conditional Spec (CS) CS+BBDT: Add Steps during Traversal CS+BBDCM: Add steps during CS All Code Motions+CS+BBDT+BBDCM Experimental Results Effectiveness of Conditional Speculation is limited without Branch Balancing Algorithms All Code Motions except CS + Conditional Spec (CS) CS+Algo 1: Add Steps during Scheduling CS+Algo 2: Insert steps during CS All Code Motions+CS+Algo 1+Algo 2 Inserting Scheduling Steps during Traversal (BBDT) and while applying Conditional Speculation (BBDCM) improves Scheduling Results by %

28 Conclusions Two Branch Balancing Techniques to Increase the Effectiveness of Code Motions – specifically Conditional Speculation Two Branch Balancing Techniques to Increase the Effectiveness of Code Motions – specifically Conditional Speculation Manage resource utilization of multiple basic blocks Manage resource utilization of multiple basic blocks Insert Scheduling Steps in Unbalanced Conditional Branches dynamically during scheduling Insert Scheduling Steps in Unbalanced Conditional Branches dynamically during scheduling Implemented in comprehensive High-Level Synthesis framework: Synthesizes Behavioral C to RTL VHDL Implemented in comprehensive High-Level Synthesis framework: Synthesizes Behavioral C to RTL VHDL Demonstrated effectiveness on large industrial applications Demonstrated effectiveness on large industrial applications With Profiling Information: insert steps in less taken Conditional Branches With Profiling Information: insert steps in less taken Conditional Branches

29 Recent Related Work Scheduling designs with conditionals Scheduling designs with conditionals Condition Vector List Scheduling [Wakabayashi 89] Condition Vector List Scheduling [Wakabayashi 89] Path Based Scheduling [Camposano 91] Path Based Scheduling [Camposano 91] Symbolic Scheduling [Radivojevic 96] Symbolic Scheduling [Radivojevic 96] WaveSched Scheduler [Lakshminarayana 98] WaveSched Scheduler [Lakshminarayana 98] Basic Block Control Graph Scheduling [Santos 99] Basic Block Control Graph Scheduling [Santos 99] Early work was on data-intensive DSP algorithms Early work was on data-intensive DSP algorithms Pipelining, Algorithmic transformations Pipelining, Algorithmic transformations

30 Thank You