Instruction Rescheduling and Loop-Unroll Department of Computer Science Southern Illinois University Edwardsville Fall, 2015 Dr. Hiroshi Fujinoki Loop_Unroll/000 CS 312 Computer Architecture & Organization

Loop_Unroll/001 Example A for-loop structure written in a high-level programming language for (i = 0; i < 1000; i++) { a[i] = a[i] ; } There are an array of floating-point number, a[i], which has 1,000 elements Add a constant, 10.19, to every element in the FP array Loop-unrolling Loop-unrolling is a technique to increase ILP for loop-structure CS 312 Computer Architecture & Organization

Loop_Unroll/002 Main Memory Assumptions (High Address) (Low Address) FFFFFF a[0] a[1] a[999]          8 bytes R1 R2 F2 = CS 312 Computer Architecture & Organization

L.D F0, 0(R1) // F0 = Mem[R1] Data_Dependency/003 After the high-level programming language statements are compiled for (i = 0; i < 1000; i++) { a[i] = a[i] ; } ADD.D F4, F0, F2 // F4 = F0+F2 S.D F4, 0(R1) // Mem[R1] = F4 DADDUI R1, R1, -8 // R1 = R1-8 BNE R1, R2, LOOP // R1  R2  LOOP LOOP: L.D F0, 0(R1) // F0 = Mem[R1] ADD.D F4, F0, F2 // F4 = F0+F2 S.D F4, 0(R1) // Mem[R1] = F4 We focus on this loop structure BNE = “Branch if NOT EQUAL” CS 312 Computer Architecture & Organization

LOAD F0, 0(R1) // F0  Mem[R1] Data_Dependency/004 After the high-level programming language statements are compiled for (i = 0; i < 1000; i++) { a[i] = a[i] ; } ADD F4, F0, F2 // F4  F0+F2 STORE F4, 0(R1) // Mem[R1]  F4 ADD R1, R1, -8 // R1  R1-8 BNE R1, R2, LOOP // R1  R2  LOOP LOOP: CS 312 Computer Architecture & Organization

Loop_Unroll/005 Assumptions (Part 2) Branch slot (for a conditional branch) = 1 cycle RAW dependency for integer ALU instructions = 1 cycle Numbers of stalled cycles for this CPU are defined as follow: Instruction producing result Instruction using result Stalled cycles FP ALU operation Another FP ALU operation FP ALU operation Store FP data Load FP data FP ALU operation Load FP data Store FP data 0 (This table appears in page 304 of the textbook) READ WRITE CS 312 Computer Architecture & Organization

Data_Dependency/006 Categorizing instruction types LOAD F0, 0(R1) // F0  Mem[R1] ADD F4, F0, F2 // F4  F0+F2 STORE F4, 0(R1) // Mem[R1]  F4 ADD R1, R1, -8 // R1  R1-8 BNE R1, R2, LOOP // R1  R2  LOOP LOOP: Floating-Point instructions Integer instructions Conditional branch CS 312 Computer Architecture & Organization

Data_Dependency/007 Identifying all pipeline hazards LOAD F0, 0(R1) // F0  Mem[R1] ADD F4, F0, F2 // F4  F0+F2 STORE F4, 0(R1) // Mem[R1]  F4 ADD R1, R1, -8 // R1  R1-8 BNE R1, R2, LOOP // R1  R2  LOOP LOOP: RAW WAR RAW Control Hazard CS 312 Computer Architecture & Organization

Data_Dependency/008 Determining stalled and flashed cycles L.D F0, 0(R1) // F0  Mem[R1] ADD.D F4, F0, F2 // F4  F0+F2 S.D F4, 0(R1) // Mem[R1]  F4 DADDUI R1, R1, -8 // R1  R1-8 BNE R1, R2, LOOP // R1  R2  LOOP LOOP: FP Load FP ALU RAW 1 FP Store 2 RAW Int ALU 0 RAW Branch 1 # of stalls Control Hazard (1 cycle flash) How many cycles stalled or flashed due to RAW and Control hazard? CS 312 Computer Architecture & Organization

Data_Dependency/009 Instruction issuing schedule w/ stalls and flash ADD.D F4, F0, F2 // F4  F0+F2 S.D F4, 0(R1) // Mem[R1]  F4 DADDUI R1, R1, -8 // R1  R1-8 BNE R1, R2, LOOP // R1  R2  LOOP L.D F0, 0(R1) // F0  Mem[R1] LOOP: stall Cycle Issued flash CS 312 Computer Architecture & Organization

Data_Dependency/0010 Technique #4: Instruction Re-Scheduling ADD.D F4, F0, F2 // F4  F0+F2 S.D F4, 0(R1) // Mem[R1]  F4 DADDUI R1, R1, -8 // R1  R1-8 BNE R1, R2, LOOP // R1  R2  LOOP L.D F0, 0(R1) // F0  Mem[R1] LOOP: stall Cycle Issued flash CS 312 Computer Architecture & Organization

Data_Dependency/011 Technique #4: Instruction Re-Scheduling ADD.D F4, F0, F2 // F4  F0+F2 S.D F4, 0(R1) // Mem[R1]  F4 DADDUI R1, R1, -8 // R1  R1-8 BNE R1, R2, LOOP // R1  R2  LOOP L.D F0, 0(R1) // F0  Mem[R1] LOOP: stall Cycle Issued stall flash stall S.D F4, 8(R1) // Mem[R1]  F4 Loop Completed Here Make sure to add 8! Delayed-branch applied CS 312 Computer Architecture & Organization

Data_Dependency/012 Technique #5: Loop-Unrolling ADD F4, F0, F2 STORE F4, 0(R1) ADD R1, R1, -8 BNE R1, R2, LOOP LOAD F0, 0(R1) LOOP: stall flash We repeat this for 1,000 times CS 312 Computer Architecture & Organization

Data_Dependency/013 Technique #5: Loop-Unrolling We repeat this for 1,000 times ADD F4, F0, F2 STORE F4, 0(R1) ADD R1, R1, -8 BNE R1, R2, LOOP LOAD F0, 0(R1) LOOP1: stall flash ADD F4, F0, F2 STORE F4, 0(R1) LOAD F0, 0(R1) LOOP2: stall ADD R1, R1, -8 BNE R1, R2, LOOP stall flash Merge Them Together CS 312 Computer Architecture & Organization

Data_Dependency/014 Technique #5: Loop-Unrolling ADD F4, F0, F2 STORE F4, 0(R1) LOAD F0, 0(R1) LOOP1: stall ADD R1, R1, -8 BNE R1, R2, LOOP stall flash ADD F4, F0, F2 STORE F4, 0(R1) LOAD F0, 0(R1) LOOP2: ADD R1, R1, -8 BNE R1, R2, LOOP stall flash LOAD F6, 8(R1) ADD F8, F6, F2 ADD R1, R1, -16 BNE R1, R2, LOOP stall flash STORE F8, 8(R1) WAW Dependency (Pseudo Dependency) = Name Dependency CS 312 Computer Architecture & Organization

Data_Dependency/015 Technique #5: Loop-Unrolling ADD F4, F0, F2 STORE F4, 0(R1) LOAD F0, 0(R1) LOOP1: stall LOAD F6, 8(R1) ADD F8, F6, F2 ADD R1, R1, -16 BNE R1, R2, LOOP stall flash STOTE F8, 8(R1) Previous: 10 Cycles  1,000 Now: 11 Cycles  500 CS 312 Computer Architecture & Organization

Data_Dependency/016 Further Improvement Is further improvement possible? Especially eliminate especially control hazards  Combine instruction-scheduling (Technique 4) and Loop-unrolling  More loop-unrolling Further eliminate stalls But how many loop-unrolling should be performed? CS 312 Computer Architecture & Organization

Data_Dependency/017 How many loop-unrolling should be performed? Too many unrolling Too few unrolling The best unrolling Loop size becomes too big Stalls still exist Only enough to eliminate stalls How can we know the best unrolling if number of loops is unknown before run-time? Exercise 2.7 (p. 144 in ED4 (Exercise 4.4 in ED3) CS 312 Computer Architecture & Organization

Code Optimization Examples by Visual Studio 2010 Data_Dependency/018 CS 312 Computer Architecture & Organization