1. EENG449b/Savvides Lec 13.1 2/24/05 February 24, 2005 Prof. Andreas Savvides Spring 2005 http://www.eng.yale.edu/courses/eeng449bG EENG 449bG/CPSC 439bG Computer Systems Lecture 13 ARM Performance Issues and Programming

2. EENG449b/Savvides Lec 13.2 2/24/05 ARM Thumb Benchmark Performance Dhrystone benchmark result Memory System Performance

3. EENG449b/Savvides Lec 13.3 2/24/05

4. EENG449b/Savvides Lec 13.4 2/24/05 ARM vs. THUMB “THUMB code will provide up to 65% of the code size and 160% of an equivalent ARM connected to a 16-bit memory system”. Advatage of ARM over THUMB –Able to manipulate 32-bit integers in a single instruction THUMB’s advantage over 32-bit architectures with 16-bit instructions –It can swith back and forth between 16-bit and 32-bit instructions –Fast interrupts & DSP Algorithms can be implemented in 32-bits and processor can switch back and forth

5. EENG449b/Savvides Lec 13.5 2/24/05

6. EENG449b/Savvides Lec 13.6 2/24/05 Not the case when you have loads and stores!!!!

7. EENG449b/Savvides Lec 13.7 2/24/05

8. EENG449b/Savvides Lec 13.8 2/24/05 Optimizing Code Execution in Hardware ARM7 uses a three stage pipeline Each instruction takes 3 cycles to execute but has a CPI of 1 2 Possible ways to increase performance –Increase CPU frequency –Reduce CPI (increase pipeline stages & optimizations)

9. EENG449b/Savvides Lec 13.9 2/24/05 Where are the other bottlenecks? Exceptions Stalls due to Memory Accesses Inefficient Software

10. EENG449b/Savvides Lec 13.10 2/24/05 Exceptions & Memory Performance Refer to handout from last class for exceptions discussion –Lec 11 of handout for exceptions –Lec 10 of handout for memory

11. EENG449b/Savvides Lec 13.11 2/24/05

12. EENG449b/Savvides Lec 13.12 2/24/05 Exception Priorities & Latencies Exception Priorities 1. Reset 2. Data Abort 3. FIQ 4. IRQ 5. Prefetch Abort 6.Software Interrupt Interrupt Latencies –FIQ Worst case: Time to pass through synchronizer + time for longest instruction to complete +time for data abort entry + time for FIQ entry = 1.4us on a 20MHz processor -IRQ Worst case Same as FIQ but if FIQ occurs right before, then you need 2xFIQ latency

13. EENG449b/Savvides Lec 13.13 2/24/05 Interrupts FIQ – does not require saving context. –ARM has sufficient state to bypass this –When leaving the interrupt handler, program should execute »SUBS PC, R14_fiq,#4 IRQ – lower priority, masked out when FIQ is entered –When leaving the hander, program should execute »SUBS PC,R14_irq,#4 Software Interrupt: SWI –Returning handler should execute »MOV PC, R14_svc

14. EENG449b/Savvides Lec 13.14 2/24/05 Exceptions

15. EENG449b/Savvides Lec 13.15 2/24/05 Instruction Latencies

16. EENG449b/Savvides Lec 13.16 2/24/05 Bus Cycle Types Nonsequential –requests a transfer to and from an address which is unrelated to the address used in the preceding cycle Sequencial –Requests a transfer to or from an address which is either the same, one word or one halfword grater than the address used in the preceding cycle Internal –Does not require transfer because it is performing an internal function

17. EENG449b/Savvides Lec 13.17 2/24/05 Optimizing Code In Software Try to perform similar optimizations as the compiler does –Loop unrolling –Eliminate redundant code –Optimize memory accesses »Use multiple load and store instructions »Avoid using 32-bit data types- this will reduce your load and store performance.

18. EENG449b/Savvides Lec 13.18 2/24/05 Note that we are switching to MIPS architecture to discuss software optimizations…

19. EENG449b/Savvides Lec 13.19 2/24/05 Running Example This code, adds a scalar to a vector: for (i=1000; i>0; i=i–1) x[i] = x[i] + s; Assume following latency all examples InstructionInstructionExecutionLatency producing resultusing result in cyclesin cycles FP ALU opAnother FP ALU op 4 3 FP ALU opStore double 3 2 Load doubleFP ALU op 1 1 Load doubleStore double 1 0 Integer opInteger op 1 0

20. EENG449b/Savvides Lec 13.20 2/24/05 FP Loop: Where are the Hazards? Loop:L.DF0,0(R1);F0=vector element ADD.DF4,F0,F2;add scalar from F2 S.D0(R1),F4;store result DADDUIR1,R1,#-8;decrement pointer 8B (DW) BNEZR1,Loop;branch R1!=zero NOP;delayed branch slot Where are the stalls? First translate into MIPS code: -To simplify, assume 8 is lowest address

21. EENG449b/Savvides Lec 13.21 2/24/05 FP Loop Showing Stalls 10 clocks: Rewrite code to minimize stalls? InstructionInstructionLatency in producing resultusing result clock cycles FP ALU opAnother FP ALU op3 FP ALU opStore double2 Load doubleFP ALU op1 1 Loop:L.DF0,0(R1);F0=vector element 2stall 3ADD.DF4,F0,F2;add scalar in F2 4stall 5stall 6 S.DF4, 0(R1);store result 7 DADDUIR1,R1,#-8;decrement pointer 8B (DW) 8stall 9 BNER1,Loop;branch R1!=zero 10stall;delayed branch slot

22. EENG449b/Savvides Lec 13.22 2/24/05 Revised FP Loop Minimizing Stalls 6 clocks, but just 3 for execution, 3 for loop overhead; How make faster? InstructionInstructionLatency in producing resultusing result clock cycles FP ALU opAnother FP ALU op3 FP ALU opStore double2 Load doubleFP ALU op1 1 Loop:L.DF0,0(R1) 2DADDUIR1,R1,#-8 3ADD.DF4,F0,F2 4stall 5BNER1,R2, Loop;delayed branch 6 S.DF4, 8(R1) Swap BNE and S.D by changing address of S.D

23. EENG449b/Savvides Lec 13.23 2/24/05 Unroll Loop Four Times (straightforward way) Rewrite loop to minimize stalls? 1 Loop:L.DF0,0(R1) 2ADD.DF4,F0,F2 3S.D0(R1),F4 ;drop DADDUI & BNE 4L.DF6,-8(R1) 5ADD.DF8,F6,F2 6S.DF8,-8(R1) ;drop DADDUI & BNE 7L.DF10,-16(R1) 8ADD.DF12,F10,F2 9S.DF12,-16(R1) ;drop DADDUI & BNE 10L.DF14,-24(R1) 11ADD.DF16,F14,F2 12S.DF16,-24(R1) 13DADDUIR1,R1,#-32;alter to 4*8 14BNER1,LOOP 14 + (4 x (1+2))+ 2= 28 clock cycles, or 7 per iteration 1 cycle stall 2 cycles stall 1 cycle stall 1 cycle stall (delayed branch)

24. EENG449b/Savvides Lec 13.24 2/24/05 Unrolled Loop Detail Do not usually know upper bound of loop Suppose it is n, and we would like to unroll the loop to make k copies of the body Instead of a single unrolled loop, we generate a pair of consecutive loops: –1st executes (n mod k) times and has a body that is the original loop –2nd is the unrolled body surrounded by an outer loop that iterates (n/k) times –For large values of n, most of the execution time will be spent in the unrolled loop Problem: Although it improves execution performance, it increases the code size substantially!

25. EENG449b/Savvides Lec 13.25 2/24/05 Unrolled Loop That Minimizes Stalls (scheduled based on the latencies from slide 4) What assumptions made when moved code? –OK to move store past DSUBUI even though changes register –OK to move loads before stores: get right data? –When is it safe for compiler to do such changes? 1 Loop:L.DF0,0(R1) 2L.DF6,-8(R1) 3L.DF10,-16(R1) 4L.DF14,-24(R1) 5ADD.DF4,F0,F2 6ADD.DF8,F6,F2 7ADD.DF12,F10,F2 8ADD.DF16,F14,F2 9S.DF4, 0(R1) 10S.DF8, -8(R1) 11S.DF12, -16(R1) 12DADDUIR1,R1,#-32 13BNER1,LOOP 14S.DF16, 8(R1) ; 8-32 = -24 14 clock cycles, or 3.5 per iteration Better than 7 before scheduling and 6 when scheduled and not unrolled

26. EENG449b/Savvides Lec 13.26 2/24/05 Compiler Perspectives on Code Movement Compiler concerned about dependencies in program Whether or not a HW hazard depends on pipeline Try to schedule to avoid hazards that cause performance losses (True) Data dependencies (RAW if a hazard for HW) –Instruction i produces a result used by instruction j, or –Instruction j is data dependent on instruction k, and instruction k is data dependent on instruction i. If dependent, can’t execute in parallel Easy to determine for registers (fixed names) Hard for memory (“memory disambiguation” problem): –Does 100(R4) = 20(R6)? –From different loop iterations, does 20(R6) = 20(R6)?

27. EENG449b/Savvides Lec 13.27 2/24/05 Compiler Perspectives on Code Movement Name Dependencies are Hard to discover for Memory Accesses –Does 100(R4) = 20(R6)? –From different loop iterations, does 20(R6) = 20(R6)? Our example required compiler to know that if R1 doesn’t change then: 0(R1)  -8(R1)  -16(R1)  -24(R1) There were no dependencies between some loads and stores so they could be moved by each other

28. EENG449b/Savvides Lec 13.28 2/24/05 Steps Compiler Performed to Unroll Check OK to move the S.D after DADDUI and BNEZ, and find amount to adjust S.D offset Determine unrolling the loop would be useful by finding that the loop iterations were independent Rename registers to avoid name dependencies Eliminate extra test and branch instructions and adjust the loop termination and iteration code Determine loads and stores in unrolled loop can be interchanged by observing that the loads and stores from different iterations are independent –requires analyzing memory addresses and finding that they do not refer to the same address. Schedule the code, preserving any dependences needed to yield same result as the original code

29. EENG449b/Savvides Lec 13.29 2/24/05 Where are the name dependencies? 1 Loop:L.DF0,0(R1) 2ADD.DF4,F0,F2 3S.DF4,0(R1) ;drop DADDUI & BNE 4L.DF0,-8(R1) 5ADD.DF4,F0,F2 6S.DF4, -8(R1) ;drop DADDUI & BNE 7L.DF0,-16(R1) 8ADD.DF4,F0,F2 9S.DF4, -16(R1) ;drop DADDUI & BNE 10L.DF0,-24(R1) 11ADD.DF4,F0,F2 12S.DF4, -24(R1) 13DADDUIR1,R1,#-32;alter to 4*8 14BNER1,LOOP 15NOP How can remove them? (See pg. 310 of text)

30. EENG449b/Savvides Lec 13.30 2/24/05 Where are the name dependencies? 1 Loop:L.DF0,0(R1) 2ADD.DF4,F0,F2 3S.D0(R1),F4 ;drop DSUBUI & BNEZ 4L.DF6,-8(R1) 5ADD.DF8,F6,F2 6S.D-8(R1),F8 ;drop DSUBUI & BNEZ 7L.DF10,-16(R1) 8ADD.DF12,F10,F2 9S.D-16(R1),F12 ;drop DSUBUI & BNEZ 10L.DF14,-24(R1) 11ADD.DF16,F14,F2 12S.D-24(R1),F16 13DSUBUIR1,R1,#32;alter to 4*8 14BNEZR1,LOOP 15NOP The Orginal “register renaming” – instruction execution can be overlapped or in parallel

31. EENG449b/Savvides Lec 13.31 2/24/05 Limits to Loop Unrolling Decrease in the amount of loop overhead amortized with each unroll – After a few unrolls the loop overhead amortization is very small Code size limitations – memory is not infinite especially in embedded systems Compiler limitations – shortfall in registers due to excessive unrolling – register pressure – optimized code may loose its advantage due to the lack of registers

32. EENG449b/Savvides Lec 13.32 2/24/05 Static Branch Prediction Simplest: Predict taken –average misprediction rate = untaken branch frequency, which for the SPEC programs is 34%. –Unfortunately, the misprediction rate ranges from not very accurate (59%) to highly accurate (9%) Predict on the basis of branch direction? –choosing backward-going branches to be taken (loop) –forward-going branches to be not taken (if) –SPEC programs, however, most forward-going branches are taken => predict taken is better Predict branches on the basis of profile information collected from earlier runs –Misprediction varies from 5% to 22%

33. EENG449b/Savvides Lec 13.33 2/24/05 Next Time Quiz – no regular lecture March 3 – hardware optimizations and HW ILP