Warp Processor: A Dynamically Reconfigurable Coprocessor Frank Vahid Professor Department of Computer Science and Engineering University of California, Riverside Associate Director, Center for Embedded Computer Systems, UC Irvine Work supported by the National Science Foundation, the Semiconductor Research Corporation, Xilinx, Intel, Motorola/Freescale Contributing Ph.D. Students: Roman Lysecky (2005, now asst. prof. at U. Arizona), Greg Stitt (Ph.D. 2006), Kris Miller (MS 2007), David Sheldon (3rd yr PhD), Scott Sirowy (1st yr PhD)

Outline Intro and Background: Warp Processors Work in progress under SRC 3-yr grant Parallelized-computation memory access Deriving high-level constructs from binaries Case studies Using commercial FPGA fabrics Application-specific FPGA Other ongoing related work Configurable cache tuning Frank Vahid, UC Riverside

Intro: Partitioning to FPGA Custom ASIC coprocessor known to speedup sw kernels Energy advantages too (e.g., Henkel’98, Rabaey’98, Stitt/Vahid’04) Power savings even on FPGA (Stitt/Vahid IEEE D&T’02, IEEE TECS’04) Con: more silicon (~10x), less power savings Pro: platform fully programmable Mass-produced Application Proc. ASIC Application Proc. FPGA Frank Vahid, UC Riverside

Intro: FPGA vs. ASIC Coprocessor – FPGA Surprisingly Competitive FPGA 34% savings versus ASIC’s 48% (Stitt/Vahid IEEE D&T’02, IEEE TECS’04) 70% energy savings & 5.4 speedup vs. 200 MHz MIPS (Stitt/Vahid DATE’05) Frank Vahid, UC Riverside

FPGA – Why (Sometimes) Better than Software C Code for Bit Reversal Hardware for Bit Reversal Bit Reversed X Value . . . . . . . . . . . Original X Value x = (x >>16) | (x <<16); x = ((x >> 8) & 0x00ff00ff) | ((x << 8) & 0xff00ff00); x = ((x >> 4) & 0x0f0f0f0f) | ((x << 4) & 0xf0f0f0f0); x = ((x >> 2) & 0x33333333) | ((x << 2) & 0xcccccccc); x = ((x >> 1) & 0x55555555) | ((x << 1) & 0xaaaaaaaa); sll $v1[3],$v0[2],0x10 srl $v0[2],$v0[2],0x10 or $v0[2],$v1[3],$v0[2] srl $v1[3],$v0[2],0x8 and $v1[3],$v1[3],$t5[13] sll $v0[2],$v0[2],0x8 and $v0[2],$v0[2],$t4[12] srl $v1[3],$v0[2],0x4 and $v1[3],$v1[3],$t3[11] sll $v0[2],$v0[2],0x4 and $v0[2],$v0[2],$t2[10] ... Binary Compilation Processor Requires between 32 and 128 cycles Processor FPGA Requires only 1 cycle (speedup of 32x to 128x) Other big reason: Concurrency Frank Vahid, UC Riverside

Warp Processing – Dynamic Partitioning of Sw Kernels to FPGA Profile application to determine critical regions 2 Profiler Partition critical regions to hardware 3 Dynamic Part. Module (DPM) Initially execute application in software only 1 µP I$ D$ Profiler µP I$ Partitioned application executes faster with lower energy consumption 5 D$ Program configurable logic & update software binary 4 FPGA FPGA Dynamic Part. Module (DPM) Frank Vahid, UC Riverside

Warp Processors – Dynamic Partitioning Binary SW Advantages vs. compiler-time partitioning No special compilers Completely transparent Separates function and architecture for architectures having FPGAs Avoid complexities of supporting different FPGAs Potentially brings FPGA advantages to ALL software Profiling Standard Compiler Binary Traditional partitioning done here Profiling CAD Tools FPGA Proc. Profiling CAD Tools FPGA Proc. Frank Vahid, UC Riverside

Warp Processing Steps (On-Chip CAD) Binary Decompilation HW Bitstream RT Synthesis Partitioning Binary Updater Updated Binary Std. HW Binary JIT FPGA Compilation Tech. Mapping/Packing Placement Logic Synthesis Routing µP I$ D$ WCLA (FPGA) Profiler DPM (CAD) Frank Vahid, UC Riverside

Warp Processing – Partitioning Applications spend much time in small amount of code 90-10 rule Observed 75-4 rule for MediaBench, NetBench Potentially large perfomance/ energy benefits from implementing critical regions in hardware Use profiling results to identify critical regions µP I$ D$ WCLA (FPGA) Profiler DPM (CAD) Frank Vahid, UC Riverside

Warp Processing – Decompilation Synthesis from binary has a challenge High-level information (e.g., loops, arrays) lost during compilation Solution –Recover high-level information: decompilation Corresponding Assembly long f( long reg2 ) { int reg3 = 0; int reg4 = 0; loop: reg4 = reg4 + mem[reg2 + reg3 << 1)]; reg3 = reg3 + 1; if (reg3 < 10) goto loop; return reg4; } Function Recovery long f( short array[10] ) { long reg4 = 0; for (long reg3 = 0; reg3 < 10; reg3++) { reg4 += array[reg3]; } return reg4; Array Recovery long f( long reg2 ) { long reg4 = 0; for (long reg3 = 0; reg3 < 10; reg3++) { reg4 += mem[reg2 + (reg3 << 1)]; } return reg4; Control Structure Recovery loop: reg1 := reg3 << 1 reg5 := reg2 + reg1 reg6 := mem[reg5 + 0] reg4 := reg4 + reg6 reg3 := reg3 + 1 if (reg3 < 10) goto loop ret reg4 reg3 := 0 reg4 := 0 Control/Data Flow Graph Creation loop: reg4 := reg4 + mem[ reg2 + (reg3 << 1)] reg3 := reg3 + 1 if (reg3 < 10) goto loop ret reg4 reg3 := 0 reg4 := 0 Data Flow Analysis Original C Code long f( short a[10] ) { long accum; for (int i=0; i < 10; i++) { accum += a[i]; } return accum; Mov reg3, 0 Mov reg4, 0 loop: Shl reg1, reg3, 1 Add reg5, reg2, reg1 Ld reg6, 0(reg5) Add reg4, reg4, reg6 Add reg3, reg3, 1 Beq reg3, 10, -5 Ret reg4 Almost Identical Representations Frank Vahid, UC Riverside

Warp Processing – Decompilation Earlier study Synthesis after decompilation often quite similar Almost identical performance, small area overhead FPGA 2005 Frank Vahid, UC Riverside

Warp Processing – RT Synthesis Maps decompiled DFG operations to hw library components Adders, Comparators, Multiplexors, Shifters Creates Boolean expression for each output bit in dataflow graph r4[0]=r1[0] xor r2[0], carry[0]=r1[0] and r2[0] r4[1]=(r1[1] xor r2[1]) xor carry[0], carry[1]= ……. ……. r1 r2 + r4 r3 8 < r5 32-bit adder 32-bit comparator Frank Vahid, UC Riverside

Warp Processing – JIT FPGA Compilation Existing FPGAs require complex CAD tools FPGAs designed to handle large arbitrary circuits, ASIC prototyping, etc. Require long execution times and large memory usage Not suitable for dynamic on-chip execution 50 MB 60 MB 10 MB 1 min Log. Syn. Tech. Map 1-2 mins Place 2-30 mins Route Solution: Develop a custom CAD-oriented FPGA (WCLA – Warp Configurable Logic Architecture) Careful simultaneous design of FPGA and CAD FPGA features evaluated for impact on CAD Add architecture features for SW kernels Enables development of fast, lean JIT FPGA compilation tools 1s <1s .5 MB 1 MB 10s 3.6 MB Frank Vahid, UC Riverside

Warp Configurable Logic Architecture (WCLA) Data address generators (DADG) and loop control hardware(LCH) Provide fast loop execution Supports memory accesses with regular access pattern Integrated 32-bit multiplier-accumulator (MAC) Frequently found within critical SW kernels DADG & LCH Configurable Logic Fabric Reg0 32-bit MAC Reg1 Reg2 ARM I$ D$ WCLA Profiler DPM DATE’04 Frank Vahid, UC Riverside

Warp Configurable Logic Architecture (WCLA) CAD-specialized configurable logic fabric Simplified switch matrices Directly connected to adjacent CLB All nets are routed using only a single pair of channels Allows for efficient routing Simplified CLBs Two 3 input, 2 output LUTs Each CLB connected to adjacent CLB to simplify routing of carry chains Currently being prototyped by Intel (scheduled for 2006 Q3 shuttle) 0L 1 1L 2L 2 3L 3 LUT a b c d e f o1 o2 o3 o4 Adj. CLB Frank Vahid, UC Riverside DATE’04

Warp Processing – Logic Synthesis ROCM - Riverside On-Chip Minimizer Two-level minimization tool Combination of approaches from Espresso-II [Brayton, et al., 1984][Hassoun & Sasoa, 2002] and Presto [Svoboda & White, 1979] Single expand phase instead of multiple iterations Eliminate need to compute off-set – reduces memory usage On average only 2% larger than optimal solution Expand dc-set on-set off-set Reduce JIT FPGA Compilation Tech. Mapping/Packing Placement Logic Synthesis Routing Irredundant On-Chip Logic Minimization, DAC’03 A Codesigned On-Chip Logic Minimizer, CODES+ISSS’03 Frank Vahid, UC Riverside

Warp Processing – Technology Mapping ROCTM - Technology Mapping/Packing Decompose hardware circuit into DAG Nodes correspond to basic 2-input logic gates (AND, OR, XOR, etc.) Hierarchical bottom-up graph clustering algorithm Breadth-first traversal combining nodes to form single-output LUTs Combine LUTs with common inputs to form final 2-output LUTs Pack LUTs in which output from one LUT is input to second LUT JIT FPGA Compilation Tech. Mapping/Packing Placement Logic Synthesis Routing Dynamic Hardware/Software Partitioning: A First Approach, DAC’03 A Configurable Logic Fabric for Dynamic Hardware/Software Partitioning, DATE’04 Frank Vahid, UC Riverside

Warp Processing – Placement ROCPLACE - Placement Dependency-based positional placement algorithm Identify critical path, placing critical nodes in center of CLF Use dependencies between remaining CLBs to determine placement Attempt to use adjacent CLB routing whenever possible CLB CLB CLB CLB CLB CLB JIT FPGA Compilation Tech. Mapping/Packing Placement Logic Synthesis Routing Dynamic Hardware/Software Partitioning: A First Approach, DAC’03 A Configurable Logic Fabric for Dynamic Hardware/Software Partitioning, DATE’04 Frank Vahid, UC Riverside

Warp Processing – Routing ROCR - Riverside On-Chip Router Requires much less memory than VPR as resource graph is smaller 10x faster execution time than VPR (Timing driven) Produces circuits with critical path 10% shorter than VPR (Routablilty driven) JIT FPGA Compilation Tech. Mapping/Packing Placement Logic Synthesis Routing Frank Vahid, UC Riverside Dynamic FPGA Routing for Just-in-Time FPGA Compilation, DAC’04

Experiments with Warp Processing Warp Processor ARM/MIPS plus our fabric Riverside on-chip CAD tools to map critical region to configurable fabric Requires less than 2 seconds on lean embedded processor to perform synthesis and JIT FPGA compilation Traditional HW/SW Partitioning ARM/MIPS plus Xilinx Virtex-E FPGA Manually partitioned software using VHDL VHDL synthesized using Xilinx ISE 4.1 ARM I$ D$ WCLA Profiler DPM ARM I$ D$ Xilinx Virtex-E FPGA Frank Vahid, UC Riverside

Warp Processors Performance Speedup (Most Frequent Kernel Only) Average kernel speedup of 41, vs. 21 for Virtex-E WCLA simplicity results in faster HW circuits SW Only Execution Frank Vahid, UC Riverside

Warp Processors Performance Speedup (Overall, Multiple Kernels) Assuming 100 MHz ARM, and fabric clocked at rate determined by synthesis Average speedup of 7.4 Energy reduction of 38% - 94% SW Only Execution Frank Vahid, UC Riverside

Warp Processors - Results Execution Time and Memory Requirements Xilinx ISE 60 MB 0.2 s DPM (CAD) 3.6MB 3.6MB 1.4s DPM (CAD) (75MHz ARM7) Frank Vahid, UC Riverside

Outline Intro and Background: Warp Processors Work in progress under SRC 3-yr grant Parallelized-computation memory access Deriving high-level constructs from binaries Case studies Using commercial FPGA fabrics Application-specific FPGA Other ongoing related work Configurable cache tuning Frank Vahid, UC Riverside

1. Parallelized-Computation Memory Access Problem Can parallelize kernel computation, but may hit memory bottleneck Solution Use more advanced memory/compilation methods A[i] c[i] + A[i+1] c[i+1] B[i] B[i+1] B[i+2] Parallelism can’t be exploited if data isn’t available Frank Vahid, UC Riverside

1. Parallelized-Computation Memory Access Method 1: Distribute data among FPGA block RAMS Concurrently accessible Memory accesses are parallelized Main Memory blockRAM blockRAM A[i] c[i] + A[i+1] c[i+1] B[i] B[i+1] A[i] c[i] + A[i+1] c[i+1] B[i] B[i+1] Frank Vahid, UC Riverside

1. Parallelized-Computation Memory Access Method 2: Smart Buffers (Najjar’2004) Memory structure optimized for application’s access patterns Takes advantage of data reuse Speedups of 2x to 10x compared to hw without smart buffers RAM 1st iteration window Controller Smart Buffer Datapath 2nd iteration window A[0] A[1] A[2] A[3] A[4] A[5] A[6] A[7] A[8] …. 3rd iteration window Smart Buffer Empty A[4] Killed A[0] A[1] A[2] A[3] Killed A[6] A[5] Killed RAM Frank Vahid, UC Riverside

2. Deriving high-level constructs from binaries Problem Some binary features unsuitable for synthesis Loops unrolled by optimizing compiler, or pointers Previous decompilation techniques didn’t consider Features OK for sw-to-sw translation, not for synthesis Solution – New decompilation techniques Convert pointers to arrays Reroll loops Others Loop Unrolling Loop Rerolling for (int i=0; i < 3; i++) accum += a[i]; Ld reg2, 100(0) Add reg1, reg1, reg2 Ld reg2, 100(1) Ld reg2, 100(2) for (int i=0; i<3;i++) reg1 += array[i]; Frank Vahid, UC Riverside

2. Deriving high-level constructs from binaries Recent study of decompilation robustness In presence of compiler optimizations, and instruction sets Energy savings of 77%/76%/87% for MIPS/ARM/Microblaze ICCAD’05 DATE’04 Frank Vahid, UC Riverside

3. Case Studies Compare warp processing (binary level) versus compiler-based (C level) partitioning for real examples H.264 study (w/ Freescale) Highly-optimized proprietary C code Results of 2 month study Competitive Also learned that simple C-coding guidelines improve synthesis Whether done from binary or source; presently developing guidelines More examples: IBM (server), others... Frank Vahid, UC Riverside

4. Using Commercial FPGA Fabrics Can warp processing utilize commercial FPGAs? Approach 1: “Virtual FPGA” – Map our fabric to FPGA Collaboration with Xilinx Initial results: 6x performance overhead, 100x area overhead Main problem is routing uP Warp fabric Map fabric onto a commercial fabric uP uP Commercial FPGA Warp fabric Investigating better methods (one-to-one mapping) “Virtual FPGA” Frank Vahid, UC Riverside

5. Application-Specific FPGA Commercial FPGAs intended for ASIC prototyping Huge range of possible designs Generality causes loss of efficiency Propose to investigate app-spec FPGAs Put on ASIC next to custom circuits and microprocessor FPGA tuned to particular circuit, but still general and reprogrammable Supports late changes, modifications to standards, etc. Customize CLB size, # of inputs, routing resources Coarse-grained components – multiply-accumulate, RAM, etc. Use retargetable CAD tool Expected results: smaller, faster FPGAs General Fabric CLB DSP-tuned Fabric MAC MAC RAM CLB CLB CLB Frank Vahid, UC Riverside

5. Application-Specific FPGA Initial results: Performance improvements up to 300%, area reductions up to 90% Frank Vahid, UC Riverside

Outline Intro and Background: Warp Processors Work in progress under SRC 3-yr grant Parallelized-computation memory access Deriving high-level constructs from binaries Case studies Using commercial FPGA fabrics Application-specific FPGA Other ongoing related work Configurable cache tuning Frank Vahid, UC Riverside

Configurable Cache Tuning Developed Runtime configurable cache (ISCA 2003) Configuration heuristics (DATE 2004, ISLPED 2005) – 60% memory-access energy savings Present focus: Dynamic tuning Way1 Way2 Way 3 Way 4 Way 1 Way 2 Way 1 Ways, line size, and total size are configurable ISLPED 2005 Frank Vahid, UC Riverside

Summary Basic warp technology Ongoing work (SRC) Developed 2002-2005 (NSF, and 1-year CSR grants from SRC) Uses binary synthesis and FPGAs Conclusion: Feasible technology, much potential Ongoing work (SRC) Improve and validate effectiveness of binary synthesis Examine FPGA implementation issues Extensive future work to develop robust warp technology Frank Vahid, UC Riverside