Autonomously Adaptive Computing: Coping with Scalability, Reliability, and Dynamism in Future Generations of Computing Roman Lysecky Department of Electrical and Computer Engineering University of Arizona rlysecky@ece.arizona.edu http://www.ece.arizona.edu/~embedded Collaborators: Frank Vahid, Greg Stitt

Introduction Past & Present: Standard Software Binaries Software Binaries of the Past Binary directly related to processor’s ISA Limited portability Instructions were executed as specified and in-order Current Software Binary Specifies application functionality but not specific to underlying processor architecture Develop new architectures for existing applications SW Application Compiler SW Binary Processor Architecture Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Introduction Past & Present: Standard Software Binaries Standard SW Binary Enabled ecosystem of applications, compilers, and architectures Provided separation of concerns Applications: Developers can focus on application Choose appropriate programming language to capture functionality Architectures: Focus on improving and developing new architectures to execute SW binary more better Compilers: Focus on optimizing application for specific architecture SW Application Compiler SW Binary Processor Architecture Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Introduction Past & Present: Standard Software Binaries Processor Architectures Many alternative architectures can exists for a given standard binary Current Software Binary Specifies application functionality in well defined manner but not specific to underlying processor architecture Develop new architectures for existing applications SW Application Compiler SW Binary VLIW Processor Architecture SuperScalar Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Reconfigurable Computing Past & Present: FPGAs Field Programmable Gate Arrays (FPGAs) Reconfigurable device that can implement any circuit simply by downloading bits Basic logic elements are N-input look up table (LUT) and flip-flops Small N-address memory that stores truth table for any N-input logic function Arrays of LUTs and routing elements (called switch matrices (SM)) LUT a b c d e f o1 o2 o3 o4 FPGA SM CLB Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Reconfigurable Computing Past & Present: FPGAs FPGAs are sometimes better than microprocessors Provide concurrency from the bit-level to the application-level C Code for Bit Reversal Circuit for Bit Reversal Bit Reversed X Value Original X Value Requires only 1 cycle (speedup of 32x to 128x) Synthesis FPGA … x = (x >>16) | (x <<16); x = ((x >> 8) & 0x00ff00ff) | ((x << 8) & 0xff00ff00); x = ((x >> 4) & 0x0f0f0f0f) | ((x << 4) & 0xf0f0f0f0); x = ((x >> 2) & 0x33333333) | ((x << 2) & 0xcccccccc); x = ((x >> 1) & 0x55555555) | ((x << 1) & 0xaaaaaaaa); Compilation sll $v1[3],$v0[2],0x10 srl $v0[2],$v0[2],0x10 or $v0[2],$v1[3],$v0[2] srl $v1[3],$v0[2],0x8 and $v1[3],$v1[3],$t5[13] sll $v0[2],$v0[2],0x8 and $v0[2],$v0[2],$t4[12] srl $v1[3],$v0[2],0x4 and $v1[3],$v1[3],$t3[11] sll $v0[2],$v0[2],0x4 and $v0[2],$v0[2],$t2[10] ... Processor SW Binary Requires between 32 and 128 cycles Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Reconfigurable Computing Past & Present: FPGAs SW Application Software HW Circuit Compiler Synthesis SW Binary Software Bitstream Processor FPGA Hardware FPGA can implement “circuits” simply by download “software” bits Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Reconfigurable Computing Past & Present: FPGAs FPGAs can be combined with microprocessors Benefits of HW/SW Partitioning Speedup of 2X to 10X 1000X possible for some highly parallelizable applications Energy reduction of 25% to 95% Software Application (C/C++) Application Profiling Critical Kernels Partitioning µP I$ D$ COPROCESSOR (FPGA) HW SW Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Reconfigurable Computing Past & Present: FPGAs Why aren’t FPGAs common? Programmability Bitstream not standardized Solution: Hide FPGA from application developer Just like the underlying processor architecture is hidden Software Application (C/C++) Application Profiling Critical Kernels Partitioning µP I$ D$ COPROCESSOR (FPGA) HW SW Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Present: Warp Processing PROFILER DYNAMICALLY DETECTS APPLICATION’S KERNELS 2 Profiler APPLICATION INITIALLY EXECUTES ON MICROPROCESSOR 1 µP I$ D$ ON-CHIP CAD MAPS KERNELS ONTO FPGA 3 WARPED EXECUTION IS 2-100X FASTER – OR – CONSUMES 75% LESS POWER 5 W-FPGA On-chip CAD CONFIGURE FPGA AND UPDATE APPLICATION BINARY 4 Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Present: Warp Processing SW Binary Bitstream Circuit Partitioning Updated SW Binary Decompilation RT Synthesis Binary Update JIT FPGA Compilation Logic Synthesis Tech. Mapping/Packing Placement Routing µP On-chip CAD I$ D$ Profiler W-FPGA Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Present: Warp Processing Warp Processing – Adaptive Computing Embeds compiler/synthesis within architecture Autonomously adapts/optimizes software binary at runtime to improve performance or reduce power consumption Performance-Driven Warp Processing (Low End) Goal: Maximize application performance over software execution Target: Low to mid-range embedded processors E.g., 100-200 MHz ARM processor Average speedup of 7.4X across several embedded benchmark applications µP On-chip CAD I$ D$ Profiler W-FPGA Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Present: Warp Processing Performance-Driven Warp Processing (High End) Target: High-end 624 MHz XScale processor Average speedup of 2.5X compared to 624 MHz XScale processor µP On-chip CAD I$ D$ Profiler W-FPGA Max Speedup: 6X Avg Speedup: 2.5X Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Past & Present: Warp Processing Low-Power Warp Processing Goal: Reduce overall power consumption without any degradation in performance Leverage dynamic voltage/frequency scaling of processor and FPGA µP On-chip CAD I$ D$ Profiler W-FPGA 1 4 7 APPLICATION EXEC (µP/FPGA) SW EXECUTION (µP) WARPED HW/SW EXECUTION (µP/FPGA) LOW-POWER WARPED EXECUTION 2 3 5 6 ON-CHIP CAD PROFILE ON-CHIP CAD µP V/FREQ SCALING FPGA FREQ SCALING SW POWER POWER POWER REDUCTION NO PERF. DECREASE PERFORMANCE SW PERF Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Past & Present: Warp Processing Low-Power Warp Processing Goal: Reduce overall power consumption without any degradation in performance Leverage dynamic voltage/frequency scaling of processor and FPGA Average reduction in power consumption of 74% µP On-chip CAD I$ D$ Profiler W-FPGA Avg Reduction: 74% Max Reduction: 97% Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Present: Warp Processing Maintains ecosystem supported by standard software binary Optimize software binary execution without developer effort – or even knowledge thereof Builds upon software binary concept to leverage benefits of FPGAs For those applications where FPGAs are beneficial Optimized at runtime, where additional information may be known µP On-chip CAD I$ D$ Profiler W-FPGA Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Present: Warp Processing Maintains ecosystem supported by standard software binary Optimize software binary execution without developer effort – or even knowledge thereof Builds upon software binary concept to leverage benefits of FPGAs For those applications where FPGAs are beneficial Optimized at runtime, where additional information may be known µP On-chip CAD I$ D$ Profiler W-FPGA Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Compile/Optimize/ Reconfigure Adaptive Computing Future: How can adaptive computing be used to solve future challenges? Adaptive Computing Any computing systems that can modify its execution at runtime How the system modifies its execution defines a systems adaptability How can adaptive computing be used to solve future challenges? Modified Application Dynamically Monitor Compile/Optimize/ Reconfigure Adapt system? Application Binary SW Application Compiler SW Binary Processor Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Computing Challenges: Multicore and Many Core Systems Computing systems have been and will continue to incorporate multiple processor cores How can applications developers best reap the rewards of theses parallel implementations? How can compilers and architectures optimize an application? Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Computing Challenges: Multicore and Many Core Systems As the number of cores increases, a many core systems looks more and more like a reconfigurable computing device The biggest challenge moving forward is programmability Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Computing Challenges: Multicore and Many Core Systems Adaptive computing for multicore and many core systems Adaptive computing can be utilized to optimize an application execution on a many core device Application execution can be optimized to utilize the best resources for the current operations Runtime re-compilation, optimization, and synthesis Can optimize the application utilizing additional information gather at runtime FPGA DSP VLIW Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Computing Challenges: Dynamism Application behavior is increasingly becoming more complex and dynamic Statically optimizing an application at compile time may lead to sub-optimal results Analyzing both the computational aspects and data characteristics being processed are needed in order to fully optimize the application execution Data being processed is not known a priori Data Characteristics Application Binary Dynamically Monitor Adapt system? Compile/Optimize/ Reconfigure Modified Application Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Computing Challenges: Reliability Permanent and transient device failures are expected to increase as we continue to shrink Electro migration NBTI Individual hardware components may fail intermittently over time Applications should execute as if no failures occurred Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Adaptive Computing Computing Challenges: Reliability Self-healing Systems Computing systems that can detect and recover from device failures autonomously Different from fault tolerant systems Application Dynamically Monitor Is system executing correctly? No Heal System Modified Application/Architecture Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008

Summary Adaptive computing can autonomously optimize application execution at runtime Continuously and dynamically re-compile/synthesizes software implementations to hardware at runtime Can optimize performance, power, reliability, … Adaptive computing will be both beneficial and necessary for future computing systems Let developers focus on developing applications, not worrying about underlying architectures Kavli-NNIN Symposium on Computing Challenges, Cornell, October 12-14, 2008