1. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB P A R A L L E L C O M P U T I N G L A B O R A T O R Y EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB A Case for FAME: FPGA Architecture Model Execution Zhangxi Tan, Andrew Waterman, Henry Cook, Sarah Bird, Krste Asanovic, David Patterson The Parallel Computing Lab, UC Berkeley ISCA ’10

2. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB A Brief History of Time  Hardware prototyping initially popular for architects  Prototyping each point in a design space is expensive  Simulators became popular cost-effective alternative  Software Architecture Model Execution (SAME) simulators most popular  SAME performance scaled with uniprocessor performance scaling 2

3. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB The Multicore Revolution  Abrupt change to multicore architectures  HW, SW systems larger, more complex  Timing-dependent nondeterminism  Dynamic code generation  Automatic tuning of app kernels  We need more simulation cycles than ever 3

4. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB The Multicore Simulation Gap  As number of cores increases exponentially, time to model a target cycle increases accordingly  SAME is difficult to parallelize because of cycle-by-cycle interactions  Relaxed simulation synchronization may not work  Must bridge simulation gap 4

5. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB One Decade of SAME Median Instructions Simulated/ Benchmark Median #Cores Median Instructions Simulated/ Core ISCA 1998267M1 ISCA 2008825M16100M  Effect is dramatically shorter (~10 ms) simulation runs 5

6. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB FAME: FPGA Architecture Model Execution  The SAME approach provides inadequate simulation throughput and latency  Need a fundamentally new strategy to maximize useful experiments per day  Want flexibility of SAME and performance of hardware  Ours: FPGA Architecture Model Execution (FAME)  (cf. SAME, Software Architecture Model Execution)  Why FPGAs?  FPGA capacity scaling with Moore’s Law  Now can fit a few cores on die  Highly concurrent programming model with cheap synchronization 6

7. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Non-FAME: FPGA Computers  FPGA Computers: using FPGAs to build a production computer  RAMP Blue (UCB 2006)  1008 MicroBlaze cores  No MMU, message passing only  Requires lots of hardware 21 BEE2 boards (full rack) / 84 FPGAs  RTL directly mapped to FPGA  Time-consuming to modify  Cool, useful, but not a flexible simulator 7

8. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB FAME: System Simulators in FPGAs 8 CORE D$ DRAM Shared L2$ / Interconnect … I$ CORE D$ I$ CORE D$ I$ CORE D$ I$ CORE D$ DRAM L2$ I$ CORE D$ I$ CORE D$ I$ L2$ Target System A Target System B Host System (FAME simulator)

9. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB A Vast FAME Design Space  FAME design space even larger than SAME’s  Three dimensions of FAME simulators  Direct or Decoupled: does one host cycle model one target cycle?  Full RTL or Abstract RTL?  Host Single-threaded or Host Multi-threaded?  See paper for a FAME taxonomy! 9

10. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB FAME Dimension 1: Direct vs. Decoupled  Direct FAME: compile target RTL to FPGA  Problem: common ASIC structures map poorly to FPGAs  Solution: resource-efficient multi-cycle FPGA mapping  Decoupled FAME: decouple host cycles from target cycles  Full RTL still modeled, so timing accuracy still guaranteed 10 R1 R2 R3 R4 W1 W2 RegFile Rd1 Rd2 Rd3 Rd4 R1 R2 W1 RegFile Rd1 Rd2 Target System RegfileDecoupled Host Implementation FSM

11. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB FAME Dimension 2: Full RTL vs. Abstract RTL  Decoupled FAME models full RTL of target machine  Don’t have full RTL in initial design phase  Full RTL is too much work for design space exploration  Abstract FAME: model the target RTL at a high level  For example, split timing and functional models (à la SAME)  Also enables runtime parameterization: run different simulations without re-synthesizing the design  Advantages of Abstract FAME come at cost: model verification  Timing of abstract model not guaranteed to match target machine 11 Abstraction Functional Model Target RTL Timing Model

12. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB FAME Dimension 3: Single- or Multi-threaded Host  Problem: can’t fit big manycore on FPGA, even abstracted  Problem: long host latencies reduce utilization  Solution: host-multithreading 12 CPU 1 CPU 2 CPU 3 CPU 4 Target Model Multithreaded Emulation Engine (on FPGA) +1 2 PC1 PC1 PC1 PC1 I$ IR GPR GPR GPR GPR1 X Y 2 D$ Single hardware pipeline with multiple copies of CPU state

13. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Metrics besides Cycles: Power, Area, Cycle Time  FAME simulators determine how many cycles a program takes to run  Computing Power/Area/Cycle Time: SAME old story  Push target RTL through VLSI flow  Analytical or empirical models  Collecting event stats for model inputs is much faster than with SAME 13

14. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB RAMP Gold: A Multithreaded FAME Simulator  Rapid accurate simulation of manycore architectural ideas using FPGAs  Initial version models 64 cores of SPARC v8 with shared memory system on $750 board  Hardware FPU, MMU, boots OS. Cost Performance (MIPS) Simulations per day Simics (SAME)$2,0000.1 - 11 RAMP Gold (FAME)$2,000 + $75050 - 100100 14

15. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB RAMP Gold Target Machine SPARC V8 CORE SPARC V8 CORE I$ D$ DRAM Shared L2$ / Interconnect SPARC V8 CORE SPARC V8 CORE I$ D$ SPARC V8 CORE SPARC V8 CORE I$ D$ SPARC V8 CORE SPARC V8 CORE I$ D$ … 64 cores 15

16. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB RAMP Gold Model Functional Model Pipeline Arch State Timing Model Pipeline Timing State  SPARC V8 ISA  One-socket manycore target  Split functional/timing model, both in hardware –Functional model: Executes ISA –Timing model: Capture pipeline timing detail  Host multithreading of both functional and timing models  Functional-first, timing- directed  Built for Xilinx Virtex-5 systems [ RAMP Gold, DAC ‘10 ] 16 CORE D$ DRAM Shared L2$ / Interconnect … 64 cores I$ CORE D$ I$ CORE D$ I$ CORE D$ I$

17. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Case Study: Manycore OS Resource Allocation  Spatial resource allocation in a manycore system is hard  Combinatorial explosion in number of apps and number of resources  Idea: use predictive models of app performance to make it easier on OS  HW partitioning for performance isolation (so models still work when apps run together)  Problem: evaluating effectiveness of resulting scheduling decisions requires running hundreds of schedules for billions of cycles each  Simulation-bound: 8.3 CPU-years for Simics!  See paper for app modeling strategy details 17

18. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Case Study: Manycore OS Resource Allocation 18

19. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Case Study: Manycore OS Resource Allocation  The technique appears to perform very well for synthetic or reduced-input workloads, but is lackluster in reality! 19

20. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB RAMP Gold Performance  FAME (RAMP Gold) vs. SAME (Simics) Performance  PARSEC parallel benchmarks, large input sets  >250x faster than full system simulator for a 64-core target system 20

21. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Researcher Productivity is Inversely Proportional to Latency  Simulation latency is even more important than throughput  How long before experimenter gets feedback?  How many experimenter-days are wasted if there was an error in the experimental setup? 21 Median Latency (days)Maximum Latency (days) FAME0.040.12 SAME7.5033.20

22. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Fallacy: FAME is too hard  FAME simulators more complex, but not greatly so  Efficient, complete SAME simulators also quite complex  Most experiments only need to change timing model  RAMP Gold’s timing model is only 1000 lines of SystemVerilog  Modeled Globally Synchronized Frames [Lee08] in 3 hours & 100 LOC  Corollary fallacy: architects don’t need to write RTL  We design hardware; we shouldn’t be scared of HDL 22

23. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Fallacy: FAME Costs Too Much  Running SAME on cloud (EC2) much more expensive!  FAME: 5 XUP boards at $750 ea.; $0.10 per kWh  SAME: EC2 Medium-High instances at $0.17 per hour 23 Runtime (hours) Cost for first experiment Cost for next experiment Carbon offset (trees) FAME257$3,750$100.1 SAME73,000$12,500 55.0  Are architects good stewards of the environment?  SAME uses energy of 45 seconds of Gulf oil spill!

24. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Fallacy: statistical sampling will save us  Sampling may not make sense for multiprocessors  Timing is now architecturally visible  May be OK for transactional workloads  Even if sampling is appropriate, runtime dominated by functional warming => still need FAME  FAME simulator ProtoFlex (CMU) originally designed for this purpose  Parallel programs of the future will likely be dynamically adaptive and auto-tuned, which may render sampling useless 24

25. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Challenge: Simulator Debug Loop can be Longer  Takes 2 hours to push RAMP Gold through the CAD tools  Software RTL simulation to debug simulator is also very slow  SAME debug loop only minutes long  But sheer speed of FAME eases some tasks  Try debugging and porting a complex parallel program in SAME 25

26. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Challenge: FPGA CAD Tools  Compared to ASIC tools, FPGA tools are immature  Encountered 84 formally-tracked bugs developing RAMP Gold  Including several in the formal verification tools!!  By far FAME’s biggest barrier  (Help us, industry!)  On the bright side, the more people using FAME, the better 26

27. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB When should Architects still use SAME?  SAME still appropriate in some situations  Pure functional simulation  ISA design  Uniprocessor pipeline design  FAME necessary for manycore research with modern applications 27

28. EECS Electrical Engineering and Computer Sciences B ERKELEY P AR L AB Conclusions  FAME uses FPGAs to build simulators, not computers  FAME works, it’s fast, and we’re using it  SAME doesn’t cut it, so use FAME!  Thanks to the entire RAMP community for contributions to FAME methodology  Thanks to NSF, DARPA, Xilinx, SPARC International, IBM, Microsoft, Intel, and UC Discovery for funding support 28 RAMP Gold source code is available: http://ramp.eecs.berkeley.edu/gold