1. RAMP/HAsim Status Update Joel Emer Michael Adler Angshuman Parashar Michael Pellauer Murali Vijayaraghavan joel.emer@intel.com michael.adler@intel.com angshuman.parashar@intel.com pellauer@csail.mit.edu vmurali@csail.mit.edu VSSAD Group Intel CSG Group, CSAIL MIT

2. 2 Infrastructure Update AWB Plug N Play –Hardware Language Support Bluespec, VHDL, Verilog –Software Language Support C++ Build Procedure –Updated to scons Automatic Parallel Build –Common data types between hardware and software Common Platform

3. 3 History and Motivation XUP Model“Sim” Model Top Level WiresBDPI XUP Board LEDs and Switches (image source: xilinx.com) Software Simulator Processes (image source: intel.com)

4. 4 History and Motivation Platform Independent Model XUP Front Panel Top Level Wires “Sim” Front Panel BDPI front_panel.writeLED() Virtual Platform XUP Board LEDs and Switches (image source: xilinx.com) Software Simulator Processes (image source: intel.com) or

5. 5 Virtual Platform Provide common set of functionalities across multiple physical platforms XUP Board PCI-express Board Intel FSB Socket Bluesim/Vsim BEE3 Leverage Asim Plug N Play –Minimize module replacements/recoding while moving across platforms

6. 6 Virtual Platform Communication Layer(s) Device 1 Virtual Devices Timing + Functional Modules Virtual Platform Platform Interface Physical Platform Device 2Device 0 Soft Connections ExeDecodeFetch Func Model Control

7. 7 Virtual Platform on XUP VGA Virtual Devices Virtual Platform Platform Interface LEDs Switches MemoryFront Panel VGA On-Board Memory Timing + Functional Modules Soft Connections ExeDecodeFetch Func Model Control

8. 8 Virtual Platform on Simulator UNIX Pipe Communication Layer VGA Virtual Devices Virtual Platform Platform Interface Bluesim MemoryFront Panel Pipe I/O Timing + Functional Modules Soft Connections ExeDecodeFetch Func Model Control

9. 9 Virtual Platform on PCIe System Communication Layer(s) VGA Virtual Devices Virtual Platform Platform Interface PCIe-based Host System MemoryFront Panel Timing + Functional Modules Soft Connections ExeDecodeFetch Func Model Control Host Software

10. 10 Communication with Host Software: Hybrid Modules Communication Layers ExecuteDecodeFetch Func ModelController Decode PCI Express Communication Layers ConsoleFront Panel Console FPGA Host Hybrid Module

11. 11 Server - Accept Request - Send Response Hybrid Module Communication: The Paradigm FPGA ModulesSoftware Modules Communication Layers ExecuteDecodeFetch Func ModelController Start Stop Step Start Stop Step PrintAssert PrintEvent PrintStat PrintAssert PrintEvent PrintStat Decode Client - Make Request - Accept Response ClientServer Controller PCI Express Communication Layers HAsim Remote Request/Response (RRR)

12. 12 Server - Accept Request - Send Response RRR Protocol Stack FPGA ModulesSoftware Modules Kernel Driver ExecuteDecodeFetch Func ModelController Low Level Driver Controller Start Stop Step Start Stop Step PrintAssert PrintEvent PrintStat PrintAssert PrintEvent PrintStat Decode Client - Make Request - Accept Response ClientServer RRR Protocol Stack Key Issues Service IDs –Automatic Assignment Request and Response Types –Flexibility –Ease of Use PCI Express

13. 13 RRR Specification Language // -------- // typedefs // -------- typedef MSG_CLASS { HOST: UINT8, FPGA: Bit#(8) }; typedef STATUS { HOST: bool, FPGA: Bool }; typedef UINT { HOST: UINT64, FPGA: Bit#(64) }; // -------------------------------------- // create a new service called CONTROLLER // -------------------------------------- service CONTROLLER { // ------------------------------------------- // declare services provided by HOST partition // ------------------------------------------- server HOST <- FPGA; { method PrintMessage(in MSG_CLASS, out STATUS); method PrintStat(in MSG_CLASS); method PrintEvent(in MSG_CLASS); }; // ------------------------------------------- // declare services provided by FPGA partition // ------------------------------------------- server FPGA <- Host; { method Start(); method Stop(); method Step(in UINT); };

14. 14 RRR Protocol Stack FPGA ModulesSoftware Modules Kernel Driver ExecuteDecodeFetch Func ModelController Low Level Driver Controller Decode Hardware Channels Channel IO CSRDMAInterrupt Channel IO CSR Interrupt

15. 15 RRR Protocol Stack FPGA ModulesSoftware Modules Kernel Driver ExecuteDecodeFetch Func ModelController Low Level Driver Controller Decode Hardware Channels Channel IO RRR ServerRRR Client CSRDMAInterrupt Channel IO RRR ServerRRR Client Channel 0Channel 1 Channel 0 CSR

16. 16 RRR Protocol Stack FPGA ModulesSoftware Modules Kernel Driver ExecuteDecodeFetch Func ModelController Low Level Driver Controller Decode Hardware Channels Channel IO RRR ServerRRR Client CSRDMAInterrupt Channel IO RRR ServerRRR Client Channel 0Channel 1 Channel 0 CSR SID 0 SID 1 RRR specification files 00 11 rrr_service_ids.bsv `define CONTROLLER_SERVICE_ID 0 `define DECODE_SERVICE_ID 1 `define MEMORY_SERVICE_ID 2 `define NUM_SERVICES 3

17. 17 RRR Protocol Stack FPGA ModulesSoftware Modules Kernel Driver ExecuteDecodeFetch Func ModelController Low Level Driver Controller Decode Hardware Channels Channel IO RRR ServerRRR Client CSRDMAInterrupt Channel IO RRR ServerRRR Client Channel 0Channel 1 Channel 0 CSR SID 0 SID 1 RRR specification files 00 11 Stub

18. 18 RRR Protocol Stack FPGA ModulesSoftware Modules Kernel Driver ExecuteDecodeFetch Func ModelController Low Level Driver Controller Decode Hardware Channels Channel IO RRR ServerRRR Client CSRDMAInterrupt Channel IO RRR ServerRRR Client Channel 0Channel 1 Channel 0 CSR SID 0 SID 1 00 11 Stub SID 1 Stub SID 0

19. 19 Switching Physical Platforms FPGA ModulesSoftware Modules Kernel Driver ExecuteDecodeFetch Func ModelController Low Level Driver Controller Decode Hardware Channels Channel IO RRR ServerRRR Client CSRDMAInterrupt Channel IO RRR ServerRRR Client Channel 0Channel 1 Channel 0 CSR SID 0 SID 1 00 11 Stub SID 1 Stub SID 0

20. 20 PCIe-based Physical Platform FPGA ModulesSoftware Modules Kernel Driver ExecuteDecodeFetch Func ModelController Low Level Driver Controller Decode Hardware Channels Channel IO RRR ServerRRR Client CSRDMAInterrupt Channel IO RRR ServerRRR Client Channel 0Channel 1 Channel 0 CSR SID 0 SID 1 00 11 Stub SID 1 Stub SID 0

21. 21 Simulation Physical Platform FPGA ModulesSoftware Modules UNIX inter-process communication ExecuteDecodeFetch Func ModelController BDPI Layer Controller Decode Channel IO RRR ServerRRR Client Channel IO RRR ServerRRR Client Channel 0Channel 1 Channel 0 SID 0 SID 1 00 11 Stub SID 1 Stub SID 0

22. 22 Dictionaries // append to STATDEFS.bsh, STATDEFS.h def STAT.BRANCHES_MISPREDICTED "mispredicted branches = "; // append to EVENTDEFS.bsh, EVENTDEFS.h def EVENT.SIM_FAILED "simulation failed"; // append to TRACEDEFS.bsh, TRACEDEFS.h def TRACE.FETCH "fetch instruction %d"; // note: 1 param def TRACE.DECODE "decode inst %d into %d"; // note: 2 params

23. 23 Infrastructure Summary Channel IO RRR Client + Server FPGA Modules Virtual Platform Platform Interface Channel IO RRR Client + Server Hardware Software Software Modules MemoryFront Panel ExeDecodeFetch Func Model ControlDecode Front Panel Memory

24. 24 Conclusion Virtual Platform –Multi-layered set of abstractions providing platform- independent interfaces to platform-dependent devices Remote Request/Response –Part of the Virtual Platform hierarchy –Facilitates communication between multiple partitions of hybrid modules –Tedious HDL plumbing code is auto-generated from specification files

25. Backup Slides

26. 26 Sample RRR Service Stub

27. 27 Outline History and Motivation Virtual Platforms Hybrid Modules and RRR Conclusion and Demo

28. 28 Hybrid Modules Channel IO RRR Client + Server FPGA Modules Virtual Platform Platform Interface Channel IO RRR Client + Server Hardware Software Software Modules MemoryFront Panel ExeDecodeFetch Func Model ControlDecode Front Panel Memory HAsim Remote Request/Response (RRR)

29. 29 RRR: Intel FSB Channel IO RRR Client + Server FPGA Modules Virtual Platform Platform Interface Channel IO RRR Client + Server Hardware Software Software Modules MemoryFront Panel FSBDriver ExeDecodeFetch Func Model ControlDecode Front Panel Memory

30. 30 RRR: Simulator Channel IO RRR Client + Server FPGA Modules Virtual Platform Platform Interface Channel IO RRR Client + Server Hardware Software Software Modules MemoryFront Panel UNIX pipe ExeDecodeFetch Func Model ControlDecode Front Panel Memory

31. 31 Hybrid Modules –Primary points of communication between FPGA and Host Software –Module functionality can be split between FPGA and software How do partitions of a hybrid module communicate with each other?

32. 32 Hybrid Modules (cont.) Module definition –hybrid_fetch_unit.awb %sources –t BSV –v PUBLIC hardware_fetch_unit.bsv %sources –t CPP –v PUBLIC software_fetch_unit.h %sources –t CPP –v PRIVATE software_fetch_unit.cpp Build process –Collects BSVs and generates “hardware” bitfile or simulation binary –Collects CPPs and generates “software” binary Runtime –Software binary loads bitfile onto FPGA, or forks off simulation “hardware” binary –Software Controller sends “start” RRR request to Hardware Controller