1. RAMP-White Hari Angepat Derek Chiou University of Texas at Austin

2. Motivation  Coherent shared memory multiprocessor simulator  Support for existing programming models Operating system support Single image Programming libraries Legacy applications 2RAMP-White

3. Outline  Leon3 Integration Default architecture RAMP-White architecture Baseline Design features Status  PPC405 MP OS Support Plan9 Operating System Porting Status RAMP-White3

4. Support for Leon3  Previous design utilized PPC405 hard-cores Dual-node, multi-image Linux kernels Segmented global address space  Added support to use Leon3/Grlib components RAMP-White4

5. Leon3 Default Architecture RAMP-White5 Leon 3 MstSlvDbgInt AHB Bus MP IntCntrl DSUEthDDR Leon 3 MstSlvDbgInt  Original Leon3MP design is shared bus model Processor cores share AHB bus Periperals connected to AHB Interrupt and DSU have direct processor links

6. RAMP-White System Architecture RAMP-White6 Leon 3 MstSlvDbgInt Leon3 shim MP IntCntrl DSUEthDDR2 Leon 3 MstSlvDbgInt AHB bus Leon3 shim Intersection Unit NIU Intersection Unit NIU Router  Adapt bus interfaces to a point-to-point connection scheme DDR2 AHB bus AHB shim

7. Processor Support  IBM PowerPC 405 Hard-core, 300Mhz Non-coherent I/D caches PLB Bus interface Uni-Processor Linux  Gaisler Leon3 Soft-core, ~65Mhz Write-through snoopy-cache-coherent AMBA AHB bus interface SMP Linux 7RAMP-White

8. Default Leon3 Core Interfaces  4 bi-directional channels Master bus interface Services icache/dcache fills Slave bus interface Invalidates dcache entries via snooping address bus Interrupt channel Driven from multiprocessor interrupt controller Debug channel Grmon DSU interface 8RAMP-White

9. Integrating Off-the-Shelf Processor  Generally there is a tight coupling between pipeline/cache/bus-interface Thus, prefer to keep existing port interface Also provides forward compatibility with soft-cores  Therefore, add bus shim to convert from processor-specific bus to White connections 9RAMP-White Leon 3 MstSlvDbgInt AHB shimInt/Dbg shim

10. Processor Adaptation Issues  Increased FPGA resources Support for RAMP-White infrastructure  Performance impact Request/reply interaction adds latency  Platform Detection Static mapping of bus connected devices Linux/Grmon use configuration registers to detect platform configuration at run-time If devices on are connected indirectly, must populate mapping correctly 10RAMP-White

11. Baseline Design  Dual Leon3 cores with bus shims  Intersection Unit acts as message handler Will be used to support pluggable coherency engines  Network interface  Ring topology 11RAMP-White

12. System Software  Bootstrap: JTAG for initial configuration Ethernet for system init, kernel loading, debug  Linux 2.6.21 SMP Kernel  SnapGear Linux root file-system  Pthreads programming libraries 12RAMP-White

13. Current Prototype  Single FPGA, dual core, RAMP-White infrastructure  ICache enabled, DCache disabled  Boots Linux in SMP mode via debug memory initialization  Ethernet and NFS mount support  Compact initramfs root file system complied from SnapGear sources 13RAMP-White

14. Current Prototype 14RAMP-White

15. Future Work  Near-term Cleanup/bug-fixing/stabilize dual-node platform Integrate simple microcoded coherency engine for Spring 2008 Parallel Comp Arch class Lab to be given out by last week of Feb Expand cache hierarchy with soft-core cache models Expand design to support multi-FPGA support 15RAMP-White

16. Multiprocessor PPC405 OS Support  Previous PPC RAMP-White design used multiple independent operating system images Pseudo-SMP support in Linux kernel was non- trivial to implement  Alternative strategy opened up by recent work over the summer by IBM in porting Plan9 to BlueGene PPC 16RAMP-White

17. Plan9 Background  Research OS from Bell Labs open-sourced in 2000  Unix-style operating system  Resources exposed as file trees  Per-process namespaces  Standard protocol for sharing resources 17RAMP-White

18. Plan9 for HPC Applications  IBM port of Plan9 on a BlueGene grid Part of DoE FastOS initiative Allows distributed resource management and sharing across a large grid Lighter weight kernel that has less intrusive effect on HPC apps Ported to PPC440 with support for JTAG-based debug and bootstrapping 18RAMP-White

19. Plan9 for RAMP-White  Smaller, lower complexity operating system  Allows flexible sharing of physical resources memory, ethernet, disk  Can expose multiple cores as CPU servers Allows easy task execution/debugging on remote cores  Leverage resurgent interest in using Plan9 for HPC HPC applications ported to Plan9 19RAMP-White

20. Porting Plan9 Worked in collaboration with Eric Van Hensbergen Resurrected the CerfCube 405 platform – Removed assumptions regarding PPC405 SoC – Adding support for Xilinx peripherals Serial port, interrupt controller, network Currently boots Plan9 with limited console – Bootstrap, virtual memory, console initialization – Still completing work on interrupt controller, network 20RAMP-White

21. Questions? 21RAMP-White