1. COM S 614 Advanced Systems Novel Communications U-Net and Active Messages

2. Problem Domain System Area Networks Cluster Computing Rack-and-stack builds COTS hardware

3. Presentation Parts U-Net — say it fast Active Messages — keep it simple

4. Cluster Computing File server clusters Database servers Storage Area Networks Web Servers Scientific Applications

5. Parallel Applications Communications Intensive Small messages 50% < 200 bytes RPC style programming

6. Traditional Network Interface NIC on IO bus DMA to kernel buffers Interrupt driven

7. Performance Barriers Multiple data copies Kernel processing bottleneck Context switch expensive

8. GoalsGoals Remove kernel Zero-copy send/receive Simple API

9. User Level Network Interface Application thinks it owns the adapter Application owns its buffer pool Application handles retries, scheduling

10. ULNI Pieces Virtual interface Large buffer pool for data Ring buffers for send/receive descriptors Memory mapped status registers

11. ULNI API Kernel used to establish VI Common functions for descriptor and pool management DMA direct to user space

12. Virtual Interface Architecture Compaq, Intel, Microsoft Standard for communications within a cluster MS provides similar WinSock Direct WinSock Disaster 10X latency over NIC 60-70% max thruput

13. ULNI Lessons Network transfers through DMA Programmed IO can be better Polling is simple, but not efficient Alternative is interrupts, worse! Unreliable, no flow feedback Point-to-point Context switch VI slow

14. ULNI Future Interface mapped to coherent memory bus Note: might waste a page of SRAM for status registers, almost all of it is wasted Polling code inserted by compiler Multicast traffic managed by switches, spanning tree Multiple NI’s an interesting research direction

15. How to say it… Don’t wait for the response Do simple things Messages look like long jumps

16. Procedure call?! Identical code on all nodes Really small messages (16 bytes) Not allowed to block “Deadlock avoidance is a rather tricky issue…”

17. Active Message Applications Parallel computation Data flow Flow control, lock management Virtual Mapped Memory Communication

18. AM Lessons Simplify buffer management High caliber initial implementation Willingness to distribute and support the code Provided a common vocabulary for developers Efficient RPC methods

19. Fini!