1. Software Overhead in Messaging Layers Pitch Patarasuk

2. Messaging Layer Bridge between the hardware functionality and the user communication requirement Network hardware feature –Arbitrary delivery order –Finite buffering –Limited fault handling User communication requirement –In-order delivery –End-to-end flow control –Reliable transmission

3. Messaging Layer (cont.)

4. Software overhead Main communication cost = hardware cost + software cost Software cost dominates hardware cost 50-70% of the software cost are the direct gap between the network features and user requirements

5. Software overhead in messaging layers Analyze the costs of communication functionality and network features What overhead might be reduced if the underlining network provide higher level of service

6. CM-5 active message layer (CMAM) CM-5 –Parallel machine up to a few thousand nodes connected with an incomplete fat tree topology Active message –Communication mechanism intended to expose full hardware flexibility and performance of modern networks –Each message contain an address of a user-level handler which is executed on message arrival with the message body as an argument –The handler extract data from the network and integrate it into the ongoing computation CMAM vs Send/Recv –User direct access to network, no OS involve, the data go directly to the user space computation

7. Software overhead cost analysis Consider implementation of 3 protocols –Single-packet delivery –Finite sequence, multi-packet delivery –Indefinite sequence, multi-packet delivery Use instruction counts for measurement

8. Single-packet delivery DescriptionSourceDestination Call/Return310 NI Setup-- Write to NI2- Read from NI-3 Check NI status712 Control Flow32 2027

9. Finite sequence, multi-packet delivery

10. Indefinite sequence, multi-packet delivery

11. Message size = 16 words

12. Message size = 1024 words

13. Messaging layer with high-level network feature Given that CMAM is considered to be very efficient, there are 2 choices to reduce software overhead –Lower user requirement –Raise level of service provided by the network Compressionless Routing (CR) –Order-preserving transmission –Deadlock freedom independent of packet acceptance guarantees –Fault-tolerant transmission at packet level

14. Single-packet delivery Has the same cost as the previous CMAM case However, it is guaranteed to be fault-free, no deadlock or buffer overflow

15. Finite sequence, multi-packet delivery

16. Indefinite sequence, multi-packet delivery

17. Message size = 16 words

18. Message size = 1024 words

19. Discussion Larger packet sizes –Reduce overhead, especially for indefinite-sequence protocol Improved network interfaces and DMA hardware –Network interface: only make basic cost faster, but not protocol cost in messaging layer Make it more important for messaging layer to be effective –DMA: only reduce cost in moving large amounts of data

20. Discussion (Cont.) Implication of network design –Improving routing performance may increase software cost, e.g. out-of-order delivery Reducing communication features –Fault-tolerance, in-order-delivery –Put burden on parallel software programmer Use instruction counts for measurement instead of latency –Latency is hard to measure in a portable fashion

21. Conclusion Software overhead is much larger than routing time and 50-70% are messaging layer overhead Software overhead in messaging layer is the cost of the gap between the network feature and the user requirement This cost can be reduced to zero if the underlining network provides functionality that the user requires.

22. Questions?