1. 1 CSTS WG CSTS WG Prototyping for Forward CSTS Performance Boulder November 2011 Martin Karch

2. 2 Prototyping for Fwd CSTS Performance Structure of the Presentation Background and Objective Measurement Set-Up Results Summary

3. 3 Background and Objective Reports from NASA: Prototyping experimental Synchronous Forward Frame Service (CLTU Orange Book) CLTU Service approach limits throughput seriously Target rate (25 mbps) could only be reached if - 7 frames of 220 bytes length - 1 data parameter of one single CLTU No radiation reports for individual CLTU (only the complete one) No investigation yet available Why can throughput not be reached when frames are transferred in a single CLTU? What is the cost of acknowledging every frame?

4. 4 Background and Objective Based on the Reports: Suggest blocking of several frames into one data parameter for Potential future Forward Frame CSTS Process Data Operation/Data Processing Procedure (FW) Objective of Prototyping Verify the blocking of data items significantly increases the throughput Investigate if bottleneck is in service provisioning (actual protocol between user and provider) Results shall support selection of most appropriate approach for the CSTS FW Forward Specification Measurements are made for Protocol Performance

5. 5 Measurement Set-Up 2 machines equipped with Xeon 4C X3460 2.8GHz/1333MHz/8MB 4GB memory Linux SLES 11 64 bit Isolated LAN 1 Gbit cable connection (no switch) SGM (SLE Ground Models) NIS (Network Interface System)

6. 6 Measurement Set-Up Provider SGM SLE Ground Models Simulation Environment SGM changed such that Receiving Thread puts CLTUs on a Queue for Radiation A ‘Radiation Thread’ removes CLTUs and discards them No further simulation of Radiation process (radiation duration) User NIS Network Interface System Simulation Environment NIS is modified to Create (as fast as possible) CLTU operation objects Immediately passes them to SLE API for transmission No interface to a Mission Control System (MCS)

7. 7 Measurement Set-Up Basis for all Steps: SGM based provider NIS based user Step 1 Measurements: Variation of CLTU length - Simulates sending many small CLTUs - In one TRANSFER DATA Invocation (1 st approximation) Step 2 Measurements: SLE API modified - Aggregate configurable number of CLTU (SEQUENCE OF Cltu) - With minimum annotation (CLTU Id, sequence count) - Send return when last data unit is acknowledged

8. 8 Step1 / Measurement 1 SGM + NIS model optimisedyes SLE API optimisedno Nagle + delayed ackon RTT0.1 ms Linear curve Proportional to CLTU size Constant Processing Time Independent of CLTU size

9. 9 Step1 / Measurement 2 SGM + NIS model optimisedyes SLE API optimisedyes Nagle + delayed ackon RTT0.1 ms

10. 10 Step1 / Measurement 3 SGM + NIS model optimisedyes SLE API optimisedyes Nagle + delayed ackoff RTT0.1 ms

11. 11 Step1 / Measurement 4 SGM + NIS model optimisedyes SLE API optimisedyes Nagle + delayed ackoff RTT400 ms Processing Time still constant Transfer-time increased

12. 12 Step1 / Measurement 5.1 Msmnt 5.1: Reference Measurement for Measurements with variations of RTT using IPerf Msmnt 5.2: Measurements using SGM + NIS

13. 13 Step1 / Measurement 5.2 SGM + NIS model optimisedyes SLE API optimisedyes Nagle + delayed ackoff RTTvariable Shows influence of transmission time only Delay is dominating factor As expected (1/RTT) Ratio Msmnt/Iperf = 0.165 (1544) Ratio Msmnt/Iperf = 0.153 (1000)

14. 14 Step1 / Measurement 5 (2) Operates with Maximum Send and Receive Buffer Question: How big must the window size be to achieve similar throughput values like above ( for the example of 40 Mbit/sec) Maximum Data Rate = Buffer size/RTT Window sizeRTT [ms] CLTU Size [byte] CLTU CountData [byte] Data Rate [Megabit/s] Send Duration [s]CLTU Rate [#/s] Send Time per CLTU [ms] 64 KB 50154450007,720,0001.71336.056138.6737.211 “ 100154450007,720,0000.86171.76169.67614.352 “ 200154450007,720,0000.431143.35134.87928.670 “ 400154450007,720,0000.216286.00517.48257.201 “ 50100050005,000,0001.59525.077199.3865.015 “ 100100050005,000,0000.79950.08099.84010.016 “ 200100050005,000,0000.400100.07949.96120.016 “ 400100050005,000,0000.200200.08724.98940.017 13MB10015445000077,200,00011.81252.287956.2611.046 25 MB10015445000077,200,00011.66652.939944.4831.059 25 MB0.115445000077,200,00040.22115.3553256.2680.307

15. 15 Step 1 Measurements Summary Linear increase of data rate with CLTU length sending as fast as possible no network delay Constant Processing Time Best results with Optimised Code - 5 to 10 % performance increase (optimised SLE API only) Nagle and Delayed Ack. switched off - (factor 2.5 lower when Nagle Alg. and Delayed Ack. are both on) No network delay Network delay 200 ms (400 RTT) Performance decrease of a factor of 400 compared to Measurement 2 (the best one) Maximum Data Rate = Buffer size/RTT We have to take care on the size of the CLTU

16. 16 What is the Cost of Confirmed Operations Data unit size = 8000 byte CLTU:207.57 Mbps RAF:318.32 Mbps ( Frame size 8000 byte, 1 frame/buffer) Increase by 53% Data unit size = 2000 byte CLTU:53,36 Mbps RAF: 85.64 Mbps ( Frame size 2000 byte, 1 frame/buffer) Increase by 60%

17. 17 Effects of Buffering (RAF) Frame SizeFrames/BufferMbpsFrame/secmsec/frame 80001322.795,1200.195 40002244.417,9030.126 20004167.0210,8450.092 10008108.1813,3100.075 8001088.6715,1020.066 4002046.2517,1280.058 2004027.0518,9690.052 1008013.7819,1220.052 Concatenation of 80 frames of 100 byte into a buffer back-to- back and then passed to the API as one frame: 322.43 Mbps Frame size = 2000 byte, 1 Frame/buffer: 85.64 Mbps

18. 18 Same in Graphical Presentation

19. 19 RAF Measurment Configuration Frame Generator SLE Service Provider Application SLE API Communication Server SLE API SLE Service User Application frame transfer buffer TCP (local) TCP frame

20. 20 Cost of ASN.1 Encoding (RAF) Result of profiling for RAF, frame size 100 byte, 80 frames per buffer: Encoding of Transfer Buffer including all contained frames: 6.42% Encoding of Transfer Buffer Invocation alone: 2.31% Effects might be caused by increased interactions / interrupts, etc.

21. 21 Summary of Observations Size of the data unit transferred has a significant impact Almost constant end to end processing time independent of buffer size Liner increase of net bitrate with data unit size Large impact on network delay due to TCP (expected) Significant additional cost of using confirmed operations Buffering of frames vs, transfer in individual frames 4 frames of 2K per buffer vs single 2K frames: factor 1.9 BUT: throughput for a single large data unit is much larger than buffer of same size containing multiple small units ASN.1 encoding for worst case test accounts for 6.4% of overall local processing time