1. Performance Evaluation of L3 Transport Protocols for IEEE 802.21 (2 nd round) Richard Rouil, Nada Golmie, and David Griffith National Institute of Standards and Technology http://www.antd.nist.gov/seamlessandsecure.shtml

2. Outline MIH transport issues investigated Simulation scenarios Performance evaluation results –UDP without MIH ACK (for reference) –UDP with MIH ACK –TCP –SCTP Conclusions

3. MIH Transport Issues Investigated Transport protocol type –UDP –TCP PoS Location –RTT between the MIH nodes Message parameters –Size –Rate Network conditions –Link congestion Priority queuing Retransmission timeout Fragmentation Congestion & rate control

4. Mobility scenarios Varying the RTT between the MN and the MoS addresses all 4 scenarios identified in draft-xxx- mipshop-mstp-solution-00: –S1: Home Network MoS, the MN and the services are located in the home network. –S2: Visited Network MoS, MN is in the visited network and mobility services are also provided by the visited network. –S3: Roaming MoS, MN is in the visited network and all services are provided by the home network. –S4: Third party MoS, MN is in home or visited network and services are provided by a 3rd party network.

5. Network Topology MN AP (co-located PoS) PoS (not co-located) CN Variable link delay, packet loss, and bandwidth Background traffic to CN for congestion MIH message exchange to PoS

6. Simulation parameters IEEE 802.11 Data rate (Mb/s)11Mb/s Coverage area – radius (m)50 Links Speed (Mb/s)100 Delay (s)0.01 UDP Max packet size (byte)1000 Header size (bytes)8 TCP Maximum segment size (bytes)1280 Min RTO (s)0.2 Max retransmissionUnlimited Queue sizeUnlimited Header size (bytes)20 IP header IPv6 header (bytes)40 MIH Function Transaction timeout (s)none Maximum number of retransmission 2 Background traffic Traffic typecbr over udp Packet size (bytes)500 Inter-arrival rate (s)0.005 Simulation variables Transport protocolUDP no ACK, UDP ACK, TCP PoS LocationCo-located, Remote Variable link Delay (s)0.01-0.5 Variable network packet loss (%)0-50 MIH messageIndication, Request/Response MIH packet size (byte)50-1000 MIH packet inter-arrival (s)Exp distribution w/ varying mean in [ 0.1 2] MIH request processing time (s)0-0.2 MIH message timeout (xRTT) (s)0.5-2 Number of background traffic nodes0-10 Rate limiting bucket size (packet)1-20 Rate limiting token rate (packet/s)1-20 Rate limiting queue size (packet)10-50

7. Transport protocol performance evaluation Performance metrics: –Transaction success rate (i.e. indication or response received) –Delay to complete a transaction –Network load –Overhead created by the MIH acknowledgement mechanisms and the transport layer –Transport throughput

8. Impact of RTT Impact of the RTT on the transport of MIH messages: –Vary the RTT between 0.1s and 0.5s –For different packet loss (0%, 10%, 20%) The following transport protocol are used: –TCP –UDP without MIH ACK –UDP with MIH ACK

9. UDP (no ACK) Performance

10. Transaction delay for indications The transaction delay equals half the RTT regardless of the packet loss.

11. Transaction delay for requests The transaction delay equals the RTT regardless of the packet loss.

12. Transaction success rate for indications Success rate is equal to 1-p with p the packet loss in the network.

13. Transaction success rate for requests Success rate is equal to (1-p) 2 with p the packet loss in the network.

14. MIH overhead

15. UDP (with ACK) Performance

16. Transaction delay for indications When there is no loss, the delays are identical to the UDP without retransmission. Packet loss causes retransmission thus increasing the delays. Delay=

17. Transaction delay for requests The results can be sectioned in three parts: RTT < ReTx: In this case, retransmissions are only due to packet loss. As the packet loss increases, the delay increases. ReTx < RTT < 2*ReTx: We observe that the transaction delays are converging. This reason is that the second retransmission is ineffective. RTT > 2*ReTx: Regardless of packet loss, the MIH will timeout twice sending and both retransmissions are ineffective as the ACK would arrive after the timeout of the second retransmission (Transaction state is invalid, and response is ignored). RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx

18. Closer look at retransmission SenderReceiver SenderReceiverSenderReceiver Timeout 1 Timeout 2 Timeout 3 Transaction fails Timeout 1 Timeout 2 Timeout 3 Transaction fails Timeout 1 Timeout 2 Timeout 3 Transaction fails RTT < timeouttimeout < RTT < 2* timeoutRTT > 2*timeout All retransmissions are effectiveSecond retransmission is ineffective. Two packets lost on the same transaction causes it to fail. All retransmissions are ineffective. Any loss will cause the transaction to fail.

19. Transaction success rate for indications Success rate equals 1-p 3 with p the packet loss in the network.

20. Transaction success rate for requests The results can be sectioned in three parts: RTT < ReTx: success rate equals (1-p 3 ) 2. ReTx < RTT < 2*ReTx: success rate equals (1-p 2 ) 2. The second retransmission is ineffective. RTT > 2*ReTx: success rate equals (1-p) 2.. Both retransmissions are ineffective. RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx

21. MIH overhead for indications The results can be sectioned in three parts: RTT < ReTx: The retransmissions are due to packet loss only. Overhead = ReTx < RTT < 2*ReTx: There is always one retransmission due to late ACK and some additional one due to packet loss. Overhead = 2(2-p) + p(2-p) 2. RTT > 2*ReTx: success rate equals (1-p) 2. There are 2 retransmissions due to late ACK. As the packet loss increases, late responses are ignored and therefore no ACK is being generated. Overhead = 3 (2-p). RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx

22. MIH overhead for requests The overhead for request/response is following a similar pattern as the overhead for indications. Note: the number of MIH Packets sent is divided by both the number of MIH requests and MIH responses sent. RTT < ReTx ReTx < RTT < 2*ReTx RTT > 2*ReTx

23. UPD with MIH ACK summary The location of the PoS (i.e. RTT) impacts the performance of the MIH ACK mechanisms. If the retransmission timeout is not adequately set, the retransmission may become ineffective: –ReTx < RTT < 2* ReTx: the results are identical to an MIH using one retransmission. –RTT > 2*ReTx: the results are similar to UDP without MIH ACK.

24. TCP Performance

25. Transaction delays for indications and requests/responses TCP retransmits the packets using an exponential backoff leading to high retransmission delays.

26. Transaction success rate for indications and requests/responses TCP’s reliability allows for a success rate of 100%.

27. MIH overhead MIH does not retransmit messages when using a reliable transport, therefore there is not MIH overhead.

28. SCTP Performance

29. Network Topology MN AP IEEE 802.11 (co-located PoS) PoS (not co-located) CN Variable link delay, packet loss, and bandwidth Background traffic to CN for congestion MIH message exchange to PoS BS IEEE 802.16 Interface 2 Interface 1 Added Access Network for multi-homing support Second interface added for multi-homing support

30. Simulation parameters IEEE 802.16 Coverage area – radius (m)500 Frame duration (s)0.005 Modulation64_QAM_3_4 SchedulingBest Effort Links Speed (Mb/s)100 Delay (s)0.01 except link to PoS interface 2 with delays of 0.05 SCTP Max packet size (byte)1500 Header size (bytes)12 Initial RTO (s)3 Min RTO (s)1 Max RTO (s)60 Number of stream1 Delayed ACKYes SACK delay (s)0.2 Simulation variables Transport protocolSCTP with and without retransmission using alternate path

31. Transaction delay for indications When multi-homing is not available, a packet loss increases the backoff time for retransmission and generates high delays. The behavior is similar to TCP. Using multi-homing, retransmissions are done on an alternate path which may provide lower loss. This is can be favorable if the primary path is using an interface ready for handover.

32. Transaction delay for requests

33. Transaction success rate for indications SCTP provides reliable transport of MIH messages by retransmitting on the same interface or using an alternate when available.

34. Transaction success rate for requests

35. MIH overhead for indications MIH does not retransmit messages when using a reliable transport, therefore there is not MIH overhead.

36. MIH overhead for requests

37. Congestion control One of the backbone network link between the MN and PoS provides limited capacity (1Mb/s). MN sends messages at a rate between 50kb/s and 1500kb/s for 30s (defined as offered load). Packet sizes are 100 bytes and 500 bytes. The following congestion control are used: –TCP –UDP with no rate limiter –UDP with Token bucket (200 bucket size, 200 token/s, 200 queue size) –UDP with Token bucket (200 bucket size, 50 token/s, 200 queue size)

38. Transaction delay for Indications While there is no congestion (Offered load less than 1Mb/s), TCP uses all the available bandwidth. During congestion, TCP queues the packets and the delays to receive the indication increase. When the offered load increases, the size of the TCP segments increases to carry multiple MIH messages thus reducing the overhead. When using UDP without rate limiter, the packets delays are low when there is no congestion. After an offered load of 600kb/s, delays appear due to the overhead generated by UDP (see throughput graph). The rate limiter generates additional delays even for low offered loads. We observe that it acts later for larger MIH messages as it generates less packets for the same offered load. Furthermore, we can note that the indication is considered successful for indications even with high delays since the receiver is not aware of when it was sent. The sender on the other side may not receive the ACK on time, may retransmit the packet, and may consider the operation has failed.

39. Transaction delay for Request/Response Similar observation for TCP and UDP without rate limiter. When using rate limiter, the delays incurred due to the queuing causes the responses to be ignored. Therefore only the initial messages succeeded with low transaction delay.

40. Transaction success rate for Indications TCP’s reliability allows for a success rate of 100%. When UDP is used without rate control, an offered load of 600kb/s or more shows decreasing success rate. This is due to the packets being drop at the congested link in the network. When control rate is used, the smaller the token rate, the lower the success rate. This is due to the queue limit of the token bucket implementation.

41. Transaction success rate for Request/Response Observation similar to indications but lesser success rate for UDP since delayed responses are ignored.

42. MIH overhead for Indications There is no MIH overhead created when using TCP since there is no retransmission at the MIH level. When using UDP, there are 2 MIH packets corresponding to the Indication and the ACK for low offered load. When no rate control is in place and the load is more than 600kb/s, increased in packet delay causes retransmissions, thus increasing the overhead. The overhead is decreasing when rate control is in place due to packets being drop at the MIH level. The smaller the token rate, the faster the queue fills up and MIH message are dropped without trying to be sent.

43. MIH overhead for Request/Response We observe an increase of overhead before it decreases again. This is happens when the sending rate of MIH packets caused by retransmissions is close to the optimum sending rate allowed by the token configuration. After this rate is passed, the packets are queued and dropped without trying to be sent.

44. Network layer load for Indications The network load measures the traffic generated by the MN’s transport layer to send MIH packets. TCP’s congestion control can be observed as the sending rate saturates close to 1Mb/s. When UDP is not using rate control, all the retransmissions increase the network load. This leads to additional congestion in the network. In our scenario, it could saturate the wireless network if other nodes are present. The rate control allows to limit the load though adequate values of token rate must be configured. We observe for example that in the case of packet size of 500 Bytes and token rate of 200, the rate control performs almost identically to TCP.

45. Network layer throughput for Indications The network throughput is the data rate measured by the receiver side, in this case the PoS. For TCP and UDP using rate control, the throughput measured is identical to the load since the load is lesser than the available bandwidth on the congested link. When no rate control is in place, the throughput saturates.

46. Summary of results on Congestion Control Congestion control is needed congestion spreads in the network due to a significant number of retransmissions. TCP’s embedded congestion control and packing provides less overhead for low congestion. When the link saturates, the exponential backoff mechanism and Head of Line situation leads to very high delays. The token bucket parameters must be adjusted to consider the congestion level and the average MIH packet size.