1. Performance Analysis for VoIP System Members R94922009 周宜穎 R94922009 周宜穎 R94922020 吳鴻鑫 R94922020 吳鴻鑫 R94922064 張嘉輔 R94922064 張嘉輔

2. Outline What is Performance Performance Bound How to analyze Performance Some Performance Analysis Exmaple

3. What is Performance ? [10] There are numerous factors that affect the performance assessments. – Human factors – Device factors – Network factors

4. Human factors -Audiovisual Quality Assessment Metrics Subjective quality assessment - MOS Objective quality assessment –Signal-to-Noise Ratio (SNR) –Mean Square Error (MSE) –Perceptual Analysis Measurement System (PAMS) –Perceptual Evaluation of Speech Quality (PESQ) –E -model E-model R-scale ( 0 to 100 ) MOS rankings and User Satisfaction

5. Human factors Voice Quality Classes

6. Device factors Essential devices such as – –VoIP endpoints – –Gateways – –MCUs (Multipoint Control Units ) – –Routers – –Firewalls – –NATs (Network Address Translators ) – –Modems – –Operating System – –Processor – –memory

7. Network factors Network congestion Link failures Routing instabilities Competing traffic General Measruement : –Delay –Jitter –Packet loss

8. The Performance Standard Delay –Good (0ms-150ms) –Acceptable (150ms-300ms) –Poor (> 300ms) Jitter –Good (0ms-20ms) –Acceptable (20ms-50ms) – Poor (> 50ms). Loss –Good (0%-0.5%) – Acceptable (0.5%-1.5%) – Poor (> 1:5%)

9. E-model ITU-T recommendation Well established computational model Using Transmission parameters to predict the quality We can get the basic Performance Standard by through the model

10. Basic formula for the E-model R-value = Ro - Is - Id - Ie + A –Ro the basic signal-to-noise ratio based on sender and receiver loudness ratings and the circuit and room noise –Is the sum of real-time or simultaneous speech transmission impairments,e.g. loudness levels, sidetone and PCM quantizing distortion –Id the sum of delay impairments relative to the speech signal, e.g., talker echo, listener echo and absolute delay –Ie the equipment impairment factor for special equipment, e.g., low bit-rate coding (determined subjectively for each codec and for each % packet loss and documented in ITU-T Recommendation G.113) –A the advantage factor adds to the total and improves the R-value for new services.

11. Estimating the R R = (Ro − Is) − Id − Ie + A Ro, Is – –do not depend on network environment Id – –This the Argument of Delay Ie – –It mostly affect by codec and packet loss A –Additional adjust argument,not considered in general

12. Estimating Estimating Id and Ie Id = Idte + Idle + Idd – –Idte -Talker echo delay – –Idle - Listener echo delay – –Idd - Long delay Ie – –It base on codec, but packet loss affect can be emulated as a function

13. Estimating Estimating Id and Ie The distortion as a function of packet loss also depends on whether or not PLC (Packet Loss Concealment) increases 4 units for codecs with PLC (in the R scale per 1% packet loss) 10 units for codecs without PLC

14. Curve Diagram

15. Test Setup Using 9 scenarios to test 27 possibilities Using NISTnet network emulator (http://snad.ncsl.nist.g ov/itg/nistnet/) create the various network health scenarios

16. MOS Vs Delay

17. MOS Vs Jitter

18. MOS Vs Loss

19. Normalized Each unit in the normalized scale corresponds to delay : 150ms jitter : 20ms loss : 0.5%.

20. The Conclusion about Performance bounds We show that end-user perception of audiovisual quality is more sensitive to the variations in end-to-end jitter than to variations in delay or loss We get a simple standard about the Performance to estimate Performance

21. How to Analyze Performance Thinking about two topic –Measurement –Network Condition Measurement mean the analysis model that estimate Measurement mean the analysis model that estimate key parameters Of course, it is the way to compute delay, jitter,packet loss

22. Two Measurement [7],[8],[9] There are two methods in performance measurement passive measurement – –records and analyzes existing traffic. active measurement – –Inject sample packets into the network.

23. Introduce a simple Measure Measurement Method in LAN sends sequences of UDP packets to unlikely values of destination port numbers (larger than 30,000) This causes the destination host’s UDP module to generate an ICMP port unreachable error when the datagram arrives

24. ICMP TCP/UDP/IP 協定若有錯誤情形發生時，會 利用 Internet Control Message Protocol （ ICMP ）協定來送錯誤訊息 。 在 ICMP 的 type 中，目前約有 15 種 The ICMP echo mechanism should be installed in host in the measurement

25. RTT of one sent packet

26. How to Compute? Ti = Bi / v + Di /v + CL + C Ti − CL =(Bi + Di) / v + C.

27. Keep estimating one-way delay (T i ) T i = (Ri − Si) −Di / v −CL/2 − C/2 This calculation assumes that all delay happens on the sending path. J i,i+1 = (T i+1 − T i ) Packet loss = packetslost / packetssent

28. How about more complicated? Precision timestamping Queuing Model Special Model for Protocol or device Seem to Traffic Analysis!?

29. Ex: SIP Traffic Model [11] A for SIP Traffic A model for SIP Traffic Two Sub Model –IP Path Model –SIP Finite State Machine

30. FSH Notation Q = State set M = fixed number of sessions C = the bottleneck transmission rate( bit/s) R = total capacity of IP Path measure in packets of D bits rtt = round trip time measured in seconds p = probability of 3xx Response ps = successful probability of packet transmission

31. Sample Computation Call Dropping rate pcd

32. Enviroment condition for VoIP performance [4], [5] The aspects about VoIP Performance Analysis Protocols –H.323 v.s. SIP Network –Ethernet network v.s. wireless LAN (WLAN) network Security for VoIP Communication –VPN protocols : PPTP v.s. IPSec

33. Delay in Ethernet Network Both SIP and H.323 incurred higher delays in secure network-to-network environment. SIP H.323

34. Jitter in Ethernet Network IPSec produced the highest jitter values for both H.323 and SIP communications.

35. Jitter in Wireless-LAN IPSec-based VoIP communications generally incurred the highest jitter values.

36. Packet Loss Rates IPSec and PPTP increased the packet loss rate in both Ethernet and WLAN. SIP H.323

37. Performance in Satellite Network [1] Also provides IP-base data services For remote region As backup links

38. The purpose The performance under –Delay –Random errors, burst errors –Link loading Two codecs –8 kb/s G.729 –6.3/5.3 kb/s G.723.1

39. Test bed configuration

40. Baseline Tests Bandwidth and bandwidth efficiency Environment –No background traffic –No error –Link delay set 270ms –Run 15min with all 24 channel

41. Bandwidth Efficiency 5 5

42. Bandwidth A single channel

43. Link Errors Tests Random Error Tests and burst Error Tests BERs (bit error rates) = BD/(B+GC) –Burst length (B) –Burst density (D) –Gap length (G) –Link capacity kb/s (C)

44. Random Error Tests

45. Burst Error Tests

46. Link Loading Tests Environment –With different link loading levels –Link errors or not –Packet loss –Packet delay

47. Tests with an Error-Free Link

49. Tests with Errors Combine effect of both link loading and link errors. Error ↑,background traffic↓ link loading level↓ link loading level↓  link loading level can’t be pre-  link loading level can’t be pre- determined determined

52. Impact of link failures on VoIP performance [3] Three major causes of performance degradation –network congestion –link failures –routing instabilities Congestion is always negligible. Link failures may be followed by long periods of routing instability. The goal is to study the impact of link failures on VoIP performance.

53. Portion of the network topology Solid arrows: primary path Dashed arrows: alternative path used after the failure

54. Impact of failures on data traffic 06:34 R 1, R 2, R 5 : link to R 4 is down 06:35 R 1, R 2, R 5 : adjacency with R 4 recovered 06:36~06:47 instable 06:36~06:47 R 4 is instable

55. Impact of failures on data traffic 06:48 06:48 R 4 finally reboots 06:59 06:59 R 4 builds its first routing table 07:17 07:17 R 1, R 2, R 5 : link to R 4 is definitely up 07:36 an alternative path is chosen

56. Impact of failures on data traffic the failure we observed in four phases – –06:34 link is down, delay↑, few packet loss –router instable, same delay, packet loss↑ –06:36~06:47 router is instable, same delay, packet loss↑ –packet loss↑ –06:48~07:04 router reboots, no delay, packet loss↑ –07:05~07:17 –07:05~07:17 router builds routing table, delay↑, packet loss↑

57. Referencs [1]Voice over IP Service and Performance in Satellite Network, IEEE Communications Magazine [2]Technique for Performance Improvement of VoIP Applications, IEEE MELECON 2002 [3]Impact of link failures on VoIP performance, ACM 1-58113-512-2/02/0005 [4]VoIP Performance Measure Using Qos Parameters, The Second International Conference [5]VoIP Performance Management, Internet Telephony Fall 2005 [6]Comparative Analysis of Traditional Telephone and VoIP System [7]VoIP Performance on differenriate service [8]Measuring Voice Readiness of Local Area Networks [9]Experimental Investigation of the Relationship between IP Network Performances and Speech Quality of VoIP [10]Performance Measurement and Analysis of H.323 Traffic [11] A Technique to Analyse Session Initiation Protocol Traffic

58. Appendix 1 Delay within the E-model Id = Idte + Idle + Idd – –Idte -Talker echo delay – –Idle - Listener echo delay – –Idd - Long delay

59. The E-model delay measures T – mean one-way delay Ta – absolute delay Tr – round-trip delay Id can be computed by three argument

60. VoIP delay estimate Drtcp – delay estimate from RTCP packets. De – coding and packetization delay (at least as large as packet size) Dj – delay introduced by jitter buffer and decoder Ds – send side’s access delay Dr – receive side’s access delay

61. Delay measures transform T = Drtcp + Dj + De + Dr Tr = 2 * Drtcp + Dj + De Ta = Drtcp + Dj + De + Dr + Ds So the following importance is …. How to estimate the Delay?

62. Estimation of Delays Estimation of Ds and Dr defaulted to zero Estimation of De –the length of a coded fram –the codec lookahead –the number of frames in the packet –the efficiency of the coder. –choosing best-case + 20% of the frame size would be a reasonable estimate of encoding delay

63. Estimation of Delays Estimation of Drtcp Drtcp is the round-trip delay estimate divided by 2. Estimation of Dj This is dependent on the VoIP gateways jitter buffer and decoder. A possible equation for Dj is: Dj = min ( codec_frame_size + 0.9 * RTP_jitter, 300 );

64. Appendix 2 Qos

65. Parameters of VoIP performance and improvement techniques [2], [6], [7] End-to-End Delay Jitter Frame erasure Out-of-order packet delay

66. End-to-End Delay The delay from the mouth of speaker to the ear of listener Network delay packet processing in both end system packet processing in network device propagation delay Others (but leave out here) speech processing speech processing speech compression speech compression speech packetization speech packetization

67. Network delay fixed part In every network note (router) IP packets are delayed propagation delay transmission delay variable part the time spent in queues of the network nodes on the transmission path

68. Reduce network delay fixed part If the network and the transmission path are fixed  shorter IP packets  shorter IP packets variable part Some advanced queue-scheduling mechanisms e.g. the IETF document RFC 2598 e.g. the IETF document RFC 2598 Using fragments time of long packets to send

70. Reduce jitter employ a playout buffer playout time =1 playout time =1 playout time =2 playout time =2 packet loss additional delay Trade-off

71. jitter absorption Three main technique fixed playout time  static playout time Adaptive adjusting of the playout time during silence periods Constantly adapting the playoit time for each individual packet

72. Frame erasure the packet does not arrive in time the packet does not arrive in time is corrupted during the transmission through the network is dropped because of the network congestion is lost because of a network malfunction just arrives too late

74. Reduce frame erasure FEC (Forward Error Correction) need additional bandwidth and increases delays because additional processing. Loss concealment be used independently or in the combination with FEC be used independently or in the combination with FEC is effective only at low loss rate of a single frame

75. Out-of-order packet delay occurs in the network with a complex topology Done in the jitter buffer reordering (using RTP header) elimination of jitter