1 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Author: Musa Aldarawsheh Senior I&V Engineer/ Nokia Networks Supervisor: Prof. Sven-Gustav Häggman Instructor: M.Sc. Karl Tigerstedt Streaming Performance over Third Generation Cellular Networks

2 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Contents Introduction PS streaming QoS theory QoS Key Performance Indicators Environment Description Measurements Tools Preliminary Streaming QoS Tests Streaming Measurements Using Nokia Internal Streaming Server Streaming Measurements Using Helix Universal Server-Mobile Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Streaming in Highly Loaded Cell With Conversational Class Users Streaming Performance Measurement in Mobile Situation GPRS Attach and PDP Context Activation Times Summary & Conclusions References

3 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Introduction This work is aimed to verify the performance of multimedia streaming service over a WCDMA network from end-user perspectives. The measurements were carried out over streaming and interactive QoS class in stationary and mobile situation, the mobility measurements include streaming during soft handover. The streaming QoS will be verified through a set of Key Performance Indicators (KPI). These includes call setup and call quality KPIs. Multimedia streaming verification was carried out in Nokia WCDMA Test Network, in Espoo (Finland) during weeks

4 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 PS Streaming QoS Theory Multimedia streaming is a technology that allows compressed live or pre-recorded video and audio content to be delivered to the end user [Lai_p398]. Streaming service considered as real time service because the user plays back the content while downloading. Examples of streaming services are video-on-demand,web- radio, and news streams. Media can be divided into two types continuous and discrete media,The first type is only handled in this report To establish a streaming session there are different protocols involved to set-up the connection between the streaming server and the streaming client, They are : RTSP (Real Time Streaming Protocol) is used to for session set-up and session control RTP (Real Time Protocol) is used to transport media data (video/audio) over unreliable connection (UDP/IP). RTCP (Real Time Control Protocol) used to carry information on the participants and session quality. RLC transfer modes for PS Streaming RLC operates in one of three modes. These are Acknowledge mode (AM), Unacknowledged mode (UM) and transparent mode(TM). The possible RLC transfer modes for PS Streaming services are either UM-Unacknowledged mode or AM- Acknowledged mode Token Bucket Algorithm: Token bucket algorithm is a reference algorithm to conform the definition of bit rate, see [23107].

5 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 QoS Key Performance Indicators QoS is the ability of the network to provide a service with assured service level to the user of the network QoE (Quality of user Experience) is how an end user perceives the usability of a service when on use (how satisfied he/she is with the service usage) The following QoS KPIs has been defined to measure the performance of multimedia streaming over WCDMA networks. Session setup time :This is the time period when the streaming client requests media content from the streaming server until the time when the session setup between client and server is ready and the media start to flow. Initial Buffering time :This is the time elapsed between the completion of session set-up and the time that media starts playing at the client's device. This time is dependent on client memory storage. Total set-up time : This is the time when the user request a media content form the streaming server until the time the media is starting to be displayed in the streaming client. Delay Experienced by the End User: This is the time taken to establish a certain service from the point when the user request the service until the user start to receive information once the service is established. The delay experienced by a streaming service user is the sum of GPRS attach time, PDP context activation and total setup times.

6 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 QoS Key Performance Indicators Packet Delay Variation (Jitter): This is the delay variation of arrived packets. Packet losses: The number of media packets (video/audio) lost during the streaming session. Throughput: The number of received bits per unit time. Total throughput includes (media + headers) throughputs. Number and length of rebufferings: This measure is of great importance to the user because the rebuffering operation can stop the audio/video playback. In this thesis only the rebuffering periods that cause a pause in video/audio playback are taken into account. Subjective Audio/Video quality: The following simple rating scale is defined through streaming QoS measurements Excellent Good Fair Poor Bad Lip synchronisation: This metrics is used to measure the synchronization between audio and video. The metrics can be subjective and objective. The former can be based on the following rating scale. subjective Lip synchronization – In synch – Just noticeably out-of-synch – Unacceptably out-of-synch The later can be calculated in the streaming client as the ratio between displayed frames to decoded frames (Lip synchronization index)

7 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Environment description The measurements were done in the NTN (Nokia Test Network), in Leppävaara area during weeks 45-52, 2004

8 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Measurements Tools Nokia OSS system for logging cell load information Nemo Outdoor Global Positioning System (GPS) Nokia 3G Test Mobile “ Nokia 7600 model” Nokia Traffic Generator Tool Helix Mobile Producer Commview Packet Analyser Ethereal Packet Analyser Matlab Tool Excel Windows Grep Nokia Internal Streaming Client Nokia Internal Streaming Server Helix Universal Server -Mobile

9 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Preliminary Streaming QoS Tests Objectives: Preliminary evaluation of multimedia streaming over Nokia WCDMA Test Network with different client –server streaming solutions. Test Description: Four tests were executed by using two different streaming client–server solutions, two tests for each solution. The client-server streaming solutions are: 1. Nokia Internal Streaming Client and Nokia Internal Streaming Server 2. Nokia Internal Streaming Client and Helix Universal Server-Mobile The test was done in stationary situation near to the WCDMA base station where the carrier to interference ratio Ec/No > -6 dB. Two different RT PS RABs were used, 64/64 and 64/128 UL/DL. Multimedia files were used with bit rates 51 and 115 kbps over 64/64 and 64/128 bearers respectively.

10 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Preliminary Streaming QoS Tests VideoAudio Subjective QoS (MOS)Good Packet Loss Ratio, Number of expected packets to be received 0.5%, %, 750 Preliminary test 1:Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1% Environment: Nokia internal streaming client + Nokia internal streaming The test results are presented in Table 1. Average video/audio packets sizes are 450 and 150 byte respectively Table 1 Streaming performance over 64/64 RT RAB The results show a good performance of multimedia streaming when using Nokia Streaming Client –Server. The MOS was “Good” for both audio and video.

11 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Preliminary Streaming QoS Tests Preliminary test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1% Environment: Nokia internal streaming client + Nokia internal streaming The test results are presented in Table 2. Average video/audio packets sizes are 381 and 313 byte respectively Table 2 Streaming performance over 64/128 RT RAB VideoAudio Subjective QoS (MOS)Fair Packet Loss Ratio, Number of expected packets to be received 1.9%, %, 300 The results show a satisfactory performance of multimedia streaming when using Nokia Streaming Client –Server solution. The MOS was “Fair” for both audio and video. Video packet losses were in acceptable level.

12 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Preliminary Streaming QoS Tests Figure 1 Helix Server throughput over 64/64 RT bearer Streaming Server sending bit rate more than guaranteed channel bit rate Preliminary test 3: Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1% Environment: Nokia internal streaming client + Helix Server The test results are presented in Table 3. Average video/audio packets sizes are 485 and 150 byte respectively. The results show a slightly high video packet losses and low audio packet losses, high video packet losses explain the poor perceived video quality VideoAudio Subjective QoS (MOS)PoorFair Packet Loss Ratio, Number of expected packets to be received 6.6%, %, 299 Table 3 Streaming performance over 64/64RT RAB

13 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Preliminary Streaming QoS Tests Figure 2 Helix Server throughput over 64/128 RT bearer Preliminary test 4: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1% Environment: Nokia internal streaming client + Helix Server The test results are presented in Table 4. Average video/audio packets sizes are 333 and 150 byte respectively. The results show a very high video/audio packet loss, which explain the poor perceived video/audio quality. Streaming Server sending bit rate more than guaranteed channel bit rate VideoAudio Subjective QoS (MOS)Bad Packet Loss Ratio, Number of expected packets to be received 29.6 %, %, 301 Table 4 Streaming performance over 64/128RT RAB

14 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Preliminary Streaming QoS Tests The streaming performance over 64/64 and 64/128 UL/DL real time bearers was bad when using Helix server. The streaming performance over 64/64 and 64/128 UL/DL real time bearers was satisfactory when using Nokia internal streaming server. The tests had shown that streaming server was very often sending media packets with short delays and the data rate exceeded the channel maximum bit rate. Data rate that exceed the maximum channel rate will be discarded according to token bucket algorithm. The current implementation of token bucket algorithm is done according to 3GPP specification [TS Annex B], which is not handling appropriately the burst traffic situation. As a result of these findings the token bucket algorithm was modified to be able to tolerate traffic burstiness conditions. This modification allows streaming class to use traffic shaping functionality, which smooth the traffic pattern and decrease the burstiness of the network. All the measurements that follow were done with the modified token bucket algorithm except SHO measurements.

15 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Objectives: To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation Test 1: Multimedia streaming over 64/64 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server AT command used :at+cgeqreq=1,1,64,64,64,64,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 589 and 750 respectively. Average video and packet sizes are 485 and 150 byte respectively. Table 5 Call setup QoS KPIs over 64/64 streaming bearer, BLER 1% TrialSetup time msec Buffering Time msec Total Setup time msec The end user will experience the following delays before start to see the movie e.g in trial 1 the end user experience the following delays Attach time+PDP context time + setup time + buffering time = =18.23 sec See GPRS attach and PDP context measurements.

16 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Trial Average Throughput kbps PLR % audio PLR% video Packet Delay video Min,max,avg msec Packet Delay Audio Min,max,avg msec MOS video MOS audio Lip Sync index Subjective lip synchronizatio n Number and Length of rebufferings (msec) %1.26%0,521,96.730,671,200.5Good 0.99In synch0 / %2.05%0,333,9710,681,204.6FairGood0.69In synch0 / %0.33%0,301,9640,681,204.2Good 0.85In synch0 / 0 Table 6 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 streaming bearer, BLER target 1% Figure 3 shows the total throughput (media + header throughputs) received by the streaming client. The received throughput is within the channel bandwidth in most of the times. Figure 3 Streaming client total throughput over 64/64 RT bearer

17 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Objectives: To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation Test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 2136 and 310 respectively. Average video and packet sizes are 333 and 150 byte respectively. Table 7 Call setup QoS KPIs over 64/128 RT streaming bearer, BLER 1% TrialSetup time msec Buffering Time msec Total Setup time msec

18 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Packet losses are in acceptable level Subjective QoS was good Lip sync index shows a satisfactory values No pause and rebufferings occurred during the streaming session. The received throughput is within the channel bandwidth in most of the times. Trial Average Throughput kbps PLR % audio PLR% video Packet Delay video Min,max,avg msec Packet Delay Audio Min,max,avg msec MOS video MOS audio Lip Sync index Subjective lip synchronization Number and Length of rebufferings (msec) %0.33 %0,171,28.410,621,202Good 0.57 Just noticeably out of synch 0 / %0.23%0,160,2820,601,201Good 0.97 In Synch 0 / 0 Table 8 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 streaming bearer, BLER target 1% Figure 4 Streaming client total throughput over 64/128 RT bearer

19 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Objectives: To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation Test 3: Multimedia streaming over 64/256 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average video and packet sizes are 341 and 321 byte respectively. Table 9 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 % TrialSetup time msec Buffering Time msec Total Setup time msec

20 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Real Time Bearers in Unloaded Cell Trial Average Throughput kbps PLR % audio PLR% video Packet Delay video Min,max,avg msec Packet Delay Audio Min,max,avg msec MOS audio MOS video Lip Sync index Subjective lip synchronization Number and length of rebufferings msec %0.93 %0,821,16.910,991,207FairPoor0.46Unacceptably out-of-synch 5 / % 0,4707,1710,1192,206FairBad0.34Unacceptably out-of-synch 11 / %0.8%0,3345,17.60,1192,209FairBad0.38Unacceptably out-of-synch 5 / 5860 Figure 5 shows the total throughput received by the streaming client. The total throughput was exceeding the channel bandwidth in most of the times. Subjective video/audio quality was not totally satisfactory Lip sync index shows a very low values which indicates that the video and audio are out of sync Many rebufferings occurred during the video. Table 10 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 streaming bearer, BLER target 1% Figure 5 Streaming client total throughput over 64/128 RT bearer

21 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell TrialSetup Time msec Buffering Time msec Total Setup time msec Objectives: Total number of video and audio packets sent by streaming server is 2136 and 300 To verify the performance of multimedia streaming over NRT bearer in a stationary situation Test 4: Multimedia streaming over 64/64 kbps UL/DL NRT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server respectively. Average video and packet sizes are 333 and 321 byte respectively. Table 11 Call setup QoS KPIs over 64/64 NRT streaming bearer, BLER 1 %

22 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Trial Average Throughput kbps PLR % audio PLR% video Packet Delay video Min,max,avg msec Packet Delay Audio Min,max,avg msec MOS video MOS audio Lip Sync index Subjective lip synchronization Number and Length of rebufferings (msec) % 0,555,96.730,671,200Good 0.86In Sync0 / % 0,340,9710,681,204Good 0.76In Sync0 / % 0,370,9640,680,203Good 0.82In Sync0 / 0 Table 12 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 NRT bearer, BLER target 1% No Packet losses Subjective video and audio QoS was good Lip sync index shows a good values No pause and rebuffering occurred during the streaming session.

23 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell TrialSetup Time msec Buffering Time msec Total Setup time msec Objectives: To verify the performance of multimedia streaming over NRT bearer in a stationary situation Test 5: Multimedia streaming over 64/128 kbps UL/DL NRT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server Total number of video and audio packets sent by streaming server is 2136 and 300 respectively. Average video and packet sizes are 333 and 321 byte respectively. Table 13 Call setup QoS KPIs over 64/128 NRT streaming bearer, BLER 1 %

24 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Trial Average Throughput kbps PLR % audio PLR % video Packet Delay video Min,max,avg msec Packet Delay Audio Min,max,avg msec MOS video MOS audio Lip Sync index Subjective lip synchronization Number and length of rebufferings msec % 0,231,28.320,590,200Good 0.62In Sync0 / % 0,230,28.410,661,202Good 0.81In Sync0 / 0 No Packet losses Subjective video and audio QoS was good Lip sync index shows a good values No pause and rebuffering occurred during the streaming session. The received throughput is exceeding the channel Bandwidth sometimes Table 14 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 NRT bearer, BLER target 1% Figure 6 Streaming client total throughput over 64/128 NRT bearer

25 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell TrialSetup Time msec Buffering Time msec Total setup time msec Objectives: To verify the performance of multimedia streaming over NRT bearer in a stationary situation Test 6: Multimedia streaming over 64/384 kbps UL/DL NRT PS, BLER target 1% Environment: Nokia internal streaming client + Helix Server Total number of video and audio packets sent by streaming server is 4066 and 301 respectively. Average video and packet sizes are 333 and 321 byte respectively. Table 15 Call setup QoS KPIs over 64/384 NRT streaming bearer, BLER 1 %

26 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell Trial Average Throughput kbps PLR % audio PLR% video Packet Delay video Min,max,avg msec Packet Delay Audio Min,max,avg msec MOS audio MOS video Lip Sync index Subjective lip synchronization Number and length of rebufferings msec %1.1 %0,390,15.30,1041,204GoodFair0.45Just noticeably out-of-synch 2 / %0.1%0,491,14.90,931,203GoodFair0.5Just noticeably out-of-synch 2/ 5944 Packet losses ~ 1% Subjective audio/video QoS was satisfactory Lip sync index shows a relatively low values The received throughput is exceeding the channel Bandwidth very often Table 16 End-to-End and objective/subjective video/audio QoS KPIs over 64/384 NRT bearer, BLER target 1% Figure 7 Streaming client total throughput over 64/384 NRT bearer

27 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming in Highly Loaded Cell With Conversational Class Users Objective: To verify that the guaranteed bit rate of a streaming class user is kept in a highly loaded cell. Test Description: The test was executed by starting a single video streaming session with a predefined data rate and loading the cell with conversational class users (voice calls). The throughput of streaming user was monitored. Environment: Increasing common channels power by 11 dB

28 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming in Highly Loaded Cell With Conversational Class Users Test 7: Streaming over 64/64 RT bearer Total # of send packets =1389 Total # of packets lost = 15 Packet losses ~1% Throughput decreased significantly due to packet losses High BLER DL BLER Throughput PPP Layer CPICH RSCP UE Tx Pwr CPICH Ec/No

29 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming in Highly Loaded Cell With Conversational Class Users Test 7: Streaming over 64/64 RT bearer PtxTotal DL=~ Ptxtarget DL UL RT Load New AMR calls are not admitted after 33 existing AMR calls

30 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming in Highly Loaded Cell With Conversational Class Users Test 8: Streaming over 64/128 RT bearer Total # of send packets =8118 Total # of packets lost = 36 Packet losses 0.4% Throughput is not affected by cell congestion Throughput PPP Layer DL BLER CPICH RSCP UE Tx Pwr CPICH Ec/No New AMR calls are not admitted after 37 existing AMR calls

31 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming in Highly Loaded Cell With Conversational Class Users Test 9: Streaming over 64/256 RT bearer Total # of send packets =8247 Total # of packets lost = 34 Packet losses 0.4% CPICH EcNo UE TX Power CPICH RSCP Throughput PPP Layer DL BLER New AMR calls are not admitted after 37 existing AMR calls

32 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming Performance Measurement in Mobile Situation Softer handover Soft handover Objective: To verify the performance of multimedia streaming over real time bearer during soft handover in Nokia WCDMA Test Network. The description of tests is as follow: Two tests were executed over two streaming class bearers, one test per bearer. The bearer bit rates are 64/64 and 64/128 kbps UL/DL. BLER target value was set to 1% for all tests. Multimedia files were used with bit rates 103 and 213 kbps over 64/128 and 64/256 RT PS RABs respectively. The tests were conducted by using Nokia internal streaming client and Nokia streaming server. Streaming performance was verified during soft handover. The carrier to interference ration was “Ec/No > -6” dB. Every test is repeated 2-3 times to ensure reliability. This test was executed with original token bucket algorithm

33 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer Objectives: To verify the performance of multimedia streaming during SHO Test 10: Multimedia streaming during SHO over 64/128 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Nokia internal streaming server AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 1880 and 296 respectively. Average video and packet sizes are 382 and 313 byte respectively. Table 17 Call setup QoS KPIs over 64/128 streaming bearer, BLER 1 % TrialSetup time msec Buffering Time msec Total Setup time msec

34 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer Video packet losses are high due to token packet algorithm problem Lip sync index shows a satisfactory values No rebufferings occurred during the streaming session. Trial Average Throughput kbps PLR % audio PLR% video Packet Delay video Min,max,avg msec Packet Delay Audio Min,max,avg msec MOS video MOS audio Lip Sync index Subjective lip synchronization Number and Length of rebufferings (msec) %7.34%0,221,3310,651,212FairGood0.58Just noticeably out-of-synch 0 / %7.5 %0,341,3460,631,210FairGood0.56Just noticeably out-of-synch 0 / %7.3%0,251,3320,570,214FairGood0.55Just noticeably out-of-synch 0 / 0 Table 18 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 RT bearer during SHO, BLER target 1% Figure 8 Streaming client total throughput over 64/128 RT bearer

35 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming Performance Measurement in Mobile Situation Streaming over 64/128 RT bearer Soft HO Softer HO Soft handover takes place between different base stations sites within one RNC (intra-RNC soft handover) or within separate controlled RNCs (inter-RNC soft handover). In softer handover the UE is controlled by at least two cells belonging to the same base station site.

36 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer Objectives: To verify the performance of multimedia streaming during SHO Test 11: Multimedia streaming during SHO over 64/256 kbps UL/DL RT PS, BLER target 1% Environment: Nokia internal streaming client + Nokia internal streaming server AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000 Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average video and packet sizes are 341 and 321 byte respectively. Table 19 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 % TrialSetup time msec Buffering Time msec Total Setup time msec

37 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer Trial Average Throughput kbps PLR % audio PLR% video Packet Delay video Min,max,avg msec Packet Delay Audio Min,max,avg msec MOS audio MOS video Lip Sync index Subjective lip synchronization Number and length of rebufferings msec %7.25%0,180,2260,460,204GoodFair0.56Just noticeably out-of-synch 0 / %6.95%0,170,2110,445,202GoodFair0.56Just noticeably out-of-synch 0 / %7.05%0,330,2130,511,207GoodFair0.55Just noticeably out-of-synch 0 / 0 Video packet losses are high due to token packet algorithm problem Subjective audio/video QoS was satisfactory Lip sync index shows satisfactory values No rebufferings occurred during the streaming session. Table 20 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 RT bearer during SHO, BLER target 1% Figure 9 Streaming client total throughput over 64/256 RT bearer

38 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Streaming Performance Measurement in Mobile Situation Streaming over 64/256 RT bearer Soft HO Softer HO

39 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 GPRS Attach and PDP Context Activation Times Objectives: To measure the GPRS attach and PDP context activation times over streaming class bearers. Three different RAB bit rates are used, 64/64, 64/128, and 64/256 kbps UL/DL. BLER target value was 1% for all tests. Every test was repeated 15 times to ensure reliability. Attach Times were calculated as the difference time between ATTACH_REQ and ATTACH_COMPLETE messages. PDP context activation time was calculated as a time difference between PDP_context_create_req and PDP_context_activate_ind messages Table 33 shows that average PDP context + GPRS attach time for 64/64, 64/128, and 64/256 are 6.71, 7.09, and 7.28 seconds. Table 21 Average GPRS attach+ PDP Context Activation. Real Time BearerPDP context + GPRS attach Average Time (sec) 64/ / /

40 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Summary & Conclusions The preliminary evaluation tests using Nokia internal streaming server had shown a good performance of multimedia streaming over 64/64 UL/DL real time bearer and a satisfactory performance over 64/128 UL/DL real time bearer. The preliminary evaluation tests using Helix streaming server had shown a bad performance of multimedia streaming over 64/64 and 64/128 UL/DL real time bearers. The following conclusions are drawn from the preliminary evaluation tests: Nokia streaming server has shown a better performance than Helix streaming server. The bad performance of multimedia streaming over streaming class was due to the large number of discarded packets by the token bucket algorithm. The current implementation of token bucket algorithm, which is according to 3GPP specification was not suitable to control the traffic flow of streaming servers. The results obtained after token bucket modification have shown a good performance of multimedia streaming over 64/64 and 64/128 real time bearers and the performance over 64/256 UL/DL RT bearer was not totally satisfactory. The performance of video streaming over 64/64 and 64/128 NRT bearers was good. Video and audio quality was good and video/audio packet losses were 0.0% in all trials.

41 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 Summary & Conclusions The performance of video streaming over 64/384 UL/DL NRT bearer was satisfactory. The performance of multimedia streaming over 64/128 and 64/256 UL/DL RT during SHO was satisfactory. (original token bucket algorithm + Nokia streaming server) The following conclusions are drawn from the streaming verification tests after token bucket algorithm modifications. The performance of video streaming over real time bearer was significantly improved after token bucket algorithm modification. Multimedia streaming over NRT bearers in unloaded cell show a slightly better performance than streaming over RT bearers. The streaming service user will experience a delay of ~20 seconds before the media contents starts to be displayed in the streaming client. This delay includes attach, context, and total setup times.

42 © NOKIA Streaming QoS/Musa Aldarawsheh, April 8 th 2005 References [Hol] Harri Holma and Antti Toskala, ”’WCDMA for UMTS Radio Access For Third Generation Mobile Communications”, Second Edition, John Wiley & Sons, LTD, 2002 [Lai] Jaana Laiho, Achim Wacker and Tomas Novosad, “ Radio Network Planning and Optimisation for UMTS”, John Wiley & Sons, LTD, 2002, 484 pages. [22105] 3GGP TS Service aspects; Services and Service Capabilities, version 4.1.0, January 2001, 31 pages. [23107] 3GPP Technical Specification : QoS concept and Architecture, version 4.6.0, December 2002, 38 pages