© Omnitele Ltd Analysis of Service Quality in 3G Mobile Networks Heidi Lagerström Supervisor: Professor Heikki Hämmäinen Instructors: M.Sc. Sami Vesala & M.Sc. Katja Koivu

© Omnitele Ltd Contents 1.Introduction to the study Background, research problem, research methods 2.Quality of Service (QoS) in UMTS Networks 3.Measuring service quality Defining Key Performance Indicators (KPI) 4.Case study

© Omnitele Ltd Background UMTS introduces new real time services to mobile networks, such as video telephony. These real time services require QoS guarantees to function properly. For operators to maintain satisfactory service quality constant network monitoring is needed. Network measurements are based on correctly defined KPIs for each service. Operators’ possibilities to utilise QoS in practice have not been widely researched. Key Performance Indicators have not been defined for the new services from the end-user perspective.

© Omnitele Ltd Research problem Objectives: 1.What are the KPIs that measure service quality, from end user perspective, in 3G networks for the key services (AMR voice, video telephony, video streaming, web browsing and )? 2.What are the QoS mechanisms in Release 99 and how can they be used to improve service quality? How should service quality be measured in 3G networks and how the QoS mechanisms can be used to affect the service quality perceived by subscribers.

© Omnitele Ltd Research methods Literature study –3GPP, ETSI, ITU specifications –Several books and publications Interviews –Network equipment vendors: Ericsson, Nokia –Operators: Elisa –Several other radio network experts Case study –Field measurements for two operators in live networks

© Omnitele Ltd Contents 1.Introduction to the study Background, research problem, research methods 2.Quality of Service (QoS) in UMTS Networks 3.Measuring service quality Defining Key Performance Indicators (KPI) 4.Case study

© Omnitele Ltd Why do we need QoS? UMTS networks support services with very different performance requirements –Real-time services require performance guarantees –Customer acceptance closely tied to service quality Optimal usage of network resources –Radio resources scarce –Cost-effectiveness –Return of investment Service and user differentiation –Meet different needs of customers (e.g. business vs. consumer) –Support different services (real-time vs. best effort) Competitive advantage!

© Omnitele Ltd QoS Traffic Classes Traffic classCharacteristicsExample application ConversationalPreserve time relation between information entities of the stream. Conversational pattern (stringent and low delay) Speech Video calls StreamingPreserve time relation between information entities of the stream. Real-time streaming video InteractiveRequest-response pattern. Preserve payload content. Web browsing BackgroundDestination is not expecting the data within a certain time. Preserve payload content. File downloading Demanding Delay Jitter Demanding Bit rate Jitter Tolerant Delay and bit rate can vary Integrity Easiest Delay and bit rate can vary Integrity

© Omnitele Ltd QoS Profile Attributes R99 QoS attributeExample value Residual BER10 –5 SDU error ratio10 –4 Delivery of erraneous SDUsNo Maximum SDU size (octets)1500 Delivery orderNo Transfer delay100 ms (conversational) 280 ms (streaming) ARP1, 2 or 3 Traffic ClassConversational, streaming, interactive, background THP1, 2 or 3 (same as ARP) Maximum allowed bit ratee.g. 64, 128 or 384 kbps Maximum guaranteed bit ratee.g. 64, 128 or 384 kbps Depends on operator’s QoS strategy Depends on the QoS strategy and UE/RNC capabilities

© Omnitele Ltd QoS Differentiation Conversational RAB Streaming RAB Interactive RAB, THP/ARP = 1 Interactive RAB, THP/ARP = 3 Background RAB MMS Web browsing Push-to-talk Streaming Video telephony Each service gets the treatment it requires according to the QoS profile Network resources are shared according to the service needs Network resources can be used more efficiently

© Omnitele Ltd QoS Mechanisms External IP (Internet) UE Node B TE Firewall RNC3G-SGSN3G-GGSN UTRAN Iu Gn IP PS Domain Inter-PLMN Backbone NT DiffServ on transport level IP (ATM QoS for CS ) Diffserv on transport level IP Diffserv in Gi Different channel types ATM QoS Different QoS techniques in different parts of the network Appropriate QoS must be provided in every network so that the user can experience good service quality PDP context with the requested QoS capabilities RRM

© Omnitele Ltd Operators’ QoS Strategy Conversational RAB Streaming RAB Interactive RAB, THP/ARP = 1 Interactive RAB, THP/ARP = 3 Background RAB RNC 3G GGSN 3G SGSN Application server Node B HLR User profiles are stored in HLR. Each user can have several user profiles, which correspond to different services and are mapped to different bearers according to the operator’s strategy. Operators can practise user differentiation by giving each user set of QoS profiles, which he/she is entitled to use Operators can practise service differentiation by mapping each service to the bearer that meets its requirements Meet the needs of different customers Offer each service the quality it requires Optimise network resource usage

© Omnitele Ltd Contents 1.Introduction to the study Background, research problem, research methods 2.Quality of Service (QoS) in UMTS Networks 3.Measuring service quality Defining Key Performance Indicators (KPI) 4.Case study

© Omnitele Ltd Measuring network performance RNC 3G GGSN 3G SGSN Application server Node B UTRANCore nwExternal nw E2E service quality, QoE Performance statistics from application server Network statistics from different counters and interfaces Customer feedback Network Performance Monitoring  Optimisation

© Omnitele Ltd Defining the right KPIs Different services have different quality requirements –KPIs must be defined separately for each of the key services KPI categories –Service Accessibility –Service Integrity –Service Retainability With inadequate performance indicators and monitoring –Hidden problems in network performance and user perceived quality of service –Poorly defined indicators may show better quality than in the reality Incorrect formulas and counters Unreasonable measurement periods (too much averaging etc.)

© Omnitele Ltd Example: Voice Services - CS Customer demand IndicatorMeasure Service accessability Availability & Coverage Call setup success rate Call setup delay Ec/No, RSCP Admission control RAB assignment Service integrityVoice qualityNoisy frames (FER), MOS Service retainability Call drop rateHandover failure No coverage Interference

© Omnitele Ltd Example: Data Services - PS Customer demandIndicatorsMeasures Service accessability Availability & Coverage Access success rate Service access delay Ec/No, RSCP Admission control Attach, PDP context activation, IP service setup Service integrityVideo quality Audio quality Web page download time sending time, etc. BLER, FER, throughput, delay, jitter Service retainability Dropped data connection Connection timeouts Dropped PDP context/attach No coverage etc. Handover failure

© Omnitele Ltd Contents 1.Introduction to the study Background, research problem, research methods 2.Quality of Service (QoS) in UMTS Networks 3.Measuring service quality Defining Key Performance Indicators (KPI) 4.Case study

© Omnitele Ltd Measurement plan Operator 1Operator 2 Drive test: AMR speechXX Drive test: FTP downloadXX Video telephonyXN/A StreamingXX Web page downloadXX XX Data connection: attach, PDP context activation, RTT, FTP DL & UL XX Tools: Nemo Outdoor, Optimi x-AppMonitor, Ethereal

© Omnitele Ltd AMR voice – Drive test statistics Good < 21dBm Good > -92dBm Good > -10dB Shoud be ~100% Reasons for call failure: Ec/N0 was not at adequate level Call setup was unsuccessful (unsuccessful RACH procedure) Look at L3 signalling

© Omnitele Ltd CPICH coverage – Ec/N0 According to Ec/N0 values both operators have good coverage. Couple of RED areas, which need to be further investigated! Operator 1 Operator 2 If large interference areas are generated, the problem could be minimised later by adjusting the antenna direction or height, or by down tilting the antenna or by slightly tuning the pilot power levels.

© Omnitele Ltd Data Connection Typical RTT in UMTS network is ~200ms, which enables good quality conversational PS services, such as VoIP.

© Omnitele Ltd Video streaming In mobile phone display ~60 kbps streaming bit rate produces good video quality.

© Omnitele Ltd Web browsing Sample web page 319 kB

© Omnitele Ltd Conclusions In 3G networks QoS management is required –Real-time services require QoS guarantees –Need to support different kinds of services –With QoS mechanisms operators can use their network resources more efficiently and gain competitive advantage To maintain and improve the network performance and user experienced service quality constant monitoring and performance follow-up is needed –Successful network measurements are based on correct KPI definitions –A combination of end-to-end field measurements, interface probes, network element counter statistics and customer feedback is required The measurement results show that there are big differences in the performance of operators’ UMTS networks –Currently UMTS networks are not fully optimised  there is a clear need for optimisation! –Majority of 3G measuring equipment and terminals are still quite immature

© Omnitele Ltd © Omnitele Ltd For more information about Omnitele, please visit our web site

© Omnitele Ltd © Omnitele Ltd KPI Definitions

© Omnitele Ltd AMR Speech KPIs Parameters Trigger points Place a callAlerting messageSpeech interchange Intentional termination of session Service accessibility Service access time Service coverage Service retainability T0 T1 T2T3 Speech quality Channel request ALERTINGStart of audio streamRELEASE Codec usage

© Omnitele Ltd Video Telephony KPIs Parameters Trigger points Video call Request Alerting messageAudio/video output Intentional termination of session Service accessibility Service access time Service coverage Video call setup time Video call setup success ratio Service retainability Video quality Speech quality Audio/video synchronisation T0 T1 T2T3 Channel request ALERTING / Call accepted Audio/video output starts Audio/video output ends RELEASE

© Omnitele Ltd Video Streaming KPIs Parameters Trigger points Stream Request Buffering message appears on player Stream reproduction Intentional termination of session Service accessibility Service access time Service coverage Streaming reproduction start delay Streaming reproduction start failure Streaming reproduction cut-off ratio Video quality Audio quality Audio/video synchronisation T0 T1 T2T3 RTSP: SETUP RTP: payload 1 st data packet BUFFERING Streaming reproduction starts – picture appears PLAY RTSP TEARDOWN Video/audio stream ends

© Omnitele Ltd Web Browsing KPIs Parameters Trigger points Service accessData transfer Intentional termination of session Service accessibility Service access time Service coverage Service retainability Web page download time T0 T1 T2T3 1st TCP [SYN] 1st HTTP: GETHTTP: FIN/ACK Reception of last data packet  Display data

© Omnitele Ltd KPIs Parameters Trigger points Service access sending download Service accessibility Service access time Service coverage Sending timeReceiving time T0 T1 T2T4 Service retainability 1st TCP [SYN] SMTP: 250 ACK (HELO) Last data packet send TCP [FIN/ACK] Last data packet received TCP [FIN/ACK] T3 IMAP: FETCH Body