Content and presentation of Recommendation E.MQoS ITU Workshop on “Benchmarking QoS Evaluation of Multimedia Networks” (Ouagadougou, Burkina Faso, 18 July 2013) Content and presentation of Recommendation E.MQoS Joachim Pomy, SG 12 Rapporteur Consultant, Opticom GmbH Consultant@joachimpomy.de Ouagadougou, Burkina Faso, 18 July 2013

Draft New Rec. E.MQoS E-series: E.800-series: Title: Overall network operation, telephone service, service operation and human factors E.800-series: Quality of telecommunication services: concepts, models, objectives and dependability planning Title: QoS Aspects for Popular Services in Mobile Networks

Scope of E.MQoS (1/2) Assessment of Quality of Service Summary of basics of QoS, always seen from the user's perspective Differences to Quality of Experience (QoE) Guidance to assure QoS assessments conducted in a meaningful way Definition of Quality of Service parameters and their computation QoS parameters and their computation for popular services abstract definition = generic description of parameters abstract equation user and technical trigger points Harmonized definitions are prerequisites for the comparison of QoS measurements and measurement results Typical procedures for Quality of Service measurement equipment Measurement procedures needed to perform the measurements of QoS parameters

Scope of E.MQoS (2/2) Requirements for QoS measurement equipment Defines minimum requirements of QoS measurement equipment values and trigger points needed to compute the QoS parameter allow performing measurements in a reliable and reproducible way Definition of typical measurement profiles Specifies typical measurement profiles required to enable benchmarking of different mobile networks both within and outside national boundaries. Post processing and statistical methods Describes procedures for statistical calculations for QoS measurement of mobile networks using probing systems. Network based Quality of Service measurements E2E QoS measurements inside the network without direct access to the end point terminal.

Services are not standardized Focus on popular services Widely used in the market Test results are meaningfull to customers Examples Web radio E-mail clients Web portals

QoS aspects of Mobile

Main structure of E.MQoS 6 Assessment of Quality of Service 7 Definition of Quality of Service parameters and their computation 8 Typical procedures for Quality of Service measurement equipment 9 Requirements for Quality of Service measurement equipment 10 Definition of typical measurement profiles 11 Post processing and statistical methods 12 Network based Quality of Service measurements Appendices (informative)

6 Assessment of Quality of Service 6.1 QoS Background 6.2 QoS Assessment Process 6.3 Basic Settings for QoS Assessments 6.4 Service Independent QoS Criteria 6.5 Service dependent QoS Criteria

7 Definition of Quality of Service parameters and their computation 7.1 QoS Parameter Basics 7.2 Service independent QoS Parameters 7.3 Direct Services QoS Parameters 7.4 Store‑and‑forward (S&F) Services QoS Parameters

7.2 Service independent QoS Parameters 7.2.1 Radio Network Unavailability [%] 7.2.2 Network Non Accessibility [%] 7.2.3 Attach Failure Ratio [%] 7.2.4 Attach Setup Time [s] 7.2.5 PDP Context Activation Failure Ratio [%] 7.2.6 PDP Context Activation Time [s] 7.2.7 PDP Context Cut‑off Ratio [%] 7.2.8 Data Call Access Failure Ratio [%] 7.2.9 Data Call Access Time [s] 7.2.10 DNS Host Name Resolution Failure Ratio [%] 7.2.11 DNS Host Name Resolution Time [s]

7.3 Direct Services QoS Parameters 7.3.1 File Transfer (FTP) 7.3.2 Mobile Broadcast 7.3.3 Ping 7.3.4 Push to Talk over Cellular (PoC) 7.3.5 Streaming Video 7.3.6 Telephony 7.3.7 Video Telephony 7.3.8 Web Browsing (HTTP) 7.3.9 Web Radio 7.3.10 WLAN service provisioning with HTTP based authentication 7.3.11 Wireless Application Protocol (WAP) 7.3.12 IMS Multimedia Telephony 7.3.14 Group Call 7.3.13 E-mail

7.4 Store‑and‑forward (S&F) Services QoS Parameters 7.4.1 Generic Store‑and‑forward Parameters 7.4.2 E‑mail 7.4.3 Multimedia Messaging Service (MMS) 7.4.4 Short Message Service (SMS) Short Data Service (SDS)

8 Typical procedures for Quality of Service measurement equipment 8.1 Aim of measurement 8.2 Classification of services 8.3 General aspects for all types of services 8.4 Telephony measurements 8.5 Store-and-forward services measurement 8.6 Data measurements

8.4 Telephony measurements 8.4.1 General aspects 8.4.2 Speech telephony 8.4.4 Group Call 8.4.3 Video telephony

8.5 Store-and-forward services measurement 8.5.1 General aspects 8.5.2 SMS measurements 8.5.3 MMS 8.5.4 E-Mail 8.5.5 SDS

8.6 Data measurements 8.6.1 Common aspects 8.6.2 FTP 8.6.3 HTTP 8.6.4 E-mail 8.6.5 WAP 8.6.6 Streaming Video 8.6.7 Media Download

9 Requirements for Quality of Service measurement equipment 9.1 Overview 9.2 General requirements 9.3 Fixed QoS Test-equipment (FQT) 9.5 Mobile based measurement equipment 9.4 Mobile QoS Test-equipment (MQT)

10 Definition of typical measurement profiles 10.1.3 Usage Profiles for Data Sessions 10.1.1 Classification of measurement environments 10.1.2 Service profiles

11 Post processing and statistical methods 11.1 Important measurement data types in mobile communications 11.2 Distributions and moments 11.3 Visualization techniques 11.4 Time series modelling 11.5 Data aggregation 11.6 Assessment of performance indices

12 Network based QoS measurements 12.1 Network Measurement Basics 12.2 Measuring QoS Parameters in the Network 12.3 Comparing Network and End- point Test Measurements

Appendices (informative) I Examples for measuring trigger points II Streaming explanations III Push to Talk over Cellular Information IV QoS parameter export V Which parameters have an impact on the effect of blocking ? VI Reference {SMS | SDS} VII Content integrity checking VIII Transfer times versus used data rate and content size IX Examples of statistical calculations X The concept of QoE reporting XI Examples of Network Based QoS Measurements XII 3GPP SA5 "UE Management"

Confidence Intervalls for Different Sample Sizes (1) The following examples show the effect of different sample sizes in a measurement campaign. It is also based on the Pearson-Clopper formulas for the calculation of confidence intervals. Therefore, the examples are valid in a generic way and even for small sample sizes. For higher sample numbers, the calculation of confidence intervals based on the approximation of a normal distribution can be applied. Three different graphs are depicted: Sample sizes in the range: between 100 and 1 100 samples; between 1 100 and 2 100 samples; and between 1 000 and 11 000 samples.

Confidence Intervalls for Different Sample Sizes (2)

Confidence Intervalls for Different Sample Sizes (3)

Confidence Intervalls for Different Sample Sizes (4)

Schedule for draft new Rec. E.MQoS 31 August 2013 Final timeline for submitting comments directly to the editors 7 November 2013 Submission as Contribution to SG 12 20 November 2013 Deadline for Contributions submitted to TSB 12 December 2013 Planned Consent of SG12 Plenary January 2014 Start of Alternate Approval Procedure (AAP) March 2014 Pre-Publication ??? 2014 Publication

Any questions ? Joachim Pomy Telecommunications & Int'l Standards Germany Tel.: +49 177 78 71958 Email: consultant@joachimpomy.de