1© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Next Generation Internet and Quality of Experience Kalevi Kilkki EuroFGI IA.7.6 Workshop on Socio-Economic Issues of NGI Santander, Spain June 2007

2© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Content What is Quality of Experience? How could we analyze it?

3© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Historical perspective: Papers in IEEE database QoS vs. QoE Quality of Service or QoS Quality of Experience or QoE Terms in abstract Relative share of QoS papers with the keyword 1 = average share of QoS papers over all networking papers Too many QoS papers is bad for any networking technology because the scope of QoS is too narrow

4© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Very brief literature study about Quality of Experience Quotations from QoE papers Liu Li-yuan, et al, 2006 “The function of quality of experience (QoE) evaluation includes two aspects: to monitor the experience of user on-line, then to control and justify the service based on the QoE to ensure that the quality of service can highly meet the requirements of the user.” Lopez, et al, 2006 “Quality of experience (QoE) has been defined as an extension of the traditional quality of service (QoS) in the sense that QoE provides information regarding the delivered services from an end-user point of view.” Some more references in extra slides

5© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / What does QoE mean? Definition = meaning (not purpose) It is questionable to define QoE as an extension of something (QoS), or through a specific goal (optimization of service). Wikipedia In business terminology QoE is also known as "Quality of User Experience," and is a subjective measure of a customer's experiences with a vendor. QoE as regards telecommunications networks is a purely subjective measure from the user’s perspective of the overall value of the service provided. Obviously some confusion about the genuine meaning of QoE Let us take a somewhat broader view

6© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / The big picture: Ecosystem modeling Communications ecosystem is immensely complex To manage the complexity, it can be divided into three areas Human behavior Business analysis Technical performance Ultimate goal is to give credible estimates about benefits obtained by persons, user and customer behavior, and business opportunity (€) Application Service provider User Network operator Network Customer Device Device vendor Content owner Group member Person € €

7© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Modeling human behavior Person with different roles Person when thinking about fundamental needs and overall satisfaction User when using applications or services Customer when selecting and buying products and services Group member when interacting with others Quality of Experience is related to all roles User Customer Person Application Group member Network QoS QoE MOS Mean Opinion Score Vendor / Provider Marketing

8© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Dictionary approach Dictionary Experience: the totality of a person's perceptions, feelings, and memories Quality: the basic character or nature of something  QoE: the basic nature of a person's perceptions, feelings, and memories – far-reaching definition, hard to apply in a concrete analysis Still, we may keep QoE as the most generic concept, and then define additional terms for each role From modeling viewpoint that is the most reasonable approach Person QoE

9© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / What are the sub-concepts? Quality of User Experience (QUE) Quality of Customer Experience (QCE) Quality of Group Experience (QGE)? Quality of xxx Experience (Q?E) Google hits Quality of Experience Quality of User Experience Quality of Customer Experience Quality of Group Experience 9 Quality of Service Mean Opinion Score User Customer Person Group member QoS QoE MOS Application Network QUE QCE QGE ?

10© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / An approach to model QUE Application User behavior User experience Motivation Perceived value User Pricing Service provider Person 36 €/month current cost per session (time * effort + money) 50 QUE measure

11© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / End-user model – improving quality Application User behavior User experience Motivation Perceived value User Pricing Service provider Person +19 € +10 sessions +30% The value of service is deemed 30% higher because of improved display properties

12© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / End-user model – improving usability Application User behavior User experience Motivation Perceived value User Pricing Service provider Person +12 € +16 sessions -0.2 €/session Service is easier to use (30 s/session)  cost of session is decreased by 0.2 €

13© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / End-user model – improving network Application User behavior User experience Motivation Perceived value User Pricing Service provider Person +4 € +6 sessions success rate +12% Service availability is improved from 80% to 90% Network

14© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Summary Quality of Experience Key concept when we try to understand what the Next Generation Internet is able to provide to end-users (to us!) Covers all relevant aspects that define how satisfied a person is with a service Shall be divided into sub- concepts (because analysis must be modular in any case) The ultimate objective is to assess how users and customers behave with new service e.g. provided by Next Generation Internet – A lot work is needed to accomplish this task User Customer Person Group member QoS QoE MOS Application Network QUE QCE QGE ?

15© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Comments or questions?

16© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Abstract Quality of Service (QoS) was the Term ten years ago. Particularly QoS was the answer to the question why should telecom operators and providers rely on ATM rather than on best effort Internet. Obviously, a network with QoS was able to provide better service than a network without QoS. However, this QoS viewpoint was pretty technical, while end-users are not usually at all bothered about technical performance. What they really care about is the experience they are able to obtain, and Internet provided, even without any QoS mechanisms, a lot of new experiences, like web-browsing, and search engines. Thus if and when we want to make realistic analysis what network technologies and services will succeed, we must consider very seriously what are the real issues that define the real behavior of end-users. Quality of Experience (QoE, sometimes QoX) is a key term in this task. QoE is a subjective measure from the user's perspective of the overall value of the service provided. QoS is just one, often minor ingredient of Quality of Experience analysis. Other essential issues include the expectations of the end-users, the content of the service, the importance of service for the end-user, the characteristics of the device, and the price of the service. The challenging task of QoE research is to create a framework that is able combine these diverse aspects under a common umbrella in a way that we are able to somehow predict the behavior of end users when new services are offered to them. The talk will discuss about how this task can be carried out. Kalevi Kilkki Principal Scientist Nokia Siemens Networks kalevi.kilkki(at)nsn.com Selected publications available at

17© Nokia Siemens Networks Workshop on Socio-Economic Issues of NGI / Kilkki / Quality of Experience in some papers (IEEE) Heddaya, A.S., An economically scalable Internet, Computer, Vol. 35, Issue 9, Sept. 2002, p. 93 – 95 For example, downloading a two-hour movie to a home entertainment system or PC for later viewing can yield high QoE for the movie-viewing experience, even if the available bandwidth is of low quality in terms of packet loss, packet delay, and bandwidth variation. Pereira, F., A Triple User Characterization Model for Video Adaptation and Quality of Experience Evaluation, 7th Workshop on Multimedia Signal Processing, Oct The notion of 'quality of service' has to evolve to something more encompassing like 'quality of experience' where user satisfaction considers not only the sensorial and perceptual dimensions but also the important emotional dimension. Liu Li-yuan; Zhou Wen-an; Song Jun-de, The Research of Quality of Experience Evaluation Method in Pervasive Computing Environment. 1st International Symposium on Pervasive Computing and Applications, 3-5 Aug. 2006, p The function of quality of experience (QoE) evaluation includes two aspects: to monitor the experience of user on-line, then to control and justify the service based on the QoE to ensure that the quality of service can highly meet the requirements of the user. Zapater, M.N. et al., A Proposed Approach for Quality of Experience Assurance of IPTV, 1st Int. Conf. on the Digital Society, Jan There is a need to ensure quality on an end-to-end basis and at a higher abstraction layer by combining QoS and QoE in one integrated framework. Lopez, D.; Gonzalez, F.; Bellido, L.; Alonso, A., Adaptive multimedia streaming over IP based on customer oriented metrics, International Symposium on Computer Networks, June 2006, p Quality of experience (QoE) has been defined as an extension of the traditional quality of service (QoS) in the sense that QoE provides information regarding the delivered services from an end-user point of view. Siller, M.; Woods, J.C., QoS arbitration for improving the QoE in multimedia transmission, Int. Conf. on Visual Information Engineering, 7-9 July 2003 p This contribution presents a Quality of Experience (QoE) framework. It evaluates QoE using QoS metrics, network feedback, and dynamic user requirements. and proposes a definition for QoE. In (3) QoE is considered as all the perception elements of the network and performance relative to expectations of the users/ subscribers. Patrick, A.S. et al, A QoE sensitive architecture for advanced collaborative environments, First International Conference on Quality of Service in Heterogeneous Wired/Wireless Networks, 2004, p Quality of Experience can be defined as the characteristics of the sensations, perceptions, and opinions of people as they interact with their environments.