Quality Is in the Eye of the Beholder: Meeting Users ’ Requirements for Internet Quality of Service Anna Bouch, Allan Kuchinsky, Nina Bhatti HP Labs Technical Reports APRIL 2000, CHI 2000 Conference Baekcheol Jang SA. Lab. Cs. Dept. KAIST

Issue With increasing usage of Internet services, the topic of providing adequate QoS for the internet has become a focus of research. Traditional QoS metrics such as response time and delay no longer suffice to fully describe quality of service as perceived by users. => New QoS metrics are needed.

Problem Investigate Users ’ Requirements for Internet Quality of Service. –Objective System Quality: The majority of research on QoS is systems oriented, focusing on traffic analysis, scheduling, and routing. –User level QoS It is not yet known how objective system quality relates to users ’ subjective perceptions of quality.

Contents Present the results of quantitative experiments that establish a mapping between objective and perceived QoS in the context of Internet commerce. How do contextual factors influence users ’ perception of QoS ? It is possible to correlate objective measures of QoS with subjective judgements made by users, and therefore influence system design. The utility of future Internet can be maximized by integrating users ’ requirements for QoS into system design.

Strong Points Identify that relationship exists between objective and subjective QoS. Indicate the need for technology to meet user requirements. The importance of user requirement. Present an example that integrates users ’ requirements for QoS into system design.

Week Points Presented information is not enough to be applied in real world. Practical? It is so complex. There is not any measurement that can prove their approach. Conceptual model? –What underlying conceptual models influence users ’ judgements of QoS?

New ideas of extension related to the problem or in a new problem domain. Refer to further research. Developer is not user. Such investigation activity is applied to all case.

Research Question(1/2) To what extent is there a mapping between objective and subjective QoS? –This question was posed to investigate the level of objective QoS that was considered acceptable to users. What contextual factors influence users ’ tolerance of Internet QoS? –Our perspective challenges the assumption that there is a strict correlation between objective levels of quality received by users and their perceptions of that quality. –We investigate the factors that influence users ’ perception of QoS by asking if users ’ tolerance for QoS.

Research Question(2/2) What underlying conceptual models influence users ’ jedgements of QoS? –In addition to investigating contextual factors, we wanted to understand the reasons behind the influence of such factors on users ’ judgements of QoS. –This question, therefore, was designed to investigate how users ’ conceptual models related to their perceptions of QoS.

Method: Research approach Our research approach combined the gathering of quantitative and qualitative data. –Qualitative data: in order to address how contextual factors that influence the definition of thresholds relate to users ’ conceptual models. We conducted experimental work to provide information on tolerance thresholds.

Method: Participants 30 male participants, aged between 18 and 68, in the study: appropriately homogenous group of users was selected. The following criteria were applied when selecting participants. –Use the Internet for at least 2 hours per week. –Have made at least 2 purchases on the Internet in the last year. –Have at least an intermediate level of self- assessed skill with using computers.

Method: Task Purchase a home computer system using the HP shopping Village Web site. Purchase each component of the computer system separately. Participants access 22 Web pages during the task. A set pattern of actions were repeated through the task. –View a class of similar products. –Select a specific product from a class of products. –Add the chosen product to their shopping cart. –View the contents of their shopping cart.

Method: Experimental Conditions Delay range: 2 – 73 seconds. Investigating latency: Non incremental loading –Investigate whether the latency influenced user perceptions of the delay of page delivery. –Classification of latency(condition 1) –Control of latency(condition 2): can be infered about users ’ tolerance from their behavior when they controlled the quality. Investigating Incremental loading.(condition 3) –Investigate whether users would more tolerant of delay when Web pages loaded incrementally instead of all at once.

Figure 1,2 & UI

Table 1

Finding A mapping between objective QoS and users ’ subjective perceptions of that QoS can be identified and qualified. This mapping is influenced by a number of contextual factors including the type of task in which the user is engaged, the method of page loading, and cumulative time of interaction. Users ’ conceptual models underlie the influence of contextual factors on subjective perceptions of QoS.

To What extent is there a mapping between objective and subjective QoS? Classification of Latency.(C1) –Table 2, shows this classification. Control of Latency.(C2) –Average tolerance: 8.57sec –Standard deviation: 5.87sec Classification of Latency.(C3) –Table 2, Be almost 6 times higher.

Figure 4, Table 2

What contextual factors influence users ’ tolerance of internet QoS ? Influenced by their expectations of delay. The amount of time users allocate to the task. Understanding system-level operations. The company whose products are advertised. Duration of interaction. –Users ’ tolerance for delay decreased as the length of time they spent interacting with the system increased. –By figure 6: show the maximum delay tolerated by a participant in condition 2. Task Variation.(By figure 5)

Figure 5

Figure 6

What underlying conceptual models influence users ’ judgements of QoS ? Qualitative data showed that participants possess a conceptual model of the way that networks store and access information. This conception influenced their tolerance for delay. Our result show that users ’ conceptual models of the way in which networks operate can significantly influence their tolerance of QoS in predictable way. Consequently, an understanding of users ’ conceptual models, and, perhaps more importantly, the behavior which is driven by them Is a crucial step in accommodating user demand.

Implication for System Design(1) The perspective of this work is not only to understand user behavior relating to QoS but to interpret those findings into solutions for real-world problems. Server implement mechanisms that dynamically control the processing and delivery of information in response to users ’ request. Use scheduling algorithm. Prioritize request.

Implication for System Design(2) Our results have shown that there are contextual factors that influence user ’ s tolerance for latency. These factors can be used in prioritizing requests in the server. A central finding in our study was that users ’ tolerance for latency decreases over the duration of interaction with a Web site. This phenomenon can be considered when performing server scheduling optimizations.

Implications for Further HCI Research Further work is needed to establish this trend for other patterns of latency where the magnitude of delay is more precisely controlled. Further work is needed to establish the effect of incremental loading between conditions with an identical range of latency. Further studies of users ’ perceptions of QoS should investigate the validity of our findings in different domains, such as an entertainment Web site.

Implications for Further HCI Research Further work is needed to investigate the influence of the other contextual factors reported in this paper on users ’ perceptions of real time QoS. The combination of results from different domains and for different applications, would make it possible to create generalized conceptual models for predicting how tolerance changes according to a number of contextual factors.

Conclusions(1/2) Investigate users ’ requirements for Internet QoS. The task in which users are engaged, the length of time they have been interacting with a site, and the method of page loading affects the acceptability of QoS. Tolerance of delay is influenced by users ’ conceptual models of how the system works.

Conclusions(2/2) Poor Web site performance leads to poor company image and often compromises users ’ conceptions of the security of the site. It is possible to integrate users ’ requirements into systems design. Only through such integration will it be possible to achieve the customer satisfaction that leads to the success of any commercial system.