1. Decision-Making in Recommender Systems: The Role of User's Goals and Bounded Resources Paolo Cremonesi Antonio Donatacci Franca Garzotto Robert Turrin Human Decision Making in Recommender Systems (Decisions@ RecSys’ 12), Sep 2012 Aybike Avsaroglu

2. 2 OUTLINE Introduction Background Information Recommender Systems Problem Definition Methodology Experimental Results Conclusion

3. 3 INTRODUCTION Discuss some aspects of user experience with Recommendation Systems which may affect the decision making process of the user The nature of user goals The dynamic characteristics of the resources space (e.g., availability during the research process) Hotel-booking (venere.com) application domain

4. 4 What is Recommender System?

5. 5 Recommender Systems help users Search large amounts of digital contents Identify the items more effectively Tools that help people making decisions from a vast set of alternatives Recommender Systems are software agents that elicit the interests and preferences of individual consumers […] and make recommnedations accordingly. They have the potential to support and improve the quality of the decisions comsumers make while searching for and selecting products online. (B.Xiao, I.Benbasat. E-commerce product recommendation agents: use, characteristics, and impact, March 2007)

6. 6 Recommender Systems Different system designs/paradigms availibility of exploitable data implicit and explicit feedback of users (ratings or clicks) domain charactersitics

7. 7 Recommender Systems As a fuction: Given: User model (ratings, preferences, demographics, situational context etc.) Items (with or without description of item characteristics) Find: Relevance score, used for ranking Reduce information overload by estimating relevance.

8. 8 Recommender Systems Attribute-based recommendation: based on syntactic attributes of products (science fiction books, romance films etc.) Item-to-item correlation (shopping basket recommendation) User-to-user correlation (finding users with similar tastes) Non-personalized recommendation (as in traditional stores: dish of the day, generic book recommendation)

9. 9 Recommendation Technologies Collaborative Filtering: Produce recommendations by computing the similarity between a user’s preferences and the preferences of other people do not attempt to analyse or understand the content of the items being recommended able to suggest new items to user who have similar preferences with others large group of people’s preferences are registered subgroup of people is located whose preferences are similar of the user who seeks the recommendation concept of similarity needs to be defined

10. 10 Recommendation Technologies Example: the system needs to make recommendations to customer C Customer B is very close to C (he has bought all the books C has bought). Book 5 is highly recommended Customer D is somewhat close. Book 6 is recommended to a lower extent Customers A and E are not similar at all. Weight=0

11. 11 Recommendation Technologies Collaborative: "Tell me what's popular among my peers"

12. 12 Recommendation Technologies Content-based: Recommend an item to a user, based upon a description of the item and a profile of the user’s interests Item Repreresentation: Structured: small number of attr., each item is described by the same set of attr. User profiles contains ratings for the items or user's interaction history IDNameCuisineServiceCost 1001Mike’s PizzaItalianCounterLow 1002Chris’s CaféFrenchTableMedium 1003Jacques BistroFrenchTableHigh

13. 13 Recommendation Technologies Each user is assumed to act independently, and the system requires a profile of the user’s needs or preferences The user has to provide information on her personal interests on starting to use the system, for the profile to be built The profile includes information about the items of interest, i.e. movies, books, CDs etc. Content-based filtering techniques try to identify similar items which are returned as recommendations They do not depend on having other users in the system

14. 14 Recommendation Technologies Content-based: "Show me more of the same what I've liked"

15. 15 Recommendation Technologies Hybrid: The underlying idea is that, the content is also taken into account when attempting to identify similar users for collaborative recommendations A number of systems have been developed: Fab, Tango, GroupLens’ approach etc.

16. 16 Recommendation Technologies Hybrid: combinations of various inputs and/or composition of different mechanism

17. 17 PROBLEM DEFINITION Explore decision making processes in the wide application domain of online services, specifically hotel booking Investigate some “subjective„ aspects of user experience with RSs, and some “objective„ attributes of RSs. Users' goals Design related attributes Availability of items both along the time in general and during the search process

18. 18 User Goals and Bounded Resources Scenario 1. Milan, to work with business partners from August 6 to August 10, 2012, reserve a room in a hotel in Milan for that week. Scenario 2. holiday in Milan from September 19 to September 25, 2012, reserve a room. Scenario 3. a business meeting in Milan from September 19 to September 20, 2012, reserve a room in a hotel in Milan, for one night Scenario 4. a holiday in Central Italy in mid September 2012, and will visit Rome for few days, a hotel in that period.

19. 19 User Goals and Bounded Resources

20. 20 User Goals and Bounded Resources Similar operational tasks Significant differences that may influence decision making process Different nature of user's goals Dynamic nature of services In scenarios 1,2, and 3 user's goals are sharp, and user's preferences are well defined In scenario 4, user has less strict preferences and dates are flexible (soft goals, open-ended needs) Open-ended needs are progressively elaborated during the decision process

21. 21 User Goals and Bounded Resources Differences related to dynamic, time dependent characteristic of items (availability) Scenario1 : very vast set of stable alternatives, second week of August most people and companies are on holiday Scenario2,3, and 4: limited resouces Scenario2: at the same week with Milan Fashion Week, most hotels are booked one year in advance for that event, we can expect that no hotel is available Scenario3: Same as scenario2, only for the first night of Milan Fashion Week, requirement is less demanding, there might be rooms for one night Scenario4: prefered time frame for reservation, high season in Rome, however dates are flexible

22. 22 Decision Making Process In this context, decision making process are typically modeled as “bounded rationality phenomena„ Bounded rationality: In complex decision making environments, individuals are often unable to evaluate all available alternatives due to the cognitive limitations of their minds Satisfactory only after having greatly simplified the set of available choices Cognitive effort can be reduced with a multiple stage decision making process

23. 23 Decision Making Process COMPLETE SOLUTION SET / SEARCH SET INDEPTH COMPARISION SET CONSIDERATION SETCOMMIT CHOICE

24. 24 Decision Making Process

25. 25 Challenges for RS Design and Evaluation Decision process in RSs is influenced by the characteristics of both users' goal and the resources meeting users' needs and preferences The initial step of decision process is kind of 'sense making' activity which is a focused 'search' Understand complexity of the domain The characteristics of the items Preliminary phase in which the progressive elaboration of alternatives, and transforming a soft goal to a sharp goal are occured. Providing an integrated set of interactive design strategies (search) with existing RS design strategies. Serendipity can be an important goal

26. 26 Challenges for RS Design and Evaluation Promoting crucial contents the existance of which users did not notice, so that users can stumble and get interested in them. Unbounded resources condition (scenario1) Typical RSs reduce users' decision effort and time, hence improving the quality of the decision process Bounded resources condition (scenario2-3-4) Effectiveness of traditional RSs for decision making process opens to question

27. 27 Challenges for RS Design and Evaluation RSs attempt to reduce the user decision effort so create the so called 'filter bubble' effect. (Eli Pariser) Isolating the user in a bubble that tends to exclude items that may be helpful for users' goal (serendipitous items), to show information that agrees with users' past viewpoints The intersection between the bubble and the available item set could be empty Iterating the decision process which increases users' decision effort and time, decreases the quality of the decision process

28. 28 Solution Users must be exposed seredipitous recommendations Paradigmatic shift is needed for the role of RSs in the decision process From narrowing the search set and consideration set in the case of unbounded resources, to expanding the in- depth set in the case of bounded resources Design Strategies: Support to decision making process that are strongly iterative (doing and re-doing prev. steps) Keep user engaged with the decision process Act both as filter that limits the set of valuable alternatives, and as multiplier that helps the user expand her horizons by recommending serendipitous alternatives

29. 29 METODOLOGY Define a conceptual model that provides a more comprehensive framework than existing ones, takes into account Characteristics of the goals (sharp/soft) Dynamic characteristics of items (availability) Modular architecture Easily customized to different datasets and different types of recommender algorithms Enable the researchers to manipulate and control different variables, in order to systematically assess the effects of RS use on users' decision making process

30. 30 Technical Work 6.000 accomodations 500.000 users' reviews Simulate high-season periods Recommendations from a library of 20 algorithms Hybrid recommendations can be provided User profiles are implicitly created by monitoring users' interaction with the objects Recommandations can be provided in different phases of the interaction process As alternatives when watching a page of an accomodation As a sorting option in a list of hotels

31. 31 Empirical Work An experimental setup that allows four different experimental conditions RS use conditions: asking the user to execute one between different tasks Bounded resources conditions: refers to configuration of the system, with or without RS support. Reconsider the concept of RS use in the research RS characteristics: refers to different algorithms Consumer decision processes: analyzing the user behavior under limited or unlimited items availability. Item availability can evolve with time (the longer is the decision process, the higher is the probability for the selected item to be unavailable or higher is the final price)

32. 32 Empirical Work Three key aspects of the bounded resources concept: Unavailability Time scarcity: resource may become unavailable as the time passes Price alteration: price may change depending on availability of resources

33. 33 Questionnaire 18 Questions : most of them are yes/no questions or multiple choices Examples: How much the proposed hotels match your personality? How long have you spent for booking the hotel? Are you satisfied with your final choice? Age, gender nationality, educational qualification, occupation for profiling

34. 34 RESULTS Two independent variables: RS use: with/without RS Resource availability: rooms available / shortage of rooms Four combination 15 subjects for each group (60 male aged between 24- 50) Each participant browsed the hotel catalog to search for a double room in Rome and to complete the simulated payment procedure booking the room for two nights. Then replied to a set of 18 questions related to the quality.

35. 35 RESULTS

36. 36 RESULTS Personalization: From Q6 : how much the proposed hotels match your personality? Only %10 of the users that did recieve recommendations percieved these recommendations as matching their personality Task execution time: Estimating users' effort by measuring the time required for the completion of the task Users recieving recommendations required significantly more time than users without recommandations

37. 37 RESULTS Consideration set: Users recieving recommendations explore a much larger consideration set This explains why their task execution time results are more than the users without recommendations Recommendations help users in exploring a larger number of alternatives especially in the scenario of bounded resources Percieved time: Even if users with recommendations required a significantly longer time to complete their task and explored a much larger number of hotels during their session, their percieved time is the same as the time percieved by users without recommendations

38. 38 CONCLUSION RSs do not reduce the time required to complete a decision making process Users' perception of the elapsed time is not related to the larger number of explored choices The effort of the decision making process increases in the case of bounded resources. RSs seem not able to alleviate this perceived effort RS gives larger consideration set for bounded resources case Unprofessional Very limited number of participants

39. 39 REFERENCES E. Pariser, The Filter Bubble: What the Internet Is Hiding from You. Penguin press, 2011 B. Xiao, I. Benbasat, E-Commerce Product Recommendation Agents: Use, Characteristics, and Impact, MIS Quarterly, 2007 D. Jannach, M. Zanker, A. Felfernig, G. Friedrich, Recommender Systems, Cambridge University Press M. Fasli, Agent Technology for e-Commerce N. George, D. Murugan, T. Tran, Content-based Recommendation Systems

40. 40 THANKS! Questions?