1. Working with Preferences Ronen Brafman Computer Science Department Stanford University* * On leave from Ben-Gurion University

2. Goal: Provide tools and build systems that are able to make, or able to help users make, good choices

3. Applications of Interest optimal product/item selection optimal product/item selection optimal product configuration optimal product configuration guiding program choices guiding program choices personalization personalization We target lay users with no background in decision theory

4. Goal of this Talk : Provide an overview of an approach to this problem based on the xCP-net models and algorithms, and discuss some applications

5. Constrained Optimization Process Manufacturer Constraints Preference specification Feedback  Spec Revision Optimal automobile Updated Choice

6. The Key Ideas Preference elicitation: Preference elicitation: Use natural language like statements – make it simple Use natural language like statements – make it simple Support heterogeneous types of statements Support heterogeneous types of statements Make scaling up to quantitative assessment possible Make scaling up to quantitative assessment possible Calculating optimal choices: Calculating optimal choices: Efficient methods for ordering sets Efficient methods for ordering sets Algorithms for constrained optimization Algorithms for constrained optimization Efficient data-driven incremental elicitation Efficient data-driven incremental elicitation

7. Elicitation and Modeling

8. Natural Preferences Statements Value preferences: Value preferences: Strict: I prefer isle to window seats Strict: I prefer isle to window seats Conditional: I prefer isle to window in economy class Conditional: I prefer isle to window in economy class Attribute importance: Attribute importance: Strict: Seat assignment is more important than airline Strict: Seat assignment is more important than airline Conditional: Seat assignment is less important than airline in domestic flights Conditional: Seat assignment is less important than airline in domestic flights

9. Ceteris Paribus Semantics Ceteris Paribus = all else being equal Ceteris Paribus = all else being equal “I prefer isle to window seats”  “I prefer isle to window seats”  Given two flights that differ only in seat assignment, I prefer the one with an isle seat. Given two flights that differ only in seat assignment, I prefer the one with an isle seat. Otherwise, can’t infer a preference Otherwise, can’t infer a preference “Seat assignment is less important than airline in domestic flights”  “Seat assignment is less important than airline in domestic flights”  Given two domestic flights that differ in seat assignment and airline only, I prefer the one with a better seat assignment Given two domestic flights that differ in seat assignment and airline only, I prefer the one with a better seat assignment Says nothing about two international flights Says nothing about two international flights

10. CP-nets (Boutilier, Brafman, Hoos, Poole, UAI ‘99) A qualitative, graphical model of preferences, that captures and organizes statements of conditional preferential independence. Each node represents a domain variable. Parents(v) are those variables that affect user’s preference over the values of v Parents(class) = {airline} Conditional preference table (CPT) associated with every node in the CP-net Provides an ordering over the values of the node for every possible parent context

11. A C DE F B Example of a CP-net

12. Any acyclic CP-net defines a (consistent) partial order over the outcome space. A C B Semantics and Consistency worst best

13. A C B worst best Importance Relations

14. Day of the flight Departure Time Stop-overs Class Airline More Complex Example

15. Day of the flight Departure Time Stop-over Class Airline

16. Day of the flight Departure Time Stop-oversClass Airline

17. A B CD E nodes  variables cp-arcs (directed) i-arcs (directed) ci-arcs (undirected) cp-tables ci-tables  

18. Role of Graphical Structure Users need not be aware of the underlying graphical structure Users need not be aware of the underlying graphical structure User employs statement templates User employs statement templates System constructs the network on the fly System constructs the network on the fly Graphical structure important for analysis and algorithms Graphical structure important for analysis and algorithms

19. UCP-Nets [Boutilier, Bacchus, & Brafman] Quantified CPT Quantified CPT Simplified semantics: sum of utility factors; U(abcd)= Simplified semantics: sum of utility factors; U(abcd)= f A (a)+f B (b)+f C (abc)+f D (cd) = f A (a)+f B (b)+f C (abc)+f D (cd) = 5 + 4 +.2 +.9 = 10.1 5 + 4 +.2 +.9 = 10.1 Linear utility computation Linear utility computation AB C D a > a 5 2 b > b 4 3 a b.6.1 a b.2.8 a b.3.8 a b.9.3 c.9.8 c.2.3 c d

20. UCP-nets Capture Heterogeneous Data Diverse statement compile into UCP-net: Diverse statement compile into UCP-net: Independence information maintained by graph Independence information maintained by graph Value preferences and variables preferences Value preferences and variables preferences Two-way comparisons of complete outcomes Two-way comparisons of complete outcomes Quantitative information Quantitative information Simple compilation based on linear constraints Simple compilation based on linear constraints Good for incremental refinement: Good for incremental refinement: Start with qualitative model – no cold start! Start with qualitative model – no cold start! Refine with user feedback/additional observations Refine with user feedback/additional observations

21. Using (T)CP-nets Algorithms 1. Ordering sets/selecting top-element 2. Preference-based constrained optimization

22. 1. Ordering Elements Need: Order (the many) results of a query Need: Order (the many) results of a query Naïve solution: pairwise comparison -- o>o’ ? Naïve solution: pairwise comparison -- o>o’ ? Problem: NP-hard in general Problem: NP-hard in general Alternative solution: linearize the partial order Alternative solution: linearize the partial order Insight: some linearizations are easy to obtain Insight: some linearizations are easy to obtain Key: Key: 1. One of o e o’ or o’ e o can be decided quickly 2. Answers used to generate consistent ordering

23. 2a. Preferential Optimization Finding the preferentially optimal outcome for an acyclic network is straightforward! A C DE F B A C DE F B

24. 2b. Preference-Based Constrained Optimization (PCO) Given: Given: user preferences user preferences an implicitly specified set of feasible options an implicitly specified set of feasible options Find one/all/k optimal, feasible element(s) Find one/all/k optimal, feasible element(s) Applications: Applications: Product configuration (PC, vacation) Product configuration (PC, vacation) Optimal plan selection Optimal plan selection Content/display adaptation and personalization Content/display adaptation and personalization

25. Solving PCO: Ordered Generate & Test Generate outcomes in non-increasing order Generate outcomes in non-increasing order = linearize the partial order over all possible elements Test for feasibility Test for feasibility Check for optimality: Check for optimality: First feasible outcome is optimal! First feasible outcome is optimal! Need more? Need more? Maintain set of optimal solutions Maintain set of optimal solutions New solution is optimal is not dominated by any previously generated solution New solution is optimal is not dominated by any previously generated solution

26. Generating a Non-increasing Sequence of Outcomes Topologically sort the variables Topologically sort the variables Build an assignment (search) tree by instantiating variables according to this order Build an assignment (search) tree by instantiating variables according to this order Order variable values based on the CPT Order variable values based on the CPT Leaf nodes, ordered left to right (= depth-first traversal of the tree) correspond to a non- increasing sequence of outcomes Leaf nodes, ordered left to right (= depth-first traversal of the tree) correspond to a non- increasing sequence of outcomes

27. C B 1 2 3 A

28. Improvements Use CSP pruning techniques Use CSP pruning techniques Meta-constrains on CSP solution Meta-constrains on CSP solution Branch and bound – eliminate sub-tree when: Branch and bound – eliminate sub-tree when: We assign a variable to a less preferred value We assign a variable to a less preferred value Current set of constraints as strong as for some previous value of this variable Current set of constraints as strong as for some previous value of this variable

30. Anytime Behavior First feasible solution is optimal! First feasible solution is optimal! No theoretical overhead beyond standard CSP solution No theoretical overhead beyond standard CSP solution No item withdrawn from set of current solutions No item withdrawn from set of current solutions To obtain more than one solution dominance testing required To obtain more than one solution dominance testing required Can lead to considerable computational overhead Can lead to considerable computational overhead

31. Applications

32. Flight Selection (Brafman, Domshlak, Kogan UAI’04) Instance of “selecting optimal element” problem Instance of “selecting optimal element” problem Approach: Approach: User supplies User supplies constraints (source, destination, etc.) constraints (source, destination, etc.) initial preferences initial preferences Preferences compiled into UCP-net Preferences compiled into UCP-net Top k results shown Top k results shown User provides feedback User provides feedback What she likes or not What she likes or not What she would like improved What she would like improved Top results revised accordingly Top results revised accordingly

33. Planning for Data Products (Golden et. al.) NASA collects much raw data about earth NASA collects much raw data about earth Requires extensive processing to be useful Requires extensive processing to be useful Each Earth scientists needs different processing Each Earth scientists needs different processing ImageBot generates data plans for such products ImageBot generates data plans for such products Data plans run scientific models, combine and transform data in order to achieve data goals Data plans run scientific models, combine and transform data in order to achieve data goals Many plans can generate one data product Many plans can generate one data product Each plan has different value Each plan has different value Using a simple preference language ImageBot planning algorithm is biased towards more preferred plans Using a simple preference language ImageBot planning algorithm is biased towards more preferred plans

34. Adaptive, Personalized Rich Media Presentations Presentations with diverse media elements requiring spatio-temporal synchronization Presentations with diverse media elements requiring spatio-temporal synchronization Target wide audiences Target wide audiences Audience members have different tastes, different network connections, and use different devices Audience members have different tastes, different network connections, and use different devices Goal: provide designers with tools for designing presentations that adapt to user and user context Goal: provide designers with tools for designing presentations that adapt to user and user context

35. Example: ESPN Promo Presentation with five elements Presentation with five elements Video featuring upcoming broadcast Video featuring upcoming broadcast 2 image ads 2 image ads Video ad Video ad Running text with scores/news Running text with scores/news Each media element is a variable Each media element is a variable Additional variables denote user properties, device properties, bandwidth Additional variables denote user properties, device properties, bandwidth Variable values = different content and quality options Variable values = different content and quality options

36. How Does It Work? Variables denote different presentation elements (video, ads, running text) Variables denote different presentation elements (video, ads, running text) Additional variables denote context information Additional variables denote context information TCP-nets specify preference relation over presentations TCP-nets specify preference relation over presentations Author may specify additional constraints Author may specify additional constraints At download time combine: At download time combine: TCP-net TCP-net Information about user and user context Information about user and user context Resulting constrained optimization problem solved Resulting constrained optimization problem solved Output a SMIL presentation Output a SMIL presentation

37. GenderNationality CP-Net for ESPN Promo Ad1Sports VideoTextAd2  

38. Current Issues and Projects Set preferences – specifying and computing preferred subsets when items have synergy [AAAI’06] Set preferences – specifying and computing preferred subsets when items have synergy [AAAI’06] Continuous monitoring Continuous monitoring An application that requires set preferences An application that requires set preferences Browsing large heterogeneous databases Browsing large heterogeneous databases Joint work with Scott Klemer, Ron Yeh, and Yoav Shoham supported by NSF Joint work with Scott Klemer, Ron Yeh, and Yoav Shoham supported by NSF

39. Collaborators Carmel Domshlak – Technion Carmel Domshlak – Technion All work All work Craig Boutilier – University of Toronto Craig Boutilier – University of Toronto CP-nets, UCP-nets CP-nets, UCP-nets Holger Hoos, David Poole – UBC Holger Hoos, David Poole – UBC CP-nets CP-nets Doron Friedman – University College London Doron Friedman – University College London Adaptive presentations Adaptive presentations Solomon Shimony – Ben-Gurion University Solomon Shimony – Ben-Gurion University Adaptive presentation, TCP-nets, Set preferences Adaptive presentation, TCP-nets, Set preferences

40. Summary Much recent interest in work on preference representation, elicitation, and reasoning Much recent interest in work on preference representation, elicitation, and reasoning xCP-nets offer a simple preference language and convenient graphical and algorithmic tools xCP-nets offer a simple preference language and convenient graphical and algorithmic tools UCP-nets provide good target for knowledge compilation UCP-nets provide good target for knowledge compilation Many applications in e-commerce and user interfaces Many applications in e-commerce and user interfaces

41. Compare > If dominated …     Otherwise …