1 Automated Discovery of Recommendation Knowledge David McSherry School of Computing and Information Engineering University of Ulster +

2 Overview  Approaches to retrieval in recommender systems  Rule-based retrieval (of cases) in Rubric  Automating the discovery of recommendation rules  Role of default preferences in rule discovery  Related work  Conclusions +

3 The Recommendation Challenge Often we expect salespersons to make reliable recommendations based on limited information: ☺ I’m looking for a 3-bedroom detached property To recommend an item with confidence, a salesperson has to consider: The customer’s known preferences The available alternatives All features of the recommended item  including features not mentioned by the customer

4 Are Recommender Systems Reliable? Features not mentioned in the user’s query are typically ignored in: Nearest neighbour (NN) retrieval Decision tree approaches Multi-criterion decision making  Assumed (or default) preferences are sometimes used for attributes like price  But for many attributes, no assumptions can be made about the user’s preferences

5..., reasonably priced,..., Preferences Pyramid Known preferences beds = 3 type = detached..., location = A,..., Unknown preferences Default preferences

6 CBR Recommender Systems  Descriptions of available products (e.g. houses) are stored as cases in a product dataset e.g. Loc Beds Type Weight: (3) (2) (1) Case 1: A 3 semi Case 2: B 4 det Case 3: B 3 det and retrieved in response to user queries

7 Inductive Retrieval Bedrooms?Case 2 (B, 4, det) 4 Type? 3 Case 3 (B, 3, det) det Case 1 (A, 3, semi) semi Not only are the user’s unknown preferences ignored - the user is prevented from expressing them

8 Inductive Retrieval Bedrooms?Case 2 (B, 4, det) 4 Type? 3 Case 3 (B, 3, det) det Case 1 (A, 3, semi) semi The recommended case exactly matches the user’s known preferences - but what if she prefers location A?

9 The standard CBR approach is to recommend the most similar case  The similarity of a case C to a query Q over a subset A Q of the product attributes A is: where w a is the weight assigned to a Nearest Neighbour Retrieval

10 Incomplete Queries in NN Loc Beds Type (3) (2) (1) Q : 3det Sim Case 1: A 3semi 2 Case 2: B 4 det 1 Case 3: B 3 det3 most-similar(Q) = {Case 3}

11 Incomplete Queries in NN Loc Beds Type (3) (2) (1) Q : 3det Sim Case 1: A 3semi 2 Case 2: B 4 det 1 Case 3: B 3 det3 most-similar(Q) = {Case 3}  Again, Case 3 is a good recommendation if the user happens to prefer location B

12 Incomplete Queries in NN Loc Beds Type (3) (2) (1) Q* : A3det Sim Case 1: A 3semi 5 Case 2: B 4 det 1 Case 3: B 3 det3 most-similar(Q*) = {Case 1}  But not if she prefers location A

13 Rule-Based Retrieval in Rubric  In rule-based retrieval, a possible recommendation rule for Case 3 might be: Rule 1: if beds = 3 and type = det then Case 3  Given a target query, a product dataset, and a set of recommendation rules, Rubric:  Retrieves the case recommended by the first rule that covers the target query  If none of the available rules covers the target query, it abstains from making a recommendation

14  For any case C and query Q, we say that Q → C is a dominance rule if: most-similar(Q*) = {C} for all extensions Q* of Q  As Rule 1 is not a dominance rule for Case 3, it is potentially unreliable: Rule 1: if beds = 3 and type = det then Case 3 Dominance Rules

15 A Dominance Rule for Case 3 Loc Beds Type (3) (2) (1) Q :B3 Sim Case 1: A 3semi 2 Case 2: B 4 det 3 Case 3: B 3 det5 most-similar(Q) = {Case 3}

16 A Dominance Rule for Case 3 Loc Beds Type (3) (2) (1) Q :B3 Sim Max Case 1: A 3semi 2 3 Case 2: B 4 det 34 Case 3: B 3 det5  As Cases 1 and 2 can never equal the similarity of Case 3, a dominance rule for Case 3 is: Rule 2: if loc = B and beds = 3 then Case 3

17 Coverage of a Dominance Rule  A dominance rule Q → C can be applied to any query Q* such that Q  Q* since by definition: most-similar(Q*) = {C}  Also by definition, most-similar(Q**) = {C} for any extension Q** of Q*  So no other case can equal the similarity of C regardless of the user’s unknown preferences

18 The Role of Case Dominance  A given case C 1 dominates another case C 2 with respect to a query Q if: Sim(C 1, Q*) > Sim(C 2, Q*) for all extensions Q* of Q (McSherry, IJCAI-03)  So Q → C is a dominance rule if and only if C dominates all other cases with respect to Q  This is not the same as Pareto dominance

19 Identifying Dominated Cases  A given case C 1 dominates another case C 2 with respect to a query Q if and only if: (McSherry, IJCAI-03)  Cases dominated by a given case can thus be identified with modest computational effort

20 Dominance Rule Discovery (McSherry & Stretch, IJCAI-05) Our algorithm targets maximally general dominance rules Q → C such that Q  description(C) B, 3, det B 3 det B, 3 B, det 3, det nil Case 3 dominates Case 1 and Case 2 with respect to this query Description of Case 3

21 Complexity of Rule Discovery Our discovery algorithm is applied with each case in turn as the target case For a product dataset with n cases and k attributes, where n  2 k, the worst-case complexity is: O(k  n 2  2 k ) If n < 2 k, the worst-case complexity is: O(k  n  2 2k )

22 In a dataset with k attributes, the number of rules discovered for a target case can never be more than k C  k/2  (McSherry & Stretch, IJCAI-05) With 1,000 products and 9 attributes, the maximum number of discovered rules is 126,000 Rule-set sizes tend to be much smaller in practice Maximum Rule-Set Size No. of Attributes: Maximum:

23 Digital Camera Case Base Source:McCarthy et al. (IUI-2005) No of cases: 210 Attributes:make (9), price (8), style (7), resolution (6), optical zoom (5), digital zoom (1), weight (4), storage type (2), memory (3) Discovered Rule: if make = toshiba and style = ultra compact and optical zoom = 3 then Case 201

24 Discovered Rule-Set Sizes Digital Camera Case Base (k = 9)

25 Lengths of Discovered Rules Digital Camera Case Base (k = 9)

26 Limitations of Discovered Rules Example Rule if make = sony and price = 336 and style = compact and resolution = 5 and weight = 236 then Case 29 Problem Exact numeric values (e.g., price, weight) make the rule seem unnatural/unrealistic They also limit its coverage Solution Assume the preferred price and weight are the same for all users

27 LIB and MIB Attributes  A less-is-better (LIB) attribute is one that most users would prefer to minimise e.g. price, weight  A more-is-better (MIB) attribute is one that most users would prefer to maximise e.g. resolution, optical zoom, digital zoom, memory  Often in NN retrieval, LIB and MIB attributes are treated as nearer-is-better attributes: ☺ How much would you like to pay? 300

28 LIB and MIB Attributes  A less-is-better (LIB) attribute is one that most users would prefer to minimise e.g. price, weight  A more-is-better (MIB) attribute is one that most users would prefer to maximise e.g. resolution, optical zoom, digital zoom, memory  Often in NN retrieval, LIB and MIB attributes are treated as nearer-is-better attributes: ☺ How much would you like to pay? 300 This doesn’t make sense, as it implies that the user would prefer to pay 310 than 280

29 Role of Default Preferences in Rule Discovery (McSherry & Stretch, AI-2005)  We assume the preferred value of a LIB/MIB attribute is the lowest/highest value in the case base  These preferences are represented in a default query: Q D : price = 106, memory = 64, resolution = 14, optical zoom = 10, digital zoom = 8, weight = 100  In the dominance rules Q → C now targeted by our algorithm, Q includes the default preferences in Q D  Thus the assumed preferences are implicit in the discovered rules

30 Similarity to the Default Query  We use the standard measure for numeric attributes: where x is the value in a given case and y is the preferred value  For a LIB attribute:

31 Digital Camera Case Base No of cases: 210 Attributes:make, price, style, resolution, optical zoom, digital zoom, weight, storage type, memory LIB attributes:price, weight MIB attributes:resolution, optical, digital, memory Discovered Rule: if make = sony and style = compact then Case 29

32 Q D  {sony, compact, memory stick} Q D  {sony, compact} Q D  {sony, memory stick} Q D  {compact, memory stick} Q D  {sony} Q D  {compact} Q D  {memory stick} Q D Reduced Complexity of Rule Discovery (e.g., from 512 candidate queries to 8) Dominance Rule Discovery for Case 29

33 Reduced Length of Discovered Rules DPs = Default Preferences

34 Recommendability of Cases  Only 56 of the 210 cases can be the most similar case for any query that includes the default query Q D  The reason is that most cases are dominated with respect to the default query  For most of the 56 non-dominated cases, only a single dominance rule was discovered  The discovered rules cover 29% of all queries over the attributes make, style, and storage type

35 Retrieving Stories for Case-Based Teaching (Burke & Kass, 1996)  Rule-based retrieval of stories or lessons learned by experienced salespersons  Retrieval is conservative, opportunistic, and non- mandatory  A story is retrieved at the system’s initiative and only if highly relevant  By design, retrieval in Rubric is also conservative and non-mandatory (and potentially opportunistic)  Easily combined with NN retrieval of a less strongly recommended case if no rule covers a given query

36 Incremental Nearest Neighbour (iNN) (McSherry, IJCAI-03, AICS-05, AIR 2005)  A conversational CBR approach in which:  Question selection is goal driven (i.e., maximise number of cases dominated by a target case)  Dialogue continues until it can be safely terminated (i.e., no other case can exceed the similarity of the target case)  Relevance of any question can be explained (e.g., ability to confirm the target case)  Recommendations can be justified (i.e., unknown preferences cannot affect the outcome)

37 Demand Driven Discovery of Recommendation Knowledge in Top Case Top Case:What is the preferred make? User:sony Top Case:The target case is: Case 40: sony, 455, ultra compact, 5, 4, 4, 298, MS, 32 What is the preferred style? User:why Top Case:Because if style = ultra compact this will confirm Case 40 as the recommended case What is the preferred style? User:compact Top Case:The recommended case is: Case 29: sony, 336, compact, 5, 3, 4, 236, MS, 32

38 Conclusions  Benefits of retrieval based on dominance rules:  Provably reliable because account is taken of the user’s unknown preferences  Benefits of default preferences:  An often dramatic reduction in average length of the discovered rules  Increased coverage of queries representing the user’s personal preferences  Reduced complexity of rule discovery

39 References Burke, R. and Kass, A. (1996) Retrieving Stories for Case-Based Teaching. In Leake, D. (ed.) Case-Based Reasoning: Experiences, Lessons & Future Directions. Cambridge, MA: AAAI Press, McCarthy, K., Reilly, J., McGinty, L. and Smyth, B. (2005) Experiments in Dynamic Critiquing. Proceedings of the International Conference on Intelligent User Interfaces, McSherry, D. (2003) Increasing Dialogue Efficiency in Case-Based Reasoning without Loss of Solution Quality. Proceedings of the 18 th International Joint Conference on Artificial Intelligence, McSherry, D. (2005) Explanation in Recommender Systems. Artificial Intelligence Review 24 (2) McSherry, D. (2005) Incremental Nearest Neighbour with Default Preferences. Proceedings of the 16 th Irish Conference on Artificial Intelligence and Cognitive Science, 9-18 McSherry, D. and Stretch, C. (2005) Automating the Discovery of Recommendation Knowledge. Proceedings of the 19 th International Joint Conference on Artificial Intelligence, 9-14 McSherry, D. and Stretch, C. (2005) Recommendation Knowledge Discovery. Proceedings of the 25 th SGAI International Conference on Innovative Techniques and Applications of Artificial Intelligence

40 Acknowledgements  Thanks to:  Eugene Freuder, Barry O’Sullivan, Derek Bridge, Eleanor O’Hanlon (4C)  Chris Stretch (co-author, IJCAI-05 and AI-2005)  Kevin McCarthy, Lorraine McGinty, James Reilly, Barry Smyth (UCD) for the digital camera case base

41 Compromise-Driven Retrieval (McSherry, ICCBR-03, UKCBR-05)  Similarity and compromise (unsatisfied constraints) play complementary roles  Queries can include upper/lower limits for LIB/MIB attributes (used only in assessment of compromise)  Every case in the product data set is covered by one of the recommended cases  That is, one of the recommended cases is at least as similar and involves the same or fewer compromises