1. Resolving Underconstrained and Overconstrained Systems of Conjunctive Constraints for Service Requests Muhammed J. Al-Muhammed David W. Embley Brigham Young University Sponsored in part by NSF (#0083127 and #0414644)

2. 2 The Problem: Underconstrained “I want a dodge a 2000 or newer. The Mileage should be less than 80,000 and the price should not exceed $15,000” SolutionMakeModelPriceYearMileage S1DodgeStratus9,451.002004 35,808 S2DodgeStratus14,995.002005 1,694 S3DodgeStratus14,999.002005 27,543 S4DodgeStratus2,555.001997115,424 S5DodgeStratus6,900.002001 70,000 … … S168DodgeStratus9,975.00200334,060 www.cars.com, November 2005

3. 3 The Problem: Overconstrained “I want a dodge a 2000 or newer. The Mileage should be less than 80,000 and the price should not exceed $4,000.” Sorry No car matches your criteria. www.cars.com, November 2005

4. 4 Key Observations Some (near) solutions are better than others People specify constraints on some concepts in a domain more often than other concepts

5. 5 Fundamental Concepts: reward, penalty, and expectation A reward is a positive or zero real number given to a solution for satisfying a constraint A penalty is a negative real number given to a near solution for violating a constraint An expectation for a concept is the probability that people will specify constraint for the concept

6. 6 Fundamental Concepts: Pareto Optimality Based on dominance relations  The reward for S1 is as high as the reward for S2  For at least one reward S1 has a higher reward Dominated solutions are not Pareto optimal

7. 7 Too Many Solutions: Reward-Based Ordering Calculate rewards and combine them Order solutions, highest combined reward first Select the top-m Pareto optimal solutions  Discard non-Pareto optimal solutions from the reward ordering  Return the top-m for consideration

8. 8 Example “I want a dodge a 2000 or newer. The mileage should be less than 80,000 and the price should not be more than $15,000.” Solution Make Model Price Year Mileage --------------------------------------------------------------- S1 Dodge Stratus 13,999 2005 15,775 S2 Dodge Stratus 11,998 2004 23,404 S3 Dodge Stratus 14,200 2005 16,008 S4 Dodge Stratus 14,557 2005 16,954 S5 Dodge Stratus 10,590 2003 38,608 S1 better The sameS1 better

9. 9 Too Many Solutions: Expectation-Based Constraint Elicitation Associate expectations with domain concepts Order the concepts in a domain based on their expectations  Most expected first  Example: Make > Price > Model > … Elicit additional constraints over unconstrained concepts  Most expected first  If no preferred make provided, ask for Make; if no price, ask for Price; …

10. 10 No Solution: Penalty-Based Ordering Calculate penalties and combine them Order close solutions, lowest combined penalty first Select the top-m Pareto optimal near solutions  Discard dominated near solutions from penalty ordering  Return the top-m near solutions for consideration

11. 11 No Solution: Expectation-Based Constraint Relaxation Select the near solutions violating fewer constraints than a threshold Compute the relaxation cost: r si =  k e k C k (si). Suggest constraints of the near solution with the least r si for relaxation expectation penalty

12. 12 Example Near SolutionInsuranceDistanceTimeDate S1IHC161:00 PMthe 19th s2IHC181:10 PMthe 19th s3IHC412:40 PMthe 19th s4IHC612:50 PMthe 19th s5IHC203:00 PMthe 19th Near Solution Insurance=“IHC” Expectation: 0.4 Distance  5 Expectation: 0.3 Time  “1:00 PM” Expectation: 0.8 Date=“the 20th” Expectation: 0.9 r si s10.000  0.076 0.167  0.250  0.248 S20.000  0.090 0.160  0.250  0.252 s30.0000.007  0.014  0.250  0.236 s40.000  0.007  0.250  0.233 s50.000  0.102 0.083  0.250  0.256 “I want to see a dermatologist on the 20th, 1:00 PM or after. The dermatologist should be within 5 miles from my home and must accept my IHC insurance.” Can this constraint “1:00 PM or after” be relaxed to “12:40 PM” Can this constraint “the 20th” be relaxed to “the 19th” 12:40 PMthe 19th

13. 13 Performance Analysis Tested on appointment and car purchase domains 16 human subjects  The best-5 near solutions from 19 appointments  The best-5 solutions from 32 cars Compare human selection with system selection with respect to the best-5

14. 14 Performance Analysis Human selection versus system selection: appointment

15. 15 Performance Analysis Human selection versus system selection: car purchase

16. 16 Performance Analysis Inter-observer agreement test Results  kappa 0.74 (appointment)  kappa 0.67 (car purchase) “Substantial” agreement based on kappa values