Download presentation
Presentation is loading. Please wait.
Published byHarold Patterson Modified over 9 years ago
1
User-Initiated Learning (UIL) Kshitij Judah, Tom Dietterich, Alan Fern, Jed Irvine, Michael Slater, Prasad Tadepalli, Oliver Brdiczka, Jim Thornton, Jim Blythe, Christopher Ellwood, Melinda Gervasio, Bill Jarrold
2
CALO: Intelligent Assistant for the Desktop Knowledge Worker Learn to Understand Meetings Learn to Keep User Organized Learn to Manage Email Learn to Prepare information Products Learn to Schedule and Execute CALO: Learning to be an Intelligent Assistant PAL Program Focus: “Learning in the Wild” 2
3
User-Initiated Learning All of CALO’s learning components can perform Learning In The Wild (LITW) But the learning tasks are all pre-defined by CALO’s engineers: What to learn What information is relevant for learning How to acquire training examples How to apply the learned knowledge UIL Goal: Make it possible for the user to define new learning tasks after the system is deployed 3
4
Motivating Scenario: Forgetting to Set Sensitivity TIMELINETIMELINE Scientist Sets sensitivity to confidential Sends email to team Sends email to a colleague “Lunch today?” Does not set sensitivity to confidential Collaborates on a Classified project Sends email to team Forgets to set sensitivity to confidential Research Team 4
5
Motivating Scenario: Forgetting to Set Sensitivity TIMELINETIMELINE “Please do not forget to set sensitivity when sending email” Scientist Research Team Teaches CALO to learn to predict whether user has forgot to set sensitivity Sends email to team CALO reminds user to set sensitivity 5
6
User-CALO Interaction: Teaching CALO to Predict Sensitivity SAT Based Reasoning System SPARK Procedure Instrumented Outlook Instrumented Outlook Events user Integrated Task Learning user Compose new email Modify Procedure Procedure Demonstration and Learning Task Creation User Interface for Feature Guidance User Selected Features user Feature Guidance Email + Related Objects CALO Ontology Trained Classifier Feature Guidance Machine Learner Knowledge Base Training Examples Learning Legal Features SAT Based Reasoning System Class Labels 6
7
User-CALO Interaction: Teaching CALO to Predict Sensitivity SAT Based Reasoning System SPARK Procedure Instrumented Outlook Instrumented Outlook Events user Integrated Task Learning user Compose new email Modify Procedure Procedure Demonstration and Learning Task Creation User Interface for Feature Guidance User Selected Features user Feature Guidance Email + Related Objects CALO Ontology Trained Classifier Feature Guidance Machine Learner Knowledge Base Training Examples Learning Legal Features SAT Based Reasoning System Class Labels 7
8
Initiating Learning via Demonstration LAPDOG: Transforms an observed sequence of instrumented events into a SPARK procedure SPARK representation generalizes the dataflow between the actions of the workflow 8
9
Initiating Learning via Demonstration TAILOR: Supports procedure editing For UIL, it allows adding a condition to one or more steps in a procedure 9
10
Initiating Learning via Demonstration The condition becomes the new predicate to be learned 10
11
User-CALO Interaction: Teaching CALO to Predict Sensitivity SAT Based Reasoning System SPARK Procedure Instrumented Outlook Instrumented Outlook Events user Integrated Task Learning user Compose new email Modify Procedure Procedure Demonstration and Learning Task Creation User Interface for Feature Guidance User Selected Features user Feature Guidance Email + Related Objects CALO Ontology Trained Classifier Feature Guidance Machine Learner Knowledge Base Training Examples Learning Legal Features SAT Based Reasoning System Class Labels 11
12
Inferring Feature Legality HasToField HasSubject HasBody HasAttachment HasSensitivity … Description StartDate Enddate … HasToField HasSubject HasBody HasAttachment HasSensitivity … FirstName LastName Address Phone … FirstName LastName Address Phone … ToRecipient CCRecipient PrevEmailMessage Project EmailMessage Subset of Ontology Naively, the system will use all features For example, system will use HasSensitivity Dangerous to use HasSensitivity: Has one-to-one correlation with target and is present at training time Not present at test time Feature filtering removes such features at training time 12
13
Feature Guidance Interface 13
14
User-CALO Interaction: Teaching CALO to Predict Sensitivity SAT Based Reasoning System SPARK Procedure Instrumented Outlook Instrumented Outlook Events user Integrated Task Learning user Compose new email Modify Procedure Procedure Demonstration and Learning Task Creation User Interface for Feature Guidance User Selected Features user Feature Guidance Email + Related Objects CALO Ontology Trained Classifier Feature Guidance Machine Learner Knowledge Base Training Examples Learning Legal Features SAT Based Reasoning System Class Labels 14
15
Training Instance Generation Goal: autonomously generate labeled training instances for the learning component from stored user emails Problem: actions used to create emails are not stored in the CALO knowledge base, so we need to infer how email was created {defprocedure do_rememberSensitivity.... [do: (openComposeEmailWindow $newEmail)] [do: (changeEmailField $newEmail "to")] [do: (changeEmailField $newEmail "subject")] [do: (changeEmailField $newEmail "body")] [if: (learnBranchPoint $newEmail) [do: (changeEmailField $newEmail "sensitivity")]] [do: (sendEmailInitial $newEmail)].... } Specifically, we want to know: Whether an email is an instance of the procedure? Which branch was taken during creation of the email? No such inference can be drawn 15
16
Training Instance Generation Domain Axioms Domain Axioms NewComposition ComposeNewMail ReplyComposition ReplyToMail HasAttachment (AttachFile ForwardMail) … SPARK Axioms SPARK Axioms ProcInstance (u 1 u 2 … U n ) ( forget label) (C 1 C 2 … C n ) {defprocedure do_rememberSensitivity.... [do: (openComposeEmailWindow $newEmail)] [do: (changeEmailField $newEmail "to")] [do: (changeEmailField $newEmail "subject")] [do: (changeEmailField $newEmail "body")] [if: (learnBranchPoint $newEmail) [do: (changeEmailField $newEmail "sensitivity")]] [do: (sendEmailInitial $newEmail)].... } Label Analysis Formula (LAF) Label Analysis Formula (LAF) Knowledge Base NewComposition ReplyComposition HasToField HasSubject HasBody HasAttachment … Reasoning Engine Reasoning Engine E forget ╞ (ProcInstance Label) E forget ╞ (ProcInstance Label) otherwise Positive Example Negative Example Discard email 16
17
Training Instance Generation Goal: autonomously generate labeled training instances for the learning component from stored user data Problem: actions used to create the objects are not stored in the CALO knowledge base, so we need to infer the actions SAT based reasoning system is used It employs a domain model of the UI actions and how they affect the object attributes 17
18
Training Instance Generation: The Domain Model A proposition symbol is defined for each email attribute and action Example action propositions: ComposeNewMail, ForwardMail, ReplyToMail, ModifyToField etc. Example property propositions: NewComposition, ForwardedComposition, HasCCField, HasBody etc. An action proposition is true relative to an email iff it was taken during creation of the email A property proposition is true relative to an email iff the email satisfies the corresponding property 18
19
Training Instance Generation: The Domain Model The domain model includes one axiom for each property proposition and defines the possible ways that proposition can be made true Example axioms: NewComposition ComposeNewMail ReplyComposition ReplyToMail HasAttachment (AttachFile ForwardMail) HasSubject (ModifySubject ReplyToMail ForwardMail) … 19
20
Training Instance Generation: Inferring Class Labels Given email from CALO KB and SPARK procedure, we wish to assign class label to the email We construct the Label Analysis Formula (LAF) LAF contains domain propositions + three new propositions: Label: represents truth value of the branch condition Forget: whether user intended but forgot to perform conditional action ProcInstance: whether email is instance of the procedure LAF = Domain axioms + SPARK axioms 20
21
Training Instance Generation: Inferring Class Labels SPARK axioms place constraints on ProcInstance, Label and Forget: ProcInstance (u 1 u 2 … U n ) ( forget label) (C 1 C 2 … C n ) Email is labeled as follows: Construct formula E: (1) For each property proposition P, if property is true in the email, add P to LAF else add P; (2) Add forget to the LAF Try to prove: E ╞ (ProcInstance Label) If successful, email is a positive training instance Try to prove: E ╞ (ProcInstance Label) If successful, email is a negative training instance otherwise email is not a training instance 21
22
The Learning Component Logistic Regression is used as the core learning algorithm Features Relational features extracted from ontology Incorporate User Advice on Features Apply large prior variance on user selected features Select prior variance on rest of the features through cross- validation Automated Model Selection Parameters: Prior variance on weights, classification threshold Technique: Maximization of leave-one-out cross-validation estimate of kappa 22
23
Assisting the User: Inferring Forgetting Given new email, forgetting is inferred as follows: Construct formula E: (1) For each property proposition P, if property is true in the email, add P to LAF else add P; (2) Add label to the LAF if prediction is true else add label to the LAF Try to prove: E ╞ (ProcInstance forget) If successful, stop send and remind the user Try to prove: E ╞ (ProcInstance forget) If successful, send email Otherwise, send email? 23
24
Assisting the User: Reminding 24
25
Empirical Evaluation Problems: Attachment Prediction Importance Prediction Learning Configurations Compared: No User Advice + Fixed Model Parameters User Advice + Fixed Model Parameters No User Advice + Automatic parameter Tuning User Advice + Automatic parameter Tuning User Advice: 18 keywords in the body text for each problem 25
26
Empirical Evaluation: Data Set Set of 340 emails obtained from a real desktop user 256 training set + 84 test set For each training set size, compute mean kappa ( ) using test set to generate learning curves is a statistical measure of inter-rater agreement for discrete classes is a common evaluation metric in cases when the classes have a skewed distribution 26
27
Empirical Evaluation: Learning Curves (A) No User Advice + Fixed Model Parameters (B) User Advice + Fixed Model Parameters (C) No User Advice + Model Selection (D) User Advice + Model Selection Attachment Prediction User advice is helpful but not critical Self-tuning is absolutely critical 27
28
Empirical Evaluation: Learning Curves Attachment Prediction 28
29
Empirical Evaluation: Learning Curves Attachment Prediction 29
30
Empirical Evaluation: Learning Curves Attachment Prediction 30
31
Empirical Evaluation: Learning Curves Attachment Prediction 31
32
Empirical Evaluation: Learning Curves Importance Prediction 32
33
Empirical Evaluation: Learning Curves Importance Prediction 33
34
Empirical Evaluation: Learning Curves Importance Prediction 34
35
Empirical Evaluation: Learning Curves Importance Prediction 35
36
Empirical Evaluation: Robustness to Bad Advice We intended to test the robustness of the system to bad advice Bad advice was generated as follows: Use SVM based feature selection in WEKA to produce a ranking of user provided keywords Replace top three words in the ranking with randomly selected words from the vocabulary 36
37
Empirical Evaluation: Robustness to Bad Advice Attachment Prediction 37
38
Empirical Evaluation: Robustness to Bad Advice Attachment Prediction 38
39
Empirical Evaluation: Robustness to Bad Advice Attachment Prediction 39
40
Empirical Evaluation: Robustness to Bad Advice Attachment Prediction 40
41
Empirical Evaluation: Robustness to Bad Advice Importance Prediction 41
42
Empirical Evaluation: Robustness to Bad Advice Importance Prediction 42
43
Empirical Evaluation: Robustness to Bad Advice Importance Prediction 43
44
Empirical Evaluation: Robustness to Bad Advice Importance Prediction 44
45
Empirical Evaluation: Prediction Utility We want to evaluate the utility of the system for the user We use a new metric called Critical Cost Ratio (CCR) Intuition: A measure of how high cost of forgetting should be compared to cost of interruption for the system to be useful Intuition : Hence, if CCR is low, the system is useful more often For example, if CCR=10, then cost of forgetting should be 10 times more than cost of interruption for net benefit 45
46
Empirical Evaluation: Prediction Utility Attachment Prediction 46
47
Empirical Evaluation: Prediction Utility Attachment Prediction 47 At size 256, cost of forgetting should be at least 5 times of cost of interruption to gain net benefit from the system
48
Empirical Evaluation: Prediction Utility Importance Prediction 48
49
Empirical Evaluation: Prediction Utility Importance Prediction 49
50
Lessons Learned User interfaces should support rich instrumentation, automation, and intervention User interfaces should come with models of their behavior User advice is helpful but not critical Self-tuning learning algorithms are critical for success 50
51
Beyond UIL: System-Initiated Learning CALO should notice when it could help the user by formulating and solving new learning tasks Additional Requirements Knowledge of user’s goals, user’s costs, user’s failure modes (e.g., forgetting, over-committing, typos) Knowledge of what is likely to be learnable and what is not Knowledge of how to formulate learning problems (classification, prediction, anomaly detection, etc.) 51
52
Thank you and Questions 52
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.