1. Barnan Das November 8, 2012 PhD Preliminary Exam ***Self-portraits by William Utermohlen, an American artist living in London, after he was diagnosed with Alzheimer’s disease in 1995. Utermohlen died from the consequences of Alzheimer’s disease in March 2007. Addressing Machine Learning Challenges to Perform Automated Prompting

2. 2 Worldwide Dementia population Source: World Health Organization and Alzheimer’s Association. Actual and expected number of Americans >=65 year with Alzheimer’s Payment for care in 2012 $200 billion Unpaid caregivers 15 million 36 million

3. 3

4. Automated Prompting 4 Help with Activities of Daily Living (ADLs)

5. 5  Rule-based (temporal or contextual)  Activity initiation  RFID and video-input based prompts for activity steps  Rule-based (temporal or contextual)  Activity initiation  RFID and video-input based prompts for activity steps  Learning-based  Sub-activity level prompts  No audio/video input  Learning-based  Sub-activity level prompts  No audio/video input Existing Work Our Contribution

6. System Architecture 6 Published at ICOST 2011 and Journal of Personal and Ubiquitous Computing 2012.

7. Outline of Work 7 Automated Prompting Off-line Classification of Activity Steps Imbalanced Class Distribution Overlapping Classes On-line Prediction for Streaming Sensor Events

8. Outline of Work 8 Automated Prompting Off-line Classification of Activity Steps Imbalanced Class Distribution Overlapping Classes On-line Prediction for Streaming Sensor Events

9. Off-line Classification of Activity Steps 9 prompt no-prompt

10. 10 Data Collection Experiments 8 Activities of Daily Living (ADLs) 128 older-adult participants Prompts issued when errors were committed Annotation ADLs Predefined ADL steps Prompt/No-prompt Clean Data 1 ADL step = 1 data point 17 engineered attributes Class labels = {prompt, no-prompt}

11. Class Distribution 11 Total number of data points 3980

12. Imbalanced Class Distribution 12

13. Existing Work 13  Preprocessing Sampling Over-sampling minority class Under-sampling majority class  Oversampling minority class Spatial location of samples in Euclidean feature space

14. Proposed Approach 14  Preprocessing technique  Oversampling minority class Based on Gibbs sampling Markov Chain Node Attribute Value Submitted at Journal of Machine Learning Research, 2012.

15. Proposed Approach 15 Minority Class Samples Majority Class Samples Majority Class Samples Markov Chains

16. (wrapper-based)RApidly COnverging Gibbs sampler: RACOG & wRACOG 16  Differ in sample selection from Markov chains  RACOG: Based on burn-in and lag Stopping criteria: predefined number of iterations Effectiveness of new samples is not judged  wRACOG: Iterative training on dataset, addition of misclassified data points Stopping criteria: No further improvement of performance measure (TP rate)

17. Experimental Setup 17 Datasets prompting abalone car nursery letter connect-4 Classifiers C4.5 decision tree SVM k-Nearest Neighbor Logistic Regression Other Methods SMOTE SMOTEBoost RUSBoost Implemented Gibbs sampling, SMOTEBoost, RUSBoost

18. Results (RACOG & wRACOG) 18 TP Rate Geometric Mean (TP Rate, TN Rate)

19. Results (RACOG and wRACOG) 19 ROC Curve

20. Outline of Work 20 Automated Prompting Off-line Classification of Activity Steps Imbalanced Class Distribution Overlapping Classes On-line Prediction for Streaming Sensor Events

21. Overlapping Classes 21

22. Overlapping Classes in Prompting Data 22 3D PCA Plot of prompting data

23. Existing Work 23  Discard data of the overlapping region  Treat overlapping region as a separate class

24. Tomek Links 24

25. Cluster-Based Under-Sampling(ClusBUS) 25 Published in IOS Press Book on Agent-Based Approaches to Ambient Intelligence, 2012. Form clusters Under-sampling interesting clusters

26. Experimental Setup 26

27. Results (ClusBus) 27

28. Outline of Work 28 Automated Prompting Off-line Classification of Activity Steps Imbalanced Class Distribution Class Overlap On-line Prediction for Streaming Sensor Events

29. Outline of Work 29 Automated Prompting Off-line Classification of Activity Steps Imbalanced Class Distribution Class Overlap On-line Prediction for Streaming Sensor Events

30. Unsupervised Learning of Prompt Situations on Streaming Sensor Data 30 s1s1 s2s2 s4s4 s1s1 s3s3 s2s2

31. Motivation 31  Several hundred man-hours to label activity steps  High probability of inaccuracy  Needs activity-step recognition model

32. 32 Knowledge Flow

33. Data Collection 33 ADLs SweepingMedicationCookingWatering PlantsHand WashingCleaning Kitchen Countertops Errors Abnormal OccurrenceDelayed Occurrence Participants33 Normal Activity Sequences33 Erroneous Activity Sequences33x3

34. Modeling Activity Errors 34 Abnormal Occurrence Delayed Occurrence Abnormal Occurrence Delayed Occurrence Gaussian distribution of time elapsed for n th occurrence of s i Gaussian distribution of sensor trigger frequency for n th occurrence of s i

35. 35 Modeling Delayed Occurrence Elapsed Time Sensor Frequency

36. Predicting Errors 36 At every sensor event evaluate: Likelihood of sensor s i occurrence for participant p j Probability of elapsed time for current n th occurrence of sensor s i Probability of all sensor frequency for current n th occurrence of sensor s i

37. Preliminary Experiments 37 Elapsed Time Sensor Frequency No observable trend

38. Current Obstacles 38  Noisy data Unwanted sensor events, specifically, object sensors  Erroneous activity sequences not suitable for model evaluation

39. Proposed Plan 39  Identifying suitable distributions for modeling sensor frequency and elapsed time  Finding out additional statistical measures that can model the errors better  Building generalized prompt model for all six ADLs (if at all possible(?))  Need data to evaluate proposed model Synthetically generate erroneous sequences from normal sequences(?) Collect more data if necessary

40. Publications 40 Book Chapters B. Das, N.C. Krishnan, D.J. Cook, “Handling Imbalanced and Overlapping Classes in Smart Environments Prompting Dataset”, Springer Book on Data Mining for Services, 2012. (Submitted) B. Das, N.C. Krishnan, D.J. Cook, “Automated Activity Interventions to Assist with Activities of Daily Living”, IOS Press Book on Agent-Based Approaches to Ambient Intelligence, 2012. Journal Articles B. Das, N. C. Krishnan, D. J. Cook, “RACOG and wRACOG: Two Gibbs Sampling-Based Oversampling Techniques”, Journal of Machine Learning Research, 2012. (Submitted) A.M. Seelye, M. Schmitter-Edgecombe, B. Das, D.J. Cook, “Application of Cognitive Rehabilitation Theory to the Development of Smart Prompting Technologies”, IEEE Reviews on Biomedical Engineering, 2012. (Accepted) B. Das, D.J. Cook, M. Schmitter-Edgecombe, A.M. Seelye, “PUCK: An Automated Prompting System for Smart Environments”, Journal of Personal and Ubiquitous Computing, 2012. Conferences S. Dernbach, B. Das, N.C. Krishnan, B.L. Thomas, D.J. Cook, “Simple and Complex Acitivity Recognition Through Smart Phones”, International Conference on Intelligent Environments (IE), 2012. B. Das, C. Chen, A.M. Seelye, D.J. Cook, “An Automated Prompting System for Smart Environments”, International Conference on Smart Homes and Health Telematics (ICOST), 2011. E. Nazerfard, B. Das, D.J. Cook, L.B. Holder, “Conditional Random Fields for Activity Recognition in Smart Environments”, International Symposium on Human Informatics (SIGHIT), 2010. C. Chen, B. Das, D.J. Cook, “A Data Mining Framework for Activity Recognition in Smart Environments”, International Conference on Intelligent Environments (IE), 2010. Workshops and Demos B. Das, B.L. Thomas, A.M. Seelye, D.J. Cook, L.B. Holder, M. Schmitter-Edgecombe, “Context-Aware Prompting From Your Smart Phone”, Consumer Communication and Networking Conference Demonstration (CCNC), 2012 B. Das, A.M. Seelye, B.L. Thomas, D.J. Cook, L.B. Holder, M. Schmitter-Edgecombe, “Using Smart Phones for Context- Aware Prompting in Smart Environments”, CCNC Workshop on Consumer eHealth Platforms, Services and Applications (CeHPSA), 2012. B. Das, D.J. Cook, “Data Mining Challenges in Automated Prompting Systems”, IUI Workshop on Interaction with Smart Objects Workshop (InterSO), 2011. B. Das, C. Chen, N. Dasgupta, D.J. Cook, “Automated Prompting in a Smart Home Environment”, ICDM Workshop on Data Mining for Service, 2010. C. Chen, B. Das, D.J. Cook, “Energy Prediction Using Resident’s Activity”, KDD Workshop on Knowledge Discovery from Sensor Data (SensorKDD), 2010, C. Chen, B. Das, D.J. Cook, “Energy Prediction in Smart Environments”, IE Workshop on Artificial Intelligence Techniques for Ambient Intelligence (AITAmI), 2010.

41. 41