1 Prasad Tadepalli Intelligent assistive systems Infer the goals of the human users and offer timely help; applications to assistance, tutoring; Learning hierarchical task knowledge Learning hierarchical task decomposition knowledge by watching people; learn new tasks Transfer learning of sequential task knowledge Transferring knowledge learned in previous tasks to new related tasks Learning from incomplete and biased data Learn general rules from natural language texts which are incomplete and systematically biased Learning in structured input and output spaces Learning to resolve co-references in natural language

2 Learning from Demonstrations Learn: A general plan Explicate the goal hierarchy Generalize the plan Proactive help – complete steps, prevent errors Input: Single video of assembly Recognize the activities Generate a causal annotation

3 Decision Theoretic Assistive Systems (with Alan Fern) Many examples of AI techniques being applied to assistive technologies, CALO, Task Tracer, COACH system, Electric Elves Most previous work uses problem-specific, hand-crafted solutions Lack ability to offer assistance in ways not planned for by designer No overarching framework Our Work: Gave a general, formal framework for intelligent-assistant design Evaluated in multiple domains including a folder predictor task with good results Desirable properties: Explicitly reason about models of the world and user to provide flexible assistance Handle uncertainty about the world and user Handle variable costs of user and assistive actions

4 W6W6 W7W7 W8W8 W9W9 Goal Achieved W2W2 User Action W4W4 W5W5 W3W3 Assistant Actions W1W1 Goal Initial State Each user action and assistant action has a cost Action set U Action set A Objective: minimize expected cost of episodes Episodic Interaction Model

5 Toy Domain 1: The Doorman Domain World states: (x,y) location and door status Possible goals: Get wood, gold, or food User actions: Up, Down, Left, Right, noop Open a door in current room (all actions have cost = 1) Assistant actions: Open a door, noop (all actions have cost = 0) Assistant’s Objective: Act in a way to minimize the expected cost

6 Approximate Solution Approach Goal RecognizerAction Selection Environment User UtUt AtAt OtOt P(G) Assistant WtWt 1) Estimate posterior goal distribution given observations 2) Action selection via myopic heuristics

7 Folder Navigation Assistant

8 Folder Navigation Results restricted folder set all folders considered restricted to single action multiple actions Avg. no. of clicks per open/saveAs Current Tasktracer Full Assistant Framework

9 Learning Hierarchical Task Knowledge (with Tom Dietterich)

10 Basic Approach Build a causal annotation of the trajectory using domain action models Iteratively parse the trajectory into minimally interacting subtasks EndStartGotoMGGotoDepGotoCWGotoDep a.r req.gold req.wood a.l req.gold a.* reg.* req.wood reg.*

11 Induced Wargus Hierarchy Root Harvest WoodHarvest Gold Get GoldGet Wood Goto(loc) Mine GoldChop WoodGDeposit Put Gold Put Wood WGoto(townhall)GGoto(goldmine)WGoto(forest)GGoto(townhall) WDeposit

12 Results (7 replications)

13 Lifelong Active Transfer Learning (with Alan Fern)   LB Land battles general RTS game model  SB  S1  S2 s 14 s 13 s 12 s 11 WARCRAFT:human warfare s 21 experiences in sea battle 2  L1  L2 l 14 l 13 l 12 l 11 l 21 MAGANT: ant warfare “archers behind footmen” “long range units behind short range units” “dragons behind fireants” experiences in sea battle 1 Sea battles “fast units to lure slow enemy units”

14 Learning Rules from Texts (with Tom Dietterich and Xiaoli Fern) Natural language texts are radically incomplete Worse yet, they are systematically biased. Unusual facts are mentioned with higher frequency: the so called “man bites dog phenomenon” Solution: explicitly or implicitly model the systematic bias and take it into account when counting evidence Text documents Information Extractor Extracted facts Rule Learner KB of rules teamInGame(g,t1), teamInGame(g,t2), gameLoser(g,t2)  gameWinner(g,t1)