Context-Specific Intention Awareness through Web Query ICRA 2015 – NO.137 Nominated as best conference paper award, best student paper award, best cognitive robotics paper award. Context-Specific Intention Awareness through Web Query Rui Liu Dr. Xiaoli Zhang Jeremy Webb Songpo Li Colorado School of Mines, Colorado, USA Intelligent Healthcare Robotics Lab bio, cooperation

Colorado School of Mines, Colorado, USA Introduction CSIA Explores environmental context to infer intention in robotic caregiving Implicitly infer human intention by reducing user involvement Challenges Limited knowledge of Intention-environment correlation: source, cost Lack of knowledge learning in new situations WebIA: solution Collect knowledge by web query Update knowledge by Situations-driven Colorado School of Mines, Colorado, USA Intelligent Healthcare Robotics Lab definition, important, challenging, to solve it.

Colorado School of Mines, Colorado, USA WebIA Framework Performance unsatisfied >> Learn >>Update Colorado School of Mines, Colorado, USA Intelligent Healthcare Robotics Lab flow, update

Colorado School of Mines, Colorado, USA WebIA Answering 2 questions: Is WebIA feasible ? if feasible, how good it is Colorado School of Mines, Colorado, USA Intelligent Healthcare Robotics Lab short mention 2 questions.

Colorado School of Mines, USA Methodology: WebIA Knowledge Representation Intention-centered knowledge Intention-object correlation (affordance) Intention-context correlation (environment exploration) Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Colorado School of Mines, USA Methodology: WebIA Intention Inference Engine Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Colorado School of Mines, USA Methodology: WebIA Web Information Retrieval Text from WikiHow Parsing, Pattern Matching Dependency Analysis Correlations NLP Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Colorado School of Mines, USA Methodology: WebIA Situation-driven Knowledge Update to proactively update a robot’s knowledge to adapt to untrained new situations. New situation detection: human-involved, untrained, not a synonym of trained objects Knowledge update: establish the new intention-object correlations Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Colorado School of Mines, USA Evaluation Knowledge Acquiring: queried 1470 wikihow webpages, surveyed 120 volunteers. Robot-involved experiments The object cup has been identified by NAO. IA was performed based on the commonsense knowledge generated from WikiHow. NAO points forward to the sink when the inferred intention is ‘wash’. NAO waves its right hand to notify the caregiver when the inferred intention is ‘drink’. The new situation ‘cup of coffee’ is detected; then NAO gains new knowledge by querying WikiHow to perform IA in this specific situation. Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Colorado School of Mines, USA Evaluation Cup-related Commonsense Knowledge from the Web Accuracy: Drink: 100% Wash: 85% Comparison of Knowledge from Web & Survey Consistent Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Colorado School of Mines, USA Evaluation Situation-driven knowledge Update Evaluation Intention Probability Adjustment from Old to New Situations Detailed process in One IA Situation New situation before learning: 57.5% after learning : 100% Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Colorado School of Mines, Colorado, USA Conclusion WebIA effectively learn the relation between human intention-environment and accurately inferred human intention in specific situations. Feasible: WebIA has potential to query vast and valuable knowledge from the internet to assist human-robot interaction Reasonable: Web knowledge is helpful in filling robots’ knowledge gap for new situation adaptation Economic: WebIA reduces the training cost Colorado School of Mines, Colorado, USA Intelligent Healthcare Robotics Lab effectively learn, accurately infer.

Colorado School of Mines, USA Rui Liu Xiaoli Zhang Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Colorado School of Mines, USA Authors Rui Liu, PhD Student Interests: robot knowledge, computational linguistics and machine learning ResearchGate: https://goo.gl/UUASbB Google Scholar: https://goo.gl/CpgXNV Xiaoli Zhang, Assistant Professor Interests: intelligent human-robot interaction, human intention awareness, robotics system design and control, haptics, and their applications in healthcare fields Colorado School of Mines, USA Intelligent Healthcare Robotics Lab

Context-Specific Intention Awareness through Web Query Rui Liu1, Xiaoli Zhang1,Songpo Li1 and Jeremy Webb1 1Colorado School of Mines Use web query to establish a vast and active context-related knowledge for robot to infer human intention in both trained and untrained environment.