1. Decision-Making on Robots Using POMDPs and Answer Set Programming Introduction Robots are an integral part of many sectors such as medicine, disaster rescue and the military. Real-world domains are characterized by partial observability, non- deterministic action outcomes and unforeseen dynamic changes. Probabilistic planning using partially observable Markov decision processes (POMDPs) models the uncertainty in sensing and actuation on robots in the real-world. POMDPs are not well-suited for knowledge representation or common sense reasoning. Answer set programming (ASP) is a non-monotonic logic programming paradigm that is appropriate for common sense reasoning. This research project investigates the use of a POMDP hierarchy in conjunction with ASP to enable mobile robots to reliably and efficiently localize target objects in indoor domains. POMDP Partially Observable Markov Decision Process. Defined by the tuple (S, A, T, Z, O, R): States, Actions, Transition Function, Observations, Observation Function, Reward Function. The true state cannot be observed. A probability distribution over the state is maintained (Belief state). POMDP model solved to obtain a policy that maps belief states to actions. Problem formulated as a two-level POMDP. Low-level POMDP computes the sequence of viewpoints to observe the target. High-level POMDP uses information from the low-level POMDP to determine if the observations correspond to the desired target. Answer Set Programming References 1.S. Zhang and M. Sridharan. “Active Visual Sensing and Collaboration on Mobile Robots using Hierarchical POMDPs.” AAMAS 2012. 2.S. Zhang, F. Bao and M. Sridharan. “Combining Probabilistic Planning and Logic Programming on Mobile Robots.” AAAI 2012. 3.S. Zhang, M. Sridharan and X. Li. “To Look or Not to Look: A Hierarchical Representation for Visual Planning on Mobile Robots.” ICRA 2011. Erratic Robot Three wheeled Hokuyo laser range finder (30 meter, 270° range) Stereo camera Sonar On-board processor Robot Operating System (ROS) Objectives Existing POMDP hierarchy enables the robot to decide where to look, what images to process and how to process these images. The current project focuses on robustly identifying objects by observing them from different viewpoints. Given an object with multiple sides, create a POMDP to correctly identify the target object. Use ASP to model common sense reasoning. Program robot to recognize initial position. Algorithms implemented and evaluated in simulation and on wheeled robots in complex indoor domains. Future Work Combine novel POMDP hierarchy with previous research. Enable robot to set initial position using environment clues. * This material is based upon work supported by the National Science Foundation under Grant No. CNS-1005212. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the reflect the views of the National Science Foundation. High-Level POMDP Low-Level POMDP Is this the desired target? Do I have enough information to make that decision? Which side do I observe, if any? Should I move clockwise? Counter-clockwise? Localization Electronics Office Shredder Printer Living Room TV DVD Player Logical reasoning used to infer location of similar objects. Hierarchy(pictured right) part of robot’s knowledge base. Categories: Electronics, Office, Living Room. Objects: Shredder, Printer, TV, DVD Player. Rule: If object-X is found in room-Y and object-X is in category-Z, objects of category-Z can be found in room-Y. To find targets, robot must know initial position in learned map. Previously, initial position was set by clicking location on map. Initial position can now be set in code and on map. Map with robot position initialized: blue/green represents obstacles, red represents particle cloud. Integration of POMDP and Answer set programming Christian Washington Louisiana State University Department of Electrical and Computer Engineering Shiqi Zhang, Mohan Sridharan Texas Tech University Department of Computer Science