Sebastian Thrun Carnegie Mellon & Stanford Wolfram Burgard University of Freiburg and Dieter Fox University of Washington Probabilistic Algorithms for Mobile Robot Mapping

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Based on the paper A Real-Time Algorithm for Mobile Robot Mapping With Applications to Multi-Robot and 3D Mapping Best paper award at 2000 IEEE International Conference on Robotics and Automation (~1,100 submissions) Sponsored by DARPA (TMR-J.Blitch, MARS-D.Gage, MICA-S.Heise) and NSF (ITR(2), CAREER-E.Glinert, IIS-V.Lumelsky) Other contributors: Yufeng Liu, Rosemary Emery, Deepayan Charkrabarti, Frank Dellaert, Michael Montemerlo, Reid Simmons, Hugh Durrant-Whyte, Somajyoti Majnuder, Nick Roy, Joelle Pineau, …

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Open Problems 3D Mapping with EM Real Time Hybrid Expectation Maximization SLAM (Kalman filters) Motivation

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Museum Tour-Guide Robots With: Greg Armstrong, Michael Beetz, Maren Benewitz, Wolfram Burgard, Armin Cremers, Frank Dellaert, Dieter Fox, Dirk Haenel, Chuck Rosenberg, Nicholas Roy, Jamie Schulte, Dirk Schulz

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 The Nursebot Initiative With: Greg Armstrong, Greg Baltus, Jacqueline Dunbar- Jacob, Jennifer Goetz, Sara Kiesler, Judith Matthews, Colleen McCarthy, Michael Montemerlo, Joelle Pineau, Martha Pollack, Nicholas Roy, Jamie Schulte

Sebastian Thrun, Carnegie Mellon, IJCAI-2001

Mapping: The Problem n Concurrent Mapping and Localization (CML) n Simultaneous Localization and Mapping (SLAM)

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Mapping: The Problem n Continuous variables n High-dimensional (eg, 1,000,000+ dimensions) n Multiple sources of noise n Simulation not acceptable

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Milestone Approaches Mataric 1990 Kuipers et al 1991 Elfes/Moravec 1986 Lu/Milios/Gutmann 1997

Sebastian Thrun, Carnegie Mellon, IJCAI D Mapping Konolige et al, 2001Teller et al, 2000 Moravec et al, 2000

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Take-Home Message Mapping is the holy grail in mobile robotics. Every state-of-the-art mapping algorithm is probabilistic.

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Open Problems 3D Mapping with EM Real Time Hybrid Expectation Maximization Motivation SLAM (Kalman filters)

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Bayes Filters Special cases: HMMs DBNs POMDPs Kalman filters Condensation... x = state t = time z = measurement u = control  = constant

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Bayes Filters in Localization [Simmons/Koenig 95] [Kaelbling et al 96] [Burgard, Fox, et al 96]

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Bayes Filters for Mapping s = robot pose m = map t = time  = constant z = measurement u = control Mapping? Localization:

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Kalman Filters (SLAM) [Smith, Self, Cheeseman, 1990]

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Underwater Mapping with SLAM Courtesy of Hugh Durrant-Whyte, Univ of Sydney

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Large-Scale SLAM Mapping Courtesy of John Leonard, MIT

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 SLAM: Limitations n Linear n Scaling: O(N 4 ) in number of features in map n Can’t solve data association problem

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Motivation Open Problems 3D Mapping with EM Real Time Hybrid SLAM (Kalman filters) Expectation Maximization

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Unknown Data Association: EM M-Step: Mapping with known posesE-Step: Localization [Dempster et al, 77] [Thrun et al, 1998] [Shatkay/Kaelbling 1997]

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 CMU’s Wean Hall (80 x 25 meters) 15 landmarks 16 landmarks 17 landmarks27 landmarks

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 EM Mapping, Example (width 45 m)

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 EM Mapping: Limitations n Local Minima n Not Real-Time

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Motivation SLAM (Kalman filters) Open Problems 3D Mapping with EM Expectation Maximization Real Time Hybrid

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 The Goal EM: data association Not real-time Kalman filters: real-time No data association ? ?

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Real-Time Approximation (ICRA paper)   Incremental ML

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Incremental ML: Not A Good Idea path robot mismatch

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Real-Time Approximation   Our ICRA Paper

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Real-Time Approximation Yellow flashes: artificially distorted map (30 deg, 50 cm)

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Importance of Posterior Pose Estimate Without pose posteriorWith pose posterior

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Online Mapping with Posterior Courtesy of Kurt Konolige, SRI, DARPA-TMR [Gutmann & Konolige, 00]

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 CAD map Accuracy: “The Tech” Museum, San Jose 2D Map, learned

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Multi-Robot Mapping n Every module maximizes likelihood n Pre-aligned scans are passed up in hierarchy map Cascaded architecture map …… Aligned map Pre-aligned scans

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Multi-Robot Exploration DARPA TMR Maryland 7/00DARPA TMR Texas 7/99 (July. Texas. No air conditioning. Req to dress up. Rattlesnakes)

Sebastian Thrun, Carnegie Mellon, IJCAI D Volumetric Mapping

Sebastian Thrun, Carnegie Mellon, IJCAI D Structure Mapping

Sebastian Thrun, Carnegie Mellon, IJCAI D Texture Mapping

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Fine-Grained Structure: Can We Do Better?

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Motivation SLAM (Kalman filters) Expectation Maximization Open Problems Real Time Hybrid 3D Mapping with EM

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Multi-Planar 3D Mapping Idea: Exploit fact that buildings posses many planar surfaces n Compact models n High Accuracy n Objects instead of pixels

Sebastian Thrun, Carnegie Mellon, IJCAI D Multi-Plane Mapping Problem Entails five problems –Generative model with priors: Not all of the world is planar –Parameter estimation: Location and angle of planar surfaces unknown –Outlier identification: Not all measurements correspond to planar surfaces (other objects, noise) –Correspondence: Different measurements correspond to different planar surfaces –Model selection: Number of planar surfaces unknown

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Expected Log-Likelihood Function [Liu et al, ICML-01]

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 EM To The Rescue! ** ****

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Results With EM (95% of data explained by 7 surfaces) Without EM With: Deepayan Chakrabarti, Rosemary Emery, Yufeng Liu, Wolfram Burgard, ICML-01 error

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 The Obvious Next Step EM for object mapping EM for concurrent localization

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Underwater Mapping (with University of Sydney) With: Hugh Durrant-Whyte, Somajyoti Majunder, Marc de Battista, Steve Scheding

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Motivation SLAM (Kalman filters) Expectation Maximization Real Time Hybrid 3D Mapping with EM Open Problems

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Take-Home Message Mapping is the holy grail in mobile robotics. Every state-of-the-art mapping algorithm is probabilistic. Sebastian has one cool animation!

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Open Problems n 2D Indoor mapping and exploration n 3D mapping (real-time, multi-robot) n Object mapping (desks, chairs, doors, …) n Outdoors, underwater, planetary n Dynamic environments (people, retail stores) n Full posterior with data association (real-time, optimal)

Sebastian Thrun, Carnegie Mellon, IJCAI-2001 Open Problems, con’t n Mapping, localization n Control/Planning under uncertainty n Integration of symbolic making n Human robot interaction Literature Pointers: n “Robotic Mapping” at n “Probabilistic Robotics” AI Magazine 21(4)

Sebastian Thrun, Carnegie Mellon, IJCAI