Motion Planning: A Journey of Robots, Digital Actors, Surgical Instruments, Molecules and Other Artifacts Jean-Claude Latombe Computer Science Department Stanford University

Goal of Motion Planning u Compute motion strategies, e.g.: –geometric paths –time-parameterized trajectories –sequence of sensor-based motion commands u To achieve high-level goals, e.g.: –go from A to B without colliding with obstacles –assemble product P –build map of environment E –find object O

Goal of Motion Planning u Compute motion strategies, e.g.: –geometric paths –time-parameterized trajectories –sequence of sensor-based motion commands u To achieve high-level goals, e.g.: –go from A to B without colliding with obstacles –assemble product P –build map of environment E –find object O

Goal of Motion Planning u Compute motion strategies, e.g.: –geometric paths –time-parameterized trajectories –sequence of sensor-based motion commands u To achieve high-level goals, e.g.: –go from A to B without colliding with obstacles –assemble product P –build map of environment E –find object O

Basic Problem

Extensions to the Basic Problem u Moving obstacles u Multiple robots u Movable objects u Assembly planning u Goal is to acquire information by sensing –Model building –Object finding/tracking u Nonholonomic constraints u Dynamic constraints u Optimal planning u Uncertainty in control and sensing u Exploiting task mechanics (sensorless motions) u Physical models and deformable objects u Integration of planning and control

Extensions to the Basic Problem u Moving obstacles u Multiple robots u Movable objects u Assembly planning u Goal is to acquire information by sensing –Model building –Object finding/tracking u Nonholonomic constraints u Dynamic constraints u Optimal planning u Uncertainty in control and sensing u Exploiting task mechanics (sensorless motions) u Physical models and deformable objects u Integration of planning and control

Extensions to the Basic Problem u Moving obstacles u Multiple robots u Movable objects u Assembly planning u Goal is to acquire information by sensing –Model building –Object finding/tracking u Nonholonomic constraints u Dynamic constraints u Optimal planning u Uncertainty in control and sensing u Exploiting task mechanics (sensorless motions) u Physical models and deformable objects u Integration of planning and control

Extensions to the Basic Problem u Moving obstacles u Multiple robots u Movable objects u Assembly planning u Goal is to acquire information by sensing –Model building –Object finding/tracking u Nonholonomic constraints u Dynamic constraints u Optimal planning u Uncertainty in control and sensing u Exploiting task mechanics (sensorless motions) u Physical models and deformable objects u Integration of planning and control

Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

Early Work Shakey (Nilsson, 1969): Visibility graph

Mathematical Foundations C = S 1 x S 1 Lozano-Perez, 1980: Configuration Space

Computational Analysis Reif, 1979: Hardness (lower-bound results)

Exact General-Purpose Path Planners - Schwarz and Sharir, 1983: Exact cell decomposition based on Collins technique - Canny, 1987: Silhouette method

Heuristic Planners Khatib, 1986: Potential Fields

Nonholonomic Robots Laumond, 1986

Underactuated Robots Lynch, Shiroma, Arai, and Tanie, 1998

Part Orientation Godlberg, 1993

Assembly Sequence Planning Wilson, 1994: Non-Directional Blocking Graphs

Manipulation Planning Tsai-Yen Li, 1994

Deformable Objects Kavraki, Lamiraux, and Holleman 1998

Target Finding Guibas, Latombe, LaValle, Lin, and Motwani, 1997 Lin, and Motwani, 1997

Integration of Planning and Control Brock and Khatib, 1999

Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

Robot Programming and Placement David Hsu, 1999

Design for Manufacturing and Servicing General Electric General Motors

Design of Large Facilities EDF and LAAS-CNRS (MOLOG project), 1999

Verification of Building Code Charles Han, 1998

Graphic Animation of Digital Actors Koga, Kondo, Kuffner, and Latombe, 1994 The Motion Factory

Plan Sense Act Digital Actor = Virtual Robot! Graphic Animation of Digital Actors Kuffner, 1999

Vision module image Actor camera image Graphic Animation of Digital Actors u Segment environment u Render false-color scene offscreen u Scan pixels & record IDs Simulated Vision

Graphic Animation of Digital Actors

Surgical Planning Cyberknife System (Accuray, Inc.) CARABEAMER Planner Tombropoulos, 1997

Prediction of Molecular Motions Amit Singh, 1999

Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

Approaches to Motion Planning u Goal: Answer queries about the connectivity of a certain space (e.g., the collision-free subset of configuration space)

Approaches to Motion Planning u Old view (Latombe, 1991): –Roadmaps –Cell decomposition –Potential field

Approaches to Motion Planning u Old view (Latombe, 1991): –Roadmaps –Cell decomposition –Potential field u New View (Latombe, 2000): –Finding criticalities –Random sampling

Criticality-Based Motion Planning Retraction on Voronoi Diagram (O’Dunlaing and Yap, 1982)

Criticality-Based Motion Planning Part orientation (Goldberg, 1993)

Criticality-Based Motion Planning Non-Directional Blocking Graphs for assembly planning (Wilson, 1994)

Criticality-Based Motion Planning Non-Directional Preimage for landmark-based navigation (Lazanas, 1995)

Criticality-Based Motion Planning Non-Directional Preimage for landmark-based navigation (Lazanas, 1995)

Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997)

Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997)

Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997)

Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997) Example of an information state = (1,1,0) 0 : the target does not hide beyond the edge 1 : the target may hide beyond the edge

Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997) Recontaminated area

Criticality-Based Motion Planning u Advantage: –Completeness u Drawbacks: –Computational complexity –Difficult to implement

Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

Random-Sampling Planning admissible space qbqbqbqb qgqgqgqg milestone [Kavraki, Svetska, Latombe,Overmars, 95] (Probabilistic Roadmap)

Motivation Computing an explicit representation of the admissible space is hard, but checking that a point lies in the admissible space is fast

Why Does it Work? [Kavraki, Latombe, Motwani, Raghavan, 95] Relation with Art-Gallery problems

In Theory, Random-Sampling Planning… u Is probabilistically complete, i.e., whenever a solution exists, the probability that it finds one tends toward 1 as the number N of milestones increases u Under general hypotheses, the rate of convergence is exponential in N, i.e.: Prob[failure] = K exp(-N) u Computational gain is obtained against a “small” loss of completeness

Expansiveness of Admissible Space

Lookout of F1 The admissible space is expansive if each of its subsets has a large lookout Prob[failure] = K exp(-N)

In practice, Random-Sampling Planners… u Are fast u Deal effectively with many-dof robots u Deal well with complex admissibility constraints u Are easy to implement u Have solved complex problems

Real-Time Planning with Dynamic Constraints air bearing gaz tank air thrusters obstacles robot (Kindel, Hsu, Latombe, and Rock, 2000)

Total duration : 40 sec

Interactive Planning of Manipulation Motions Reach Grab Transfer Release Return Kuffner, 1999

Random-Sampling Radiosurgical Planning Cyberknife (Neurosurgery Dept., Stanford, Accuray) Tombropoulos, 1997 CARABEAMER Planner

Random-Sampling Radiosurgical Planning Dose to the Critical Region Critical Tumor Fall-off of Dose Around the Tumor Dose to the Tumor Region Fall-off of Dose in the Critical Region

Random-Sampling Radiosurgical Planning

2000 < Tumor < < B2 + B4 < < B4 < < B3 + B4 < < B3 < < B1 + B3 + B4 < < B1 + B4 < < B1 + B2 + B4 < < B1 < < B1 + B2 < < Critical < < B2 < 500 T C B1 B2 B3 B4 T

Sample Case 50% Isodose Surface 80% Isodose Surface Conventional system’s plan CARABEAMER’s plan

Randomized Next-Best View Planning (Gonzalez, 2000)

Randomized Next-Best View Planning (Gonzalez, 2000)

Randomized Next-Best View Planning (Gonzalez, 2000)

Randomized Next-Best View Planning (Gonzalez, 2000)

Randomized Next-Best View Planning (Gonzalez, 2000)

Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

Reconfiguration Planning for Modular Robots Xerox, Parc Mark Yim, 1999

Planning Minimally Invasive Surgery Procedures Amidst Soft Tissue Structures

Truly Autonomous Interactive Digital Actors with Nice-Looking Motions A Bug’s Life (Pixar/Disney) Toy Story (Pixar/Disney) Tomb Raider 3 (Eidos Interactive)Final Fantasy VIII (SquareOne)The Legend of Zelda (Nintendo) Antz (Dreamworks)

Generating Energetically Plausible Docking and Folding Motions of Proteins

Conclusion u Over the last decade there has been tremendous progress in motion planning and its application u Though motion planning originated in robotics, applications are now very diverse: design, manufacturing, graphic animation, video games, surgery, biology, etc… u Most future problems in motion planning are likely to be motivated by applications that are regarded today as non-robotics applications