1. Motion Planning: A Journey of Robots, Digital Actors, Surgical Instruments, Molecules and Other Artifacts Jean-Claude Latombe Computer Science Department Stanford University

2. Goal of Motion Planning u Compute motion strategies, e.g.: –geometric paths –time-parameterized trajectories –sequence of sensor-based motion commands u To achieve high-level goals, e.g.: –go from A to B without colliding with obstacles –assemble product P –build map of environment E –find object O

3. Goal of Motion Planning u Compute motion strategies, e.g.: –geometric paths –time-parameterized trajectories –sequence of sensor-based motion commands u To achieve high-level goals, e.g.: –go from A to B without colliding with obstacles –assemble product P –build map of environment E –find object O

4. Goal of Motion Planning u Compute motion strategies, e.g.: –geometric paths –time-parameterized trajectories –sequence of sensor-based motion commands u To achieve high-level goals, e.g.: –go from A to B without colliding with obstacles –assemble product P –build map of environment E –find object O

5. Basic Problem

6. Extensions to the Basic Problem u Moving obstacles u Multiple robots u Movable objects u Assembly planning u Goal is to acquire information by sensing –Model building –Object finding/tracking u Nonholonomic constraints u Dynamic constraints u Optimal planning u Uncertainty in control and sensing u Exploiting task mechanics (sensorless motions) u Physical models and deformable objects u Integration of planning and control

7. Extensions to the Basic Problem u Moving obstacles u Multiple robots u Movable objects u Assembly planning u Goal is to acquire information by sensing –Model building –Object finding/tracking u Nonholonomic constraints u Dynamic constraints u Optimal planning u Uncertainty in control and sensing u Exploiting task mechanics (sensorless motions) u Physical models and deformable objects u Integration of planning and control

8. Extensions to the Basic Problem u Moving obstacles u Multiple robots u Movable objects u Assembly planning u Goal is to acquire information by sensing –Model building –Object finding/tracking u Nonholonomic constraints u Dynamic constraints u Optimal planning u Uncertainty in control and sensing u Exploiting task mechanics (sensorless motions) u Physical models and deformable objects u Integration of planning and control

9. Extensions to the Basic Problem u Moving obstacles u Multiple robots u Movable objects u Assembly planning u Goal is to acquire information by sensing –Model building –Object finding/tracking u Nonholonomic constraints u Dynamic constraints u Optimal planning u Uncertainty in control and sensing u Exploiting task mechanics (sensorless motions) u Physical models and deformable objects u Integration of planning and control

10. Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

11. Early Work Shakey (Nilsson, 1969): Visibility graph

12. Mathematical Foundations C = S 1 x S 1 Lozano-Perez, 1980: Configuration Space

13. Computational Analysis Reif, 1979: Hardness (lower-bound results)

14. Exact General-Purpose Path Planners - Schwarz and Sharir, 1983: Exact cell decomposition based on Collins technique - Canny, 1987: Silhouette method

15. Heuristic Planners Khatib, 1986: Potential Fields

16. Nonholonomic Robots Laumond, 1986

17. Underactuated Robots Lynch, Shiroma, Arai, and Tanie, 1998

18. Part Orientation Godlberg, 1993

19. Assembly Sequence Planning Wilson, 1994: Non-Directional Blocking Graphs

20. Manipulation Planning Tsai-Yen Li, 1994

21. Deformable Objects Kavraki, Lamiraux, and Holleman 1998

22. Target Finding Guibas, Latombe, LaValle, Lin, and Motwani, 1997 Lin, and Motwani, 1997

23. Integration of Planning and Control Brock and Khatib, 1999

24. Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

25. Robot Programming and Placement David Hsu, 1999

26. Design for Manufacturing and Servicing General Electric General Motors

27. Design of Large Facilities EDF and LAAS-CNRS (MOLOG project), 1999

28. Verification of Building Code Charles Han, 1998

29. Graphic Animation of Digital Actors Koga, Kondo, Kuffner, and Latombe, 1994 The Motion Factory

30. Plan Sense Act Digital Actor = Virtual Robot! Graphic Animation of Digital Actors Kuffner, 1999

31. Vision module image Actor camera image Graphic Animation of Digital Actors u Segment environment u Render false-color scene offscreen u Scan pixels & record IDs Simulated Vision

32. Graphic Animation of Digital Actors

33. Surgical Planning Cyberknife System (Accuray, Inc.) CARABEAMER Planner Tombropoulos, 1997

34. Prediction of Molecular Motions Amit Singh, 1999

35. Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

36. Approaches to Motion Planning u Goal: Answer queries about the connectivity of a certain space (e.g., the collision-free subset of configuration space)

37. Approaches to Motion Planning u Old view (Latombe, 1991): –Roadmaps –Cell decomposition –Potential field

38. Approaches to Motion Planning u Old view (Latombe, 1991): –Roadmaps –Cell decomposition –Potential field u New View (Latombe, 2000): –Finding criticalities –Random sampling

39. Criticality-Based Motion Planning Retraction on Voronoi Diagram (O’Dunlaing and Yap, 1982)

40. Criticality-Based Motion Planning Part orientation (Goldberg, 1993)

41. Criticality-Based Motion Planning Non-Directional Blocking Graphs for assembly planning (Wilson, 1994)

42. Criticality-Based Motion Planning Non-Directional Preimage for landmark-based navigation (Lazanas, 1995)

43. Criticality-Based Motion Planning Non-Directional Preimage for landmark-based navigation (Lazanas, 1995)

44. Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997)

45. Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997)

46. Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997)

47. Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997) Example of an information state = (1,1,0) 0 : the target does not hide beyond the edge 1 : the target may hide beyond the edge

48. Criticality-Based Motion Planning Target finding (Guibas, Latombe, LaValle, Lin, and Motwani, 1997) Lin, and Motwani, 1997) Recontaminated area

49. Criticality-Based Motion Planning u Advantage: –Completeness u Drawbacks: –Computational complexity –Difficult to implement

50. Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

51. Random-Sampling Planning admissible space qbqbqbqb qgqgqgqg milestone [Kavraki, Svetska, Latombe,Overmars, 95] (Probabilistic Roadmap)

52. Motivation Computing an explicit representation of the admissible space is hard, but checking that a point lies in the admissible space is fast

53. Why Does it Work? [Kavraki, Latombe, Motwani, Raghavan, 95] Relation with Art-Gallery problems

54. In Theory, Random-Sampling Planning… u Is probabilistically complete, i.e., whenever a solution exists, the probability that it finds one tends toward 1 as the number N of milestones increases u Under general hypotheses, the rate of convergence is exponential in N, i.e.: Prob[failure] = K exp(-N) u Computational gain is obtained against a “small” loss of completeness

55. Expansiveness of Admissible Space

56. Lookout of F1 The admissible space is expansive if each of its subsets has a large lookout Prob[failure] = K exp(-N)

57. In practice, Random-Sampling Planners… u Are fast u Deal effectively with many-dof robots u Deal well with complex admissibility constraints u Are easy to implement u Have solved complex problems

59. Real-Time Planning with Dynamic Constraints air bearing gaz tank air thrusters obstacles robot (Kindel, Hsu, Latombe, and Rock, 2000)

60. Total duration : 40 sec

61. Interactive Planning of Manipulation Motions Reach Grab Transfer Release Return Kuffner, 1999

62. Random-Sampling Radiosurgical Planning Cyberknife (Neurosurgery Dept., Stanford, Accuray) Tombropoulos, 1997 CARABEAMER Planner

63. Random-Sampling Radiosurgical Planning Dose to the Critical Region Critical Tumor Fall-off of Dose Around the Tumor Dose to the Tumor Region Fall-off of Dose in the Critical Region

64. Random-Sampling Radiosurgical Planning

65. 2000 < Tumor < 2200 2000 < B2 + B4 < 2200 2000 < B4 < 2200 2000 < B3 + B4 < 2200 2000 < B3 < 2200 2000 < B1 + B3 + B4 < 2200 2000 < B1 + B4 < 2200 2000 < B1 + B2 + B4 < 2200 2000 < B1 < 2200 2000 < B1 + B2 < 2200 0 < Critical < 500 0 < B2 < 500 T C B1 B2 B3 B4 T

66. Sample Case 50% Isodose Surface 80% Isodose Surface Conventional system’s plan CARABEAMER’s plan

67. Randomized Next-Best View Planning (Gonzalez, 2000)

68. Randomized Next-Best View Planning (Gonzalez, 2000)

69. Randomized Next-Best View Planning (Gonzalez, 2000)

70. Randomized Next-Best View Planning (Gonzalez, 2000)

71. Randomized Next-Best View Planning (Gonzalez, 2000)

72. Outline u Some historical steps and achievements u Applications u Computational approaches: –Criticality-based motion planning –Random-sampling motion planning u Some challenging problems ahead

73. Reconfiguration Planning for Modular Robots Xerox, Parc Mark Yim, 1999

74. Planning Minimally Invasive Surgery Procedures Amidst Soft Tissue Structures

75. Truly Autonomous Interactive Digital Actors with Nice-Looking Motions A Bug’s Life (Pixar/Disney) Toy Story (Pixar/Disney) Tomb Raider 3 (Eidos Interactive)Final Fantasy VIII (SquareOne)The Legend of Zelda (Nintendo) Antz (Dreamworks)

76. Generating Energetically Plausible Docking and Folding Motions of Proteins

77. Conclusion u Over the last decade there has been tremendous progress in motion planning and its application u Though motion planning originated in robotics, applications are now very diverse: design, manufacturing, graphic animation, video games, surgery, biology, etc… u Most future problems in motion planning are likely to be motivated by applications that are regarded today as non-robotics applications