Randomized Motion Planning: From Intelligent CAD to Digital Actors to Protein Folding Nancy M. Amato Department of Computer Science Texas A&M University

The Alpha Puzzle: A MP Benchmark Objective: Separate the two tubes, one is the ‘robot’ the other is an ‘obstacle’. Motion Planning start goalobstacles (Basic) Motion Planning (in a nutshell): Given a movable object, find a sequence of valid configurations that moves the object from the start to the goal.

A (Dis)Assembly Problem: The Pentamino Puzzle Hard Motion Planning Problems: Intelligent CAD Applications Using Motion Planning to Test Design Requirements: Requirements, such as accessibility for servicing or assembly, were tested with physical “mock ups”. Digital testing saves time and costs, is more accurate, and enables more extensive testing. Maintainability Problems: Mechanical Designs from GE find part removal path

Soccer Ball: 31 dofPolyhedron: 25 dof Hard Motion Planning Problems - Highly Articulated (Constrained) Systems Digital Actors Closed chain constraint Paper Folding

Hard Motion Planning Problems: Deformable Objects Given a deformable object, find a path for the object that can be transformed into a collision free path by deforming the object.

Hard Motion Planning Problems: Flocking: Covering, Homing, Shepherding

Hard Motion Planning Problems - computational biology & chemistry Protein Folding - find folding pathways Drug Design -molecule docking

Outline C-space, Planning in C-space (basic definitions) Probabilistic Roadmap Methods (PRMs) PRM variants (OBPRM, MAPRM) User-Input: Haptically Enhanced PRMs PRMs for more `complex’ robots deformable objects flocking behaviors Computational Biology: protein folding & ligand binding Other Projects & Future Work Neuron PRM - using PRM framework to construct cortical networks! Mobile Robot Navigation and Localization – more PRMs

Configuration Space (C-Space) C-obst C-Space 6D C-space (x,y,z,pitch,roll,yaw) 3D C-space (x,y,z) 3D C-space (  )    “ robot” maps to a point in higher dimensional space parameter for each degree of freedom (dof) of robot C-space = set of all robot placements C-obstacle = infeasible robot placements 2n-D C-space (  1,  1,  2,  2,...,  n,  n )

workspace obstacles transformed to C-obstacles Figure from Latombe’91 C-obstacles

simple 2D workspace obstacle => complicated 3D C-obstacle Figure from Latombe’91 C-obstacles

robot obst x y C-obst robot Path is swept volume Motion Planning in C-space Path is 1D curve Workspace C-space Simple workspace obstacle transformed Into complicated C-obstacle!!

Most motion planning problems of interest are PSPACE-hard [Reif 79, Hopcroft et al. 84 & 86] The best deterministic algorithm known has running time that is exponential in the dimension of the robot’s C-space [Canny 86] C-space has high dimension - 6D for rigid body in 3-space simple obstacles have complex C-obstacles impractical to compute explicit representation of freespace for more than 4 or 5 dof So … attention has turned to randomized algorithms which trade full completeness of the planner for probabilistic completeness and a major gain in efficiency The Complexity of Motion Planning

1. Connect start and goal to roadmap Query processing startgoal Probabilistic Roadmap Methods (PRMs) [Kavraki, Svestka, Latombe,Overmars 1996] C-obst Roadmap Construction (Pre-processing) 2. Connect pairs of nodes to form roadmap - simple, deterministic local planner (e.g., straightline) - discard paths that are invalid 1. Randomly generate robot configurations (nodes) - discard nodes that are invalid C-obst C-space 2. Find path in roadmap between start and goal - regenerate plans for edges in roadmap

PRMs: Pros & Cons PRMs: The Good News 1. PRMs are probabilistically complete 2. PRMs apply easily to high-dimensional C-space 3. PRMs support fast queries w/ enough preprocessing Many success stories where PRMs solve previously unsolved problems C-obst start goal PRMs: The Bad News 1. PRMs don’t work as well for some problems: – unlikely to sample nodes in narrow passages – hard to sample/connect nodes on constraint surfaces Our work concentrates on improving PRM performance for such problems. start goal C-obst

Related Work Probabilistic Roadmap Methods Uniform Sampling (original) [Kavraki, Latombe, Overmars, Svestka, 92, 94, 96] Analysis of Basic PRM [Kavraki/Latombe/Motwani/Raghavan 98] PRM Roadmaps in Dilated Free space [Hsu et al, 98] Gaussian Sampling PRMs [Boor/Overmars/van der Steppen 99] PRM for Closed Chain Systems [Lavalle/Yakey/Kavraki 99] PRM for Flexible Objects [Kavraki/Lamireaux/Holleman 98] Visibility Roadmaps [Laumond et al 99] Using Medial Axis of Workspace [Kavraki et al 99, Lin/Manocha et al 00] Generating Contact Configurations [Xiao et al 99] Related Methods Ariadnes Clew Algorithm [Ahuactzin et al, 92] RRT (Rapidly Exploring Random Trees) [Lavalle/Kuffner 99]

Smarter Sampling Methods Goal: Sample nodes in Narrow Passages OBPRM: Obstacle-Based PRM MAPRM: Medial-Axis PRM

OBPRM: An Obstacle-Based PRM [IEEE ICRA’96, IEEE ICRA’98, WAFR’98] start goal C-obst To Navigate Narrow Passages we must sample in them most PRM nodes are where planning is easy (not needed) PRM Roadmap start goal C-obst Idea: Can we sample nodes near C-obstacle surfaces? we cannot explicitly construct the C-obstacles... we do have models of the (workspace) obstacles... OBPRM Roadmap

OBPRM: Finding Points on C-obstacles Basic Idea (for workspace obstacle S) 1. Find a point in S’s C-obstacle (robot placement colliding with S) 2. Select a random direction in C-space 3. Find a free point in that direction 4. Find boundary point between them using binary search (collision checks) Note: we can use more sophisticated heuristics to try to cover C-obstacle C-obst

PRM vs OBPRM Roadmaps PRM 328 nodes 4 major CCs OBPRM 161 nodes 2 major CCs

OBPRM for Paper Folding Animations - some difficult examples Box: 12 => 5 dofPeriscope: 11 dof Roadmap Statistics 218 nodes, 3 sec Roadmap Statistics 450 nodes, 6 sec

MAPRM: Medial-Axis PRM [IEEE ICRA’99, ACM SoCG’99] Key Observation: We can efficiently retract almost any configuration, free or not, onto the medial axis of the free space without computing the medial axis explicitly. C-obstacle Medial axis Basic Retraction Approach sample point c in C-Space find closest C-obst boundary b if c in collision, push past b until free push c away from b until have two closest points (on medial axis now)

MAPRM: Significance Theorem: Sampling and retracting can increase the #nodes in narrow corridors in a way that is independent of the corridor’s volume (depends on volume that bounds corridor). C-obstacle PRM (uniform sampling) Probability PRM node in corridor C-obstacle MAPRM where maps colliding nodes to closest boundary point Probability MAPRM node in corridor

MAPRM: Current Status Approximate method implemented for higher dimensional C-spaces –Methods to compute, approximately: – clearance in C-space – penetration depth in C-space Now our method of choice for most applications

Hybrid Human/Planner System [IEEE ICRA’01, IEEE ICRA’00, PUG’99] 1.Planner creates initial roadmap 2.User collects approximate path P (has collisions) –a haptic device is ideal for this 3.Planner fixes P (pushes to C-free) –techniques inspired by OBPRM ideal for this Automatic planners are good, but… Enhancing PRMs with Haptic Hints Sometimes can’t find critical configurations obvious to the user C-obstacle approximate path generated by user pushed path generated by planner

PHANToM 1. User attaches haptic device to robot, and moves it around user feels when robot touches obstacles and adjusts trajectory 2. Robot configurations sampled and passed to planner Haptic Interaction: Collecting Paths Current Applications Intelligent CAD Applications Molecule Docking in drug design Animation w/ Deformable Models

Haptic Hints: Results Flange Problem

Problem: Given a deformable object, find a path for the object so that it can achieve the goal by deforming on the way. Deformable Objects Our Approach: Find a path for a rigid version of robot (may have collisions) Deform the robot to avoid the collisions

Goal Original Problem Start Goal Robot Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original Problem Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Goal Original ProblemDeformable Version Given an object which can deform Find a path taking object from start to goal. The object is allowed to deform to avoid collision.

Approximate Solution Enable Penetration –Use approximate C-Space penetration Use scaled robots –More than one scaled model. –Smaller (Bigger ) model needs more (less) deformation. 1.Build roadmap, relaxing collision free requirement 2. Extract Approximate Path l may not be feasible for the rigid robot

Approximate Solution Query The approximate path returned is not the shortest path but the energetically most feasible path (less deformation required)

Improving Solution Bounding Box Deformation Bounding Box Deformation build a 3D voxel bounding box. —Convert it to ChainMail bounding box (3D grid of springs) —Deform ChainMail Bounding box. —Deform objects using Free Form Deformation based of deformation of the bounding box. Deformed ChainMail Bounding Box ObstacleChainMail Box Apply FFD

Group Behaviors Flocking System l What is a flocking system? –System simulating behaviors of groups of objects (e.g. a school of fish, crowds…) – flock formation is selfish l Applications –Computer graphics, VR, games –Robotics –Biological/ecological simulation

Basic Flocking Behavior l Interactive particle system –Particle system has no interactions between particles l Local Information –No central control system (individual-based model) l 3 simple Rules –Separation –Alignment –Cohesion

Problem with Basic flocking l Always emergent behavior –only local information is used –No memory l most creatures are not so stupid. At least they usually have memory. –Can’t do complex tasks, like searching. l Can we have more complex behaviors? –Yes! Using global knowledge. –However, we need to use it very carefully. l this means we don’t want our creatures to be too smart l No creature will have complete/perfect information about global information.

Adaptive Roadmaps l Yes, of course, Roadmaps! –Encodes global information. (e.g. topology) –Facility to access global information. l Adaptive roadmap edge weights l Provides indirect and cheap way to communicate between flockmates. (inspired from ant colony optimization)

Behaviors using Adaptive Roadmap l Using global information encoded in roadmap we can generate “good” complex behaviors. l Roadmaps are generated using MAPRM. l Experiments including: –Homing –Covering –Goal Searching –Shepherding

Application : Homing l Find a path from current position to goal l A very simple case –Showing that we can keep properties of basic flocking behaviors while new behaviors are added. –Comparing with most popular approach (A* search) for this problem. Goal

Flocking Behaviors: Homing Roadmap ApproachA* approach

Flocking Behaviors: Homing - 40 flock members -Environment size: 420 m * 420 m -301obstacles (6 types) -Simulation updated every 100ms 2005 A* 255 # of local minima R

Flocking Behaviors: Covering l Mine sweeping : covering the environment. l A point p is “covered” if p is in the visibility range of one or more flock member. l Similar to ant colony optimization Memory is a list of roadmap nodes that are visited by the boid. Probability of each edge been selected based on its weight. Edge with Smaller weight has higher probability to be selected.

Flocking Behaviors: Covering

- 50 flock members -Environment size: 80 m *100 m -Sensory range: 5 m Scene overview

Flocking Behaviors: Goal Searching l Search an unknown goal, and, once it’s found, all flock members should go there. Probability of each edge been selected based on its weight. Edge with larger weight has higher probability to be selected. Memory is a list of roadmap nodes that are visited by the boid.

Flocking Behaviors: Goal Searching

- 50 flock members -Environment size: 80 m *100 m -Sensory range: 5 m Scene overview

Flocking Behaviors: Homing

Flocking Behaviors: Shepherding

Protein folding is a “grand challenge” problem in biology - the deciphering of the second half of the genetic code. Our focus: “… use the protein’s known 3D structure to predict the kinetics and mechanism of folding” [Munoz & Eaton, PNAS’99] – Study protein folding pathways, folding landscape, and kinetics. – Will assist in understanding folding and function. Protein structure prediction: given a protein’s amino acid sequence, predict its 3D structure. Protein Folding

The parallel: between paper folding and protein folding Folding paths of a periscope and a 10-ALA peptide chain. unfolded folded

Protein Folding [RECOMB’01, IEEE ICRA’01] Secondary Structure  helix  sheet + + variable loops = Tertiary Structure TTCCPSIVARSNFNVCRLPGTPEALCATYTGCIIIPGATCPGDYAN Primary Structure Genomic sequence (nucleotide sequence) Protein sequence (amino acid sequence) Protein structure => function Better Treatments, Better drugs, etc One amino acid Energy minimum The Problems 1.Predict Tertiary Structure from Primary Structure 2.Determine folding pathway to known tertiary structure (our current focus)

Probabilistic Roadmap Methods (PRMs) for Protein Folding (overview) Native state Construct the roadmap: 1. Generate nodes. 2. Connect to form roadmap The Roadmap is like a net being laid down on protein’s potential landscape. A conformation Conformation space Potential Now the roadmap can be used: 1.To find a path 2.To extract multiple paths

Configuration Space (one parameter for each dof) robot (rigid, articulated, etc) Collision Detector collision free paths Conformation Space (phi/psi angles) protein (articulated linkages) Potential Calculation energetically feasible paths Both can use the same motion planning framework in an abstract C-space. PRMs for Protein Folding C-Space Moving Object Model Validity Check Roadmap Construction (A connectivity graph) Query (find valid path)

Previous work l PRM Roadmap approach for protein folding [Song & Amato, RECOMB’01, ICRA’01] –A protein’s folding behavior is determined by its energy landscape. PRM roadmaps approximate such landscapes. –Validation with hydrogen exchange experiments. l Secondary structure formation order. l Other work using PRM roadmap approach: –Protein folding ( [Apaydin et al, ICRA’01, RECOMB’02]) –Ligand binding ( [Singh, Latombe, Brutlag, ISMB’99], [Bayazit, Song, Amato, ICRA’01])

Model of a protein l amino acid: pair of phi/psi angles l protein: a sequence of amino acids. –conformation node is :

Take advantage of the known native state. map the potential landscape/funnel leading to it. sample around it and gradually grow out. generate conformations by randomly selecting phi/psi angles Criterion for accepting a node: Compute potential energy E of each node and retain it with probability P(E): PRM: Node Generation N

l Start with native structure. l Gradually grow out. Denser distribution around native state Native state

PRM: Roadmap Connection 1.Find k closest nodes for each roadmap node 2.Assign edge weight to reflect energetic feasibility: lower weight  more feasible [Singh, Latombe, Brutlag, 1999] Native state

PRMs for Protein Folding: Key Issues Validation –In RECOMB ‘01 (Song & Amato), our results validated with hydrogen exchange experiments. [Li & Woodward 1999] Energy Functions –The degree to which the roadmap accurately reflects folding landscape depends on the quality of energy calculation.

Energy Computation l Potential ( ref. Levitt’83 ) –van der Waals + hydrogen bonds + disulphide bonds + hydrophobic effect –All-atom model l Free Energy ( ref. Fiebig & Dill ’93, Munoz & Eaton’99) where

Results: l Folding Potential Landscape l Secondary structure formation order –timed contact map –experimental validation l Studying Folding Kinetics –2-state folding kinetics –calculation of folding rates –identifying 2-state, 3-state, … k-state kinetics

Potential landscape of a protein l could be very complicated l different proteins  different landscapes  different folding behaviors l a landscape may have several channels –channel A: 2-state kinetics –channel B: 3-state kinetics Native state Conformation space Potential

Distributions for different types: Potential Energy vs. RMSD for roadmap nodes all alphaalpha + betaall beta

Secondary structure formation order l Secondary structure formation order –Timed contact map –Experimental validation (A few in RECOMB’01, more here.)

Timed Contact Map: formation order for a Path protein G (domain B1) (IV:  1-4 )   1-4  3-4 Average T = 142 Formation order: ,  3-4,  1-2,  1-4 residue # 1-2 

Validating Folding Pathways Protein GB1 (56 amino acids) — 1 alpha helix & 4 beta-strands Hydrogen Exchange Results first helix, and beta 3-4 Our Paths 60%: helix, beta 3-4, beta 1-2, beta %: helix, beta 1-2, beta 3-4, beta 1-4 [Li & Woodward 1999] our paths are: from all the nodes with little structure to the native state secondary structure formation order checked on each path w/ timed contact map

Validating Folding Pathways Protein A (60 amino acids) — 3 alpha helices Hydrogen Exchange Results 3 helices form about the same time Our Paths agree, central helix (2) usually last [Li & Woodward 1999]

Validating Folding Pathways CI2 (64 amino acids) — 1 alpha helix and 5 beta strands Hydrogen Exchange Results helix, betas 2 and 3 Our Paths helix, beta 3-4 and 2-3 helix clearly first, then contacts form involving beta 3 and betas 2 and 4 [Li & Woodward 1999]

Secondary structure formation order and validation Proteins primarily from [Munoz & Eaton PNAS’99] for comparison purposes Contact us if you want us to analyze your proteins! PDB nameNum of Residues 2 nd structuresComparison w/ Exp. [Li & Woodward ’99] 1GB1561 alpha + 4 betaAgreed 1BDD603 alphaAgreed 1SHG625 betaN/a 1COA641 alpha + 4betaAgreed 1SRL645 betaN/a 1CSP677 betaN/a 1NYF673 betaN/a 1MJC697 betaN/a 2AIT747 betaN/a 1UBQ761alpha + 5 betaAgreed 1PKS791 alpha + 5 betaN/a 1PBA813 alpha + 3 betaN/a 2ABD865 alphaN/a 1BRN1103 alpha + 7 betaNot sure

Summary: comparison of models for protein folding Folding landscape Trajectory (path #) Path quality Time dependent (running time) Folding kinetics Native state needed Molecular Dynamics NoYes (1)goodYes. (very long) No Monte Carlo NoYes (1)goodYes (very long) No Statistical Model YesNoN/ANo (short)Yes (only average) Yes Our PRM approach YesYes, (many) approximate No (short)Yes, multiple kinetics Yes Lattice model Not used on real proteins

Protein Folding Summary PRM roadmaps approximate folding landscapes Efficiently produce multiple folding pathways –Secondary structure formation order –better than trajectory-based simulation methods, such as Monte Carlo, molecular dynamics Provide a good way to study folding kinetics –multiple folding kinetics in same landscape (roadmap) –natural way to study the statistical behavior of folding –more realistic than statistical models (e.g. Lattice models, Baker’s model PNAS’99, Munoz’s model, PNAS’99 )

Given: an description of a ligand molecule (robot) and a protein (obstacle). Find: a configuration of the ligand near the protein where geometric, electro-static and chemical constraints are satisfied. Ligand Binding [IEEE ICRA`01] protein ligand

Ligand Binding [IEEE ICRA`01] Automated docking algorithms – AutoDock,Dock,FlexX,FLOG,FTDock,Gold, etc. PRM Approach (Singh, Latombe, Brutlag, 1999) –rapidly explores high dimensional space –We use OBPRM: better suited for generating conformations in binding site (near protein surface) Haptic User interaction – haptics (sense of touch) helps user understand molecular interaction – User assists planner by suggesting promising regions, and planner will post-process and ‘improve’

Conclusions Diverse problems can be addressed using appropriate adaptations of PRMs key is defining appropriate models and their C-spaces validation check is very general - ranging from traditional collision detection to potential energy thresholds to... More info and Movies: Joint work with: O. Burchan Bayazit, Guang Song, Jyh-Ming Lien, Shawna Thomas, Steve Wilmarth, and the other members of our group.