Route Planning & Simulation Ming C. Lin Department of Computer Science University of North Carolina http://www.cs.unc.edu/~lin http://www.cs.unc.edu/~geom lin@cs.unc.edu

State of Art (I) Wide variety of technology, e.g. JSAF, OneSAF, DARWAR, etc. Existing systems are very powerful and offer Many capabilities Flexibility to incorporate new technologies e.g. GPU-based LoS computations

State of Art (II) Game physics engines provides pseudo physical behavior that varies Online multi-player games becomes increasingly popular

What Changes in Simulations Yesterday – tanks, vehicles, vessels, planes on open terrain and airspace Today – teams of soldiers on foot, civilians, insurgency, cluttered urban scenes, dynamic terrains & civil infrastructures

New Needs Add more realistic and higher fidelity simulations, especially for urban simulations/scenarios. Require modeling/simulation of civilians: many independent agents or “heterogeneous crowds” Physics-based computations: weather effects, deformable robots, explosions, entity simulation, spatialized sound effects, shadows, environmental effects, etc.

Real-time Path Planning for Virtual Agents in Dynamic Environments [Sud et al.; IEEE VR 2007]

Multi-Agent Navigation Graph Unified data structure for path planning of multiple agents Computed using 1st and 2nd order Voronoi diagrams 11

Multi-Agent Navigation Graph Unified data structure for path planning of multiple agents Computed using 1st and 2nd order Voronoi diagrams Advantage: Provides pairwise proximity information for all agents simultaneously Compute collision free paths of all agents from single MaNG 12

1st Order Voronoi Diagram (VD1) Agents Static Obstacle

1st Order Voronoi Diagram (VD1) Agents Static Obstacle

2nd Order Voronoi Diagram (VD2)

VD1 and VD2 VD1 VD2

Voronoi Graphs VG1 U VG2

2nd nearest nbr graph 2nd order Voronoi graph 1st order Voronoi graph VG2 captures inter-agent proximity,

MaNG Subset of the 2nd nearest neighbor graph Static Obstacle So we remove edges corresponding to static obstacles. Static Obstacle

Multi-Agent Navigation Graph Unified data structure for path planning of multiple agents Computed using 1st and 2nd order Voronoi diagrams Advantage: Reduce omputation of many 1st order Voronoi graphs to computation of a single MaNG

MaNG: Planner For each agent: Connect agent (source) to VG2 edges

MaNG: Planner For each agent: Connect agent (source) to VG2 edges Connect destination to VG1 edges

MaNG: Planner For each agent: Connect agent (source) to VG2 edges Connect destination to VG1 edges Assign edge weights 8 3 4 2 3 6 5 2 1 7 2 3 1 3

MaNG: Planner For each agent: Connect agent (source) to VG2 edges Connect destination to VG1 edges Assign edge weights Graph search 8 3 4 2 3 6 5 2 1 7 2 3 1 3

MaNG: Planner For each time step: Compute MaNG once Compute paths for all agents from same MaNG

MaNG: Planner 2nd order Voronoi diagram gives proximity to closest obstacle [Sud et al.06] Compute force fields at each step Repulsive forces from closest obstacle

MaNG Computation Computing exact Voronoi diagram difficult Non-linear boundaries High complexity

MaNG Computation Computing exact Voronoi diagram difficult Compute Discrete Voronoi Diagram (DVD) Compute closest site at finite set of points Show grid.

MaNG Computation Computing exact Voronoi diagram difficult Compute Discrete Voronoi Diagram (DVD) Interactive computation using GPU [Sud et al. 06] Culling techniques for fast 2D computation (paper) Please refer to paper for more details

Demos Fruit stealing Crowds in urban environment

Demos Fruit stealing Crowds in urban environment Dynamic goal update Swarming behavior observed Crowds in urban environment

100 Agent simulation at 9 fps Demo: Stealing Fruit - As fruits stolen, they disappear Talk about camera angles 100 Agent simulation at 9 fps 34

Demos Fruit stealing Crowds in small urban environment Dynamic obstacles

100 Agent simulation at 10 fps Demos: Crowd As agents approach the goals, they leave the environment. New agents are also inserted into the environment from the buildings or the road 100 Agent simulation at 10 fps 36

Motivation Traditional approaches Better motion paths Randomized, only considers agent geometry Not realistic motion Better motion paths Requires smoother paths, incorporating physics Start Goal

Traditional: Straight-line links Motivation Traditional approaches Randomized, only considers agent geometry Not realistic motion Better motion paths Requires smoother paths, incorporating physics Traditional: Straight-line links Start Goal

Motivation Traditional approaches Better motion paths Randomized, only considers agent geometry Not realistic motion Better motion paths Requires smoother paths, incorporating physics Better: Smoother path Goal Start

Motivation: Performance Motion planning is exponential in agent degrees of freedom Traditional: Intractable for very high number of joints, many agents, or incorporating dynamics 300 rotational joints 2,500 rotational joints

Applications Autonomous vehicles Search and rescue Medical simulations Virtual prototyping Cable planning Multi-agent planning Character animation Molecular modeling Many more!

Physically-based Motion Planning (PMP) Novel motion planning approach Goal: Generate realistic paths in a practical amount of time for very complex situations Planned trajectory for a deforming sphere

Physically-based Motion Planning Approach: Bias the planning search by artificial workspace forces and agent motion equations Obstacle avoidance Guiding paths Energy functions Agent behavior Sensing information

Planning Architecture

Applied to Deformable Agents Motion equations: Particle motion in a mass-spring lattice Artificial forces: Volume preservation Obstacle avoidance Collision resolution Goal seeking

Algorithm Overview Roadmap Generation Path Estimation Path Query Advance simulation while satisfy constraints min E(x) subject to V(x) ≤  Non-penetration Volume Preservation

Deforming cylinder in a tunnel; agent must deform to exit. Benchmark Scenarios Deforming cylinder in a tunnel; agent must deform to exit. Deforming sphere in a cup; agent deforms to quickly get to and enter the cup

Applied to Deformable Agents

Highly Articulated Chains Motion equations: Articulated body motion equation Artificial forces: Joint configuration Path following Obstacle avoidance Collision resolution

Challenges Chain planning and simulation is expensive Each joint adds exponentially Exploit temporal coherence to reduce adaptively reduce problem dimensionality Goal: Fewer active joints

Reducing dimensionality Based on “adaptive forward dynamics” Adaptively determine which sub-bodies behave most like rigid bodies and rigidify them All joints simulated At each step, 25 most important joints are simulated

System Demonstrations Chain with 300 joints must navigate a sequence of walls Chain with 600 joints must travel through a tunnel

Cable Routing on Bridge A snake robot of 500 links and 500 DOFs with only 70 DOFs used in this bridge scene

Search and Rescue A snake robot with 2,000 joints searches for a cavity in debris and then exits

Pipe inspection A snake robot with 2,000 joints inspects pipes for a leak by coiling around it

Video Demonstration

Catheterization Procedures In medical and surgical procedures, flexible catheters are often inserted in human vessels to Obtain diagnostic information (blood pressure or flow) Enhance imaging with the injection of contrast agents Provide a mechanism to deliver treatment to a specific area In medical and surgical procedures, flexible catheters or small tubes are often inserted in human vessels. Catheters can be used to obtain diagnostic information such as measures of blood pressure or blood flow. They can also be used to deliver contrast agents to enhance imaging techniques or to deliver treatment to a specific location. With this third ability, catheters are becoming important tools to fight cancer. For example,………..

Liver Chemoembolization Catheter is used to inject chemotherapy drugs directly to the blood vessel supplying a liver tumor Catheter is inserted into the femoral artery (near the groin) and advanced into the selected liver artery A fluoroscopic display and the resistance felt from the catheter are used to determine how it should be advanced, withdrawn, or rotated Chemotherapy drugs followed by embolizing agents are injected through the catheter into the liver tumor In a procedure called liver chemoembolization, the application we focus on, a catheter is used to deliver chemotherapy drugs directly to the main blood vessel supplying a liver tumor. The catheter is inserted in the femoral artery and is advanced into the selected liver artery. A fluoroscopic display and the resistance felt by the catheter are used by the doctor to guide the catheter to the target. Once the catheter is in place, chemotherapy drugs are injected through the catheter to the liver tumor. tumor catheter

Motion Planning Application Application to plan the path of a flexible catheter, inserted at the femoral artery, to a specific liver artery supplying a tumor Environment: 3D models of the liver and blood vessels obtained from the 4D NCAT phantom, a realistic computer model of the human body Catheter was modeled as a snake robot with 2500 joints with only 10% of joints simulated to achieve 10x speed up. Given this potential, we apply our motion planning techniques to this area. In this simulation, we used our motion planning technique to plan the path of a flexible catheter, inserted at the femoral artery, to a specific hepatic artery supplying a tumor inside the liver. The environment consisted of 3D models of the liver and blood vessels obtained from the 4D NCAT phantom, a realistic computer model of the human body based on the Visible Human Male CT dataset from the National Library of Medicine The catheter was modeled as a cylinder with a length of 100 cm and a diameter of 1.35 mm. The catheter model was deformable; it was set up to be flexible and snakelike so it could traverse the blood vessels, realistically.

Benchmark: Liver (Courtesy of JHU) A bird’s eye view of the entire live & arteries jhu1 - The environment from the start. A small hole can be seen on the far left artery, which is where the catheter will enter the arteries. jhu2 - A view looking from the top of the environment. Shows the liver and the complex arteries inside. jhu4 - The back side of the environment, zoomed out so that the entire environment is visible A catheter enters the left artery. A closer view of liver and the internal arteries

Applied to multiple agents Motion equation Each agent as constrained particle motion Artificial forces Roadmap following Obstacle/other agent avoidance For human agents: Social forces 10 red agents among moving spheres

Reactive Deformation Roadmaps (RDR) PMP not restricted to agents Reactive roadmaps Adjust to moving obstacles, changing environments Particle motion Physically-based limits on elasticity Link removed

Reactive Deformation Roadmap Blend together ideas from decoupled planning, fast path and roadmap modification, and replanning into a single unified framework Useful for modeling of heterogeneous crowds as well

PMP Conclusions General framework practical planning of complex agents Incorporates kinematic/geometic, dynamic, and mechanical constraints Tighter coupling with agent allows adjustable behavior