1. Spacetime Constraints Revisited J. Thomas Ngo Graduate Biophysics Program Harvard University Joe marks Cambridge Research Lab Digital Equipment Corporation

2. Spacetime Constraints Indirectly controlling physically realistic motion of articulated figures Animator Defines: –Physical structure –Actuators –Criteria for evaluating motion Computer computes optimal trajectory according to specified criteria

3. Globally Optimal Solutions Current SC techniques are local in nature Finding Globally Optimal Solutions to SC Problems is tough –Multimodality Exponential number of possible trajectories –Many may be locally optimal or near optimal –Search-space discontinuities Small change in actuators causes large change in trajectory

4. A Way to Find Globally Optimal Solutions: Global search algorithm –Generates multiple near-optimal trajectories Encode trajectories as sets of stimulus-response behavior rules Genetic algorithm for choosing behavior parameters

5. The Algorithm Dynamics module creates physically correct environment. Behavior module generates creature behaviors Search module finds values for stimulus-response parameters.

6. Dynamics Module Uses forward dynamics –Find motions from forces Articulated figure treated as autonomously deforming object with no DOF –Deformations produced kinematically by stimulus- response control algorithm. –Save CPU cycles.

7. Dynamics Module Friction is static when one joint touches floor Slippage proportional to contact force when two joints touch Friction ?

8. Behavior Module Uses parameterized algorithm based on stimulus and response Stimulus-Response Control Algorithm –Reflexes triggered by conditions sensed in environment Conditions – stimulus functions Reflexes - responses

9. Behavior Module - Responses Response – direction for changing creature shape smoothly. –Time Constant –Set of Target values for creature’s internal angles

10. Behavior Module - Responses Throw values into critically damped equation of motion for each angle time constant target value of angle actual value of angle Creature approaches target shape smoothly.

11. Behavior Module - Stimulus Functions Sense variable –Real-valued function of environment Proprioceptive senses –Um, joint angles Tactile senses –Force exerted by endpoint on floor Kinestheitic sense –Vertical velocity of center of mass Position sense –Vertical position of center of mass

12. Behavior Module - Stimulus Functions Stimulus functions –Scalar function defined over sense space Sense Variables Determined by search module Weight

13. Behavior Module - Stimulus Functions Constants normalized so sense variables fall between 0 and 1 Sensitive Region –Locus of points in sense space where stimulus-function is positive Forms a hyper-rectangle –Dimensions –Centered at

14. Behavior Module - Stimulus Response Functions Set of SR parameters consists of array of SR pairs To generate behavior: Initialize creature state from SC problem description Activate response 0 for t = 1 to T Determine deformation for time t from active response Simulate resulting dynamics for time t Measure sense variables from the environment Identify highest-valued stimulus function Activate corresponding response if stimulus positive end for Active response changed only if highest- valued stimulus function is positive –Causes response to be active for several time step Coherent motion

15. Search Module – Genetic Algorithm Parallel genetic algorithm –Written in C* on Thinking Machines CM-2 with 4096 processors Each processor evaluates one genome per generation

16. Search Module Q: How to find values for stimulus and response parameter? A: Trial and Error. (Genetic Algorithm)

17. Search Module – Genetic Algorithm Pseudocode: do parallel Randomize genome end do for generation = 1 to number_of_generations do parallel Evaluate genome Select mate from another processor Cross genome with mate Mutate genome end do end for

18. Search Module – Genetic Algorithm Parameters of one SR pair in genome

19. Search Module – Genetic Algorithm Randomization –All parameters initially chosen at random Uses probability distribution –Hill climbing to enrich initial gene pool Evaluate initial gene pool Mutate and re-evaluate solution 4 times On each processor choose best out of five –Makes population skewed in favor of multi-step behaviors

20. Search Module – Genetic Algorithm Mate Selection –Processors laid on in imaginary 64x64 grid –Each processor does 10 step random walk –If best solution is itself, no mating –Else mates with best encountered genome Good genes diffuse through population Spreading is not instantaneous so inferior solutions get some processing time Population converged when one population dominates

21. Search Module – Genetic Algorithm Crossover –Typical linear layout for crossover not meaningful –Crossover must be tailored to problem to get good performance

22. Search Module – Genetic Algorithm Mutation –Tailored specifically for the SR representation. One SR Pair creeps One SR Pair randomized form scratch –But at least one corner of sensitive region of new stimulus function must coincide with original sense-space trajectory. »Else new stimulus functions dominate trajectory or don’t do anything

23. Results – Five-Rod Fred Five-Rod Fred –5 consecutively linked rods Middle rods have same mass Terminal rods five times heavier Evaluation Function – Move COM as far right as possible Expected inch-worm behavior But…

24. Results – Five-Rod Fred After 64 generations… Curling leap

25. Results – Five-Rod Fred After 100 generations… Curling leap w/ a roll

26. Results – Five-Rod Fred Final behavior generated by 5 of 10 SR pairs –Two pairs produce initial curling motion –Two pairs produce leaping motion –One pair produces final curled shape

27. Results – Mr. Star-Man Mr. Star-Man –Five rods in star shape –All rods have equal length, mass Evaluation Function – Move COM as far right as possible

28. Results – Mr. Star-Man After 20 generations… Sideways cantering

29. Results – Mr. Star-Man After 37 generations… Ghetto tripping cartwheel

30. Results – Mr. Star-Man After 94 generations… Sideways shuffling

31. Results – Beryl Biped Beryl Biped –Headless 2D humanoid Rigid torso Jointed legs Point feet Rod masses of human proportion Evaluation Function – Move point between feet as far right as possible –If used COM, just fell down

32. Results – Beryl Biped After 100 generations… Skipping, back leg never gets In front of forward leg

33. Results – Beryl Biped Another trial… Skipping walk

34. Results – Beryl Biped Another trial… Weird walk

35. Results GA will fail to find good optima of evaluation function is no good Small changes can restore expected behavior How to change evaluation function to reward grace? ???