1. Instructor: Erol Şahin
Evolutionary Robotics -2 Karl Sim’s seminal work CENG585, Fall
Instructor:
Erol Şahin
Acknowledgement: Most of the slides are adapted from the ones prepared
by Ozgen Canan.

2. Motivation Overcome the trade-off between control and complexity
Motivation
Overcome the trade-off between control and complexity
Co-evolution of morphology and control
Benefit from the fact that survival depends on other creatures in the environment

3. Creature Morphology Genotype A directed graph
Creature Morphology
Genotype
A directed graph
May have all sorts of connections (recursive, cyclic, multiple)
There is a defined root node
Phenotype
Hierarchy of 3D parts

4.
Creature Morphology

5. Creature Morphology Nodes Dimensions Joint-type Joint-limits
Creature Morphology
Nodes
Dimensions
Determine the shape
Joint-type
Defines the number of degrees of freedom of a joint and the movement allowed for each degree of freedom
Different joint-types are rigid, revolute, twist, universal, bend-twist, twist-bend, spherical
Joint-limits
The point beyond restoring spring forces are exerted

6. Creature Morphology Nodes (cont’d) Recursive limit Local neurons
Creature Morphology
Nodes (cont’d)
Recursive limit
Number allowed for replicating oneself
Local neurons
Part of the control system
Control the joints between this node and the parent node
Connections
Define the placement of a child part realtive to its parent
Parameters are position, orientation, scale and reflection

7.
Creature Control
A neural system accepts input sensor values and provides output to the effectors
Signals are continuous variables which may have positive and negative values

8. Creature Control Sensors Joint angle sensors Contact sensors
Creature Control
Sensors
Joint angle sensors
Give the current value for each degree of freedom
Contact sensors
Return 1.0 for if there is contact, -1.0 otherwise
In every face of every part
Photosensors
Provide the cordinates of the normalized light source direction relative to the orientation of the part
They are specialized for the competing behavior
Sensors are included in the control system if they are related with the behavior

9. Creature Control Effectors Each control a degree of freedom of a joint
Creature Control
Effectors
Each control a degree of freedom of a joint
Exert a joint force
Bound with a maximum-strength proportional to the cross sectional area between parts

10. Creature Control Neurons Compute diverse functions
Creature Control
Neurons
Compute diverse functions
Sum, product, divide, sum-threshold, greater-than, sign-of, min, max, abs, if, interpolate, sin, cos, atan, log, expt, sigmoid, integrate, differentiate, smooth, memory, oscillate-wave, oscillate-saw
The number of inputs depends on the function, and it is at most 3
Some functions retain states and give time varying outputs

11. Morphology and Control
Morphology and Control
Neural nets related to the movement of a joint are contained in the related morphological element
Each duplicated segment has a similar control system
Connections are allowed from adjacent parts in the hierarchy

13. Morphology and Control
Morphology and Control
Part-independent neurons
Copied once into the phenotype
Achieve global synchronization or centralized control

14. Unassociated (central)
Unassociated
(central)
body
flipper

15.
Physical Simulation
Articulated body dynamics
Collision tests check the intersection of parts, which are defined as world-space bounding boxes
Connected parts are permitted to interpenetrate to some degree, adjusted boxes are used
Collision response is accomplished by a hybrid model using both impulses and penalty spring forces
Viscosity effect for swimming
For each surface a viscous surface resists the normal component of velocity, proportional to its surface area and normal velocity magnitude

16.
Behavior Selection
Fitness values are assigned after a fixed duration of simulation
Creatures are assigned a zero fitness value, if they have
More than a specified number of parts
Persistent interpenetrating parts
Simulations are stopped beforehand if creatures
Fail to move at all
Move “somewhat” worse than minimum fitness of the previous generation

17. Behavior Selection Swimming Gravity is turned off
Behavior Selection
Swimming
Gravity is turned off
Viscous water resistance effect is on
Swimming speed is used as a fitness value
Distance traveled by the creature’s center of mass per unit time
Continuous movement is rewarded over that from a single initial push (final phase velocity has a stronger weight)

18. Behavior Selection Walking Any kind of land locomotion
Behavior Selection
Walking
Any kind of land locomotion
Gravity is turned on
Viscous water resistance effect is turned off
A static ground plane with friction is added
Horizontal speed is the fitness value
Simple fall over must be eliminated
Initially bring the height of center of mass to a stable minimum with simulation runs, where there are no friction and effector forces

19. Behavior Selection Following (a light source) Photosensors are enabled
Behavior Selection
Following (a light source)
Photosensors are enabled
Average following speed of several runs, in which the location of the light source is changed, is used as a fitness value
Creatures can be evolved for both water and land environments

20. Creature Evolution Initial population Population size is 300
Creature Evolution
Initial population
Random generation from scratch
A genotype from a previous evalutation can be used as a seed
A seed can be designed by hand
Population size is 300
Survival rate is 0.2
If 20% of the population have no non-zero fitness values, the gap is filled with genotype generation
Rest of the population is created by
Mutation, 40%
Mating, 60%
Only survivors generate the offspring, proportional to their fitness values

21. Mutation The internal parameters of each node are changed
Mutation
The internal parameters of each node are changed
Each parameter has a mutation probability
If the parameter has a set of legal values, a new value is picked (boolean, function type)
If the parameter is a scalar, a random number from a Gaussian-like distribution is added to it
A new random node is added (left unconnected)
Connection parameters are changed, connections can be made point elsewhere

22. Mutation Connections are added or removed
Mutation
Connections are added or removed
Only applied to morphological nodes as the neurons require a fixed number of inputs
Unconnected elements are garbage collected
Mutations are inversely proportional with the size of the creature
Nested graphs are mutated in a top-down fashion

23. Mating Align nodes of the parents as a row Crossover Grafting
Mating
Align nodes of the parents as a row
Crossover
Switch the copying source at some crossover points
Start to copy nodes of the first parent to the child
Grafting
Split and merge the parents from a random node

24. Mating Copy the connections and rearrange them if necessary
Mating
Copy the connections and rearrange them if necessary
30% of the next genration is produced by crossover, 30% by grafting
Offspring has fewer mutation probabilities

25. Parallel Implementation
Parallel Implementation
Genetic algorithm is implemented on a Connection Machine CM-5
A single processor performs the genetic algorithm
Fitness tests are performed on other processors
It might take 3 hours for 100 generations on a 32 processor CM-5

26. Results A population converges through homogenity
Results
A population converges through homogenity
However different runs produce different populations
Most successful creatures from many seperate evolutions are inspected

27. Swimming Symmetrical flippers Snake-like multi-segmented creatures
Swimming
Symmetrical flippers
Snake-like multi-segmented creatures

28. Walking Wag up an appendage and rock back and forth
Walking
Wag up an appendage and rock back and forth
Push or pull (inchworm style)
Leg-like appendages

29. Following Jumping Performed in land and water
Jumping
Following
Performed in land and water
Some relative locations make some creatures fail
Some swimming creatures used steering fins, some adjust the angle of their paddles

30. The Contest Photosensors for two different colors
The Contest
Photosensors for two different colors
The source of one color is the center of a cube
The other is located at the center of mass of the opponent
Hence there are “cube sensors” and “opponent sensors”
Light sources are visible even if they are blocked

31. Prevent simple falling over strategies
Prevent simple falling over strategies
Creatures must start with stable states with respect to gravity

32. The Contest The aim is to gain the most control over the cube
The Contest
The aim is to gain the most control over the cube
The creature’s final distances to the cube are used to calculate their fitness values
Let d1 and d2 be the final shortest distances to the cube
f1 = 1.0 +
d2 + d1
d2 - d1
Surrounding the cube is rewarded by assigning shorter distance values, when a creature is touching the cube on the side that opposes its center of mass

33.
The Contest
“All versus best” competition pattern is used for fitness calculations
Both single-species and two-species evolutions are performed

34. Results Rate of evolutionary progress varies
Results
Rate of evolutionary progress varies
Counter strategies were developed
Varying strategies
Push opponent away
Move the cube, then approach
Cover the cube and block opponent’s access
Not all survivors can follow the cube wherever it moves

38. Future Work [for Karl Sims]
Future Work [for Karl Sims]
Experiment with additional types of fitness evaluation method
A changing world can be physically simulated
Use genotypes of creatures that could actually be used as robots
Shape details and external materials, such as fur, hair, eyes tentacles, could be added
Other types of contests may be defined
Cooperative behavior and teams of creatures can be simulated
Larger number of species can be evolved
A more realistic environment in which, many creatures may compete and cooperate, can be built

39.
Discussion
Subjective conclusion, completely commented on behavior techniques, advance in travelling speeds could be given
Many aspects discussed throughout the papers were not mentioned in the results, e.g. competition strategies
Or were not discussed in conclusion, e.g. usage of multi-species

40. Discussion Poor design and evolution of neural system
Discussion
Poor design and evolution of neural system
Topology is not changed
Replicated nodes exhibit similar behavior
Why are so many functions needed?
There are better genetic algorithms for neurons
Does it have to be genetic? Gradient descent may be used in hybrid.
Physical simulation has unrealistic shortcuts, e.g. interpenetrating boxes
High complexity, can trade some aspects of the creatures, like joint-types