1. Machine Learning Evolutionary Algorithm (2)

2. Evolutionary Programming

3. EP quick overview Developed: USA in the 1960’s Early names: D. Fogel
Typically applied to:
traditional EP: machine learning tasks by finite state machines
contemporary EP: (numerical) optimization
Attributed features:
very open framework: any representation and mutation op’s OK
crossbred with ES (contemporary EP)
consequently: hard to say what “standard” EP is
Special:
no recombination
self-adaptation of parameters standard (contemporary EP)

4. Evolutionary Programming (EP)
There is no fixed structure for representation.
There is only mutation operation, and cross-over is not used in this method.
Each child is determined by its parent in a way of mutation.
So, we can conclude that there are three steps:
Initialize population and calculate fitness values for initial population
Mutate the parents and generate new population
Calculate fitness values of new generation and continue from the second step.

5. EP technical summary tableau
Representation
Finite State Machine
Recombination
None
Mutation
Gaussian perturbation
Parent selection
Deterministic
Survivor selection
Probabilistic (+)
Specialty
Self-adaptation of mutation step sizes (in meta-EP)

6. Prediction by finite state machines
Finite state machine (FSM):
States S
Inputs I
Outputs O
Transition function  : S x I  S x O
Transforms input stream into output stream
Can be used for predictions, e.g. to predict next input symbol in a sequence

7. FSM example Consider the FSM with: S = {A, B, C} I = {0, 1}
O = {a, b, c}
 given by a diagram

8. FSM as predictor Consider the following FSM Task: predict next input
Quality: % of in(i+1) = outi
Given initial state C
Input sequence
Leads to output
Quality: 3 out of 5

9. Introductory example: evolving FSMs to predict primes
P(n) = 1 if n is prime, 0 otherwise
I = N = {1,2,3,…, n, …}
O = {0,1}
Correct prediction: outi= P(in(i+1))
Fitness function:
1 point for correct prediction of next input
0 point for incorrect prediction
Penalty for “too much” states

10. Introductory example: evolving FSMs to predict primes
Parent selection: each FSM is mutated once
Mutation operators (one selected randomly):
Change an output symbol
Change a state transition (i.e. redirect edge)
Add a state
Delete a state
Change the initial state

11. Genetic Programming

12. THE CHALLENGE
"How can computers learn to solve problems without being explicitly programmed? In other words, how can computers be made to do what is needed to be done, without being told exactly how to do it?"
 Attributed to Arthur Samuel (1959)

13. GP quick overview Developed: USA in the 1990’s Early names: J. Koza
Typically applied to:
machine learning tasks (prediction, classification…)
Attributed features:
competes with neural nets and alike
needs huge populations (thousands)
slow
Special:
non-linear chromosomes: trees, graphs
mutation possible but not necessary (disputed!)

14. GP technical summary tableau
Representation
Tree structures
Recombination
Exchange of subtrees
Mutation
Random change in trees
Parent selection
Fitness proportional
Survivor selection
Generational replacement

15. REPRESENTATIONS Decision trees If-then production rules Horn clauses
Neural nets
Bayesian networks
Frames
Propositional logic
Binary decision diagrams
Formal grammars
Coefficients for polynomials
Reinforcement learning tables
Conceptual clusters
Classifier systems

16. A COMPUTER PROGRAM
BBB121

17. Introductory example: credit scoring
Bank wants to distinguish good from bad loan applicants
Model needed that matches historical data ID
No of children
Salary
Marital status
OK?
ID-1 2
45000
Married
ID-2
30000
Single 1
ID-3
40000
Divorced …

18. Introductory example: credit scoring
A possible model:
IF (NOC = 2) AND (S > 80000) THEN good ELSE bad
In general:
IF formula THEN good ELSE bad
Only unknown is the right formula, hence
Natural fitness of a formula: percentage of well classified cases of the model it stands for
Natural representation of formulas (genotypes) is: parse trees

19. Introductory example: credit scoring
IF (NOC = 2) AND (S > 80000) THEN good ELSE bad
can be represented by the following tree
AND S 2
NOC
80000
> =

20. Tree based representation

21. Tree based representation
(x  true)  (( x  y )  (z  (x  y)))

22. Tree based representation
while (i < 20) {
i = i +1 }

23. Tree based representation
In GA, ES, EP chromosomes are linear structures (bit strings, integer string, real-valued vectors, permutations)
Tree shaped chromosomes are non-linear structures
In GA, ES, EP the size of the chromosomes is fixed
Trees in GP may vary in depth and width

24. CREATING RANDOM PROGRAMS
Creation.avi (creation.gif converted to AVI movie file)

25. Tree based representation
Symbolic expressions can be defined by
Terminal set T
Function set F (with the arities of function symbols)
Adopting the following general recursive definition:
Every t  T is a correct expression
f(e1, …, en) is a correct expression if f  F, arity(f)=n and e1, …, en are correct expressions
There are no other forms of correct expressions
In general, expressions in GP are not typed (closure property: any f  F can take any g  F as argument)

26. CREATING RANDOM PROGRAMS
Available functions F = {+, -, *, %, IFLTE}
Available terminals T = {X, Y, Random-Constants}
The random programs are:
Of different sizes and shapes
Syntactically valid
Executable

27. GA flowchart
GP flowchart

28. Mutation
(+ 2 3 (* X 7) (/ Y 5)) + 2 3 * / X 7 Y 5

29. Mutation First pick a random node (+ 2 3 (* X 7) (/ Y 5)) + 2 3 * X 7

30. Mutation
Delete the node and its children, and replace with a randomly generated program
(+ 2 3 (+ (* 4 2) 3) (/ Y 5)) + 2 3 + / * 3 Y 5 4 2

31. Crossover
(+ X (* 3 Y))
(- (/ 25 X) 7) + - X * / 7 3 Y 25 X

32. Crossover (+ X (* 3 Y)) (- (/ 25 X) 7)
Pick a random node in each program + - X * / 7 3 Y 25 X

33. Crossover (+ X (* (/ 25 X) Y)) (- 3 7) Swap the two nodes + - X * 3 7

34. Mutation cont’d Mutation has two parameters:
Probability pm to choose mutation vs. recombination
Probability to chose an internal point as the root of the subtree to be replaced
Remarkably pm is advised to be 0 (Koza’92) or very small, like 0.05 (Banzhaf et al. ’98)
The size of the child can exceed the size of the parent

35. Recombination
Most common recombination: exchange two randomly chosen subtrees among the parents
Recombination has two parameters:
Probability pc to choose recombination vs. mutation
Probability to chose an internal point within each parent as crossover point
The size of offspring can exceed that of the parents

36. Selection Parent selection typically fitness proportionate
Over-selection in very large populations
rank population by fitness and divide it into two groups:
group 1: best x% of population, group 2 other (100-x)%
80% of selection operations chooses from group 1, 20% from group 2
for pop. size = 1000, 2000, 4000, 8000 x = 32%, 16%, 8%, 4%
motivation: to increase efficiency, %’s come from rule of thumb
Survivor selection:
Typical: generational scheme (thus none)
Recently steady-state is becoming popular for its elitism

37. Initialisation Maximum initial depth of trees Dmax is set
Full method (each branch has depth = Dmax):
nodes at depth d < Dmax randomly chosen from function set F
nodes at depth d = Dmax randomly chosen from terminal set T
Grow method (each branch has depth  Dmax):
nodes at depth d < Dmax randomly chosen from F  T
nodes at depth d = Dmax randomly chosen from T
Common GP initialisation: ramped half-and-half, where grow & full method each deliver half of initial population

38. FIVE MAJOR PREPARATORY STEPS FOR GP
Determining the set of terminals
Determining the set of functions
Determining the fitness measure
Determining the parameters for the run
Determining the method for designating a result and the criterion for terminating a run
BBB3666 (converted to BMP from eps)
The following were cut as parameter subpoints so that the text fit on a slide
population size
number of generations
minor parameters

39. Building a Better Mouse
Apply Genetic Programming to the problem of navigating a maze
What are our terminal and function sets?
Function Set = {If-Movement-Blocked, While-Not-At-Cheese*}
Terminal Set = {Move-Forward, Turn-Left, Turn-Right}
* While-Not-At-Cheese will be used exclusively as the root node of the parse tree

40. Building a Better Mouse
How to get the starving mouse to the cheese?
One possible solution:
While not at the cheese
If the way ahead is blocked
Turn left 90 degrees
Otherwise
Move forward
Turn right 90 degrees
Cheese?
Blocked?
Is there a better solution for this maze? How good is this solution?

41. Building a Better Mouse
A fitness function:
Each function and terminal other than the root node shall cost one unit to execute
If the mouse spends more than 100 units, it dies of hunger
The fitness measure for a program is determined be executing the program, then squaring the sum of the total units spent and the final distance from the exit
A lower fitness measure is preferable to a higher fitness measure
Cheese?
Blocked?

42. Building a Better Mouse
While not at the cheese (12996)
If the way ahead is blocked
Turn left 90 degrees
Otherwise
Move forward one space
Turn right 90 degrees

43. Building a Better Mouse
While not at the cheese (12996)
If the way ahead is blocked
Turn left 90 degrees
Otherwise
Move forward one space
Turn right 90 degrees
Mutation:
While not at the cheese (12996)
If the way ahead is blocked
Turn left 90 degrees
Otherwise

44. Building a Better Mouse
While not at the cheese (12996)
If the way ahead is blocked
Turn left 90 degrees
Otherwise
Move forward one space
Turn right 90 degrees
Crossover:
While not at the cheese (11664)
If the way ahead is blocked
Move forward one space
Turn right 90 degrees
Otherwise
While not at the cheese (12996)
Turn left 90 degrees

45. Building a Better Mouse
(after 4202 generations, with 1000 programs per generation)
While not at the cheese
If the way ahead is blocked
Turn right 90 degrees
Move forward one space
Otherwise
Turn left 90 degrees
Fitness measure: 2809
Is this better?

46. Example
We seek to find the Boolean function that returns particular Boolean output values (0 or 1). The Boolean even-k-parity function of k Boolean arguments returns T (true=1) if an even number of its Boolean arguments are T, and otherwise returns F (false =0) (fitness function). Let the input are D0, D1, and D2. The output is S.
Terminal set: D0, D1 and D2
Function set: AND, OR, NAND, and NOR.
Fitness cases: sum of 8-combinations of the three Boolean arguments D0, D1, and D2 (Truth Table) cases do not equals the correct value of the even-3-parity function (error)
construct 4 genes for this problem using Genetic Programming
compute the fitness for these genes
perform one crossover (best two genes) and one mutation operator for worst gene
construct the new population

47. Summary