1. Finding a Sorting Algorithm Using Genetic Programming May 22, 2003 Hyunkoo Jee

2. Defining a Virtual Machine and the Syntax of the Programming Language (1/4) Resources that can be used by the VM Array of registers : A[0], …, A[N-1] Index registers : I1, I2 (initial value : 0) Machine Instructions and the Syntax Syntax One instruction (and its operands) per one line An instruction can have at most two operands Line number is assigned as 0, 1, 2, …

3. Defining a Virtual Machine and the Syntax of the Programming Language (2/4) Example 0: PUT I2 ← I1 1: INC I2 2: EXCH 3: BR (I2 >= N-1) 5 4: BR (I1 < I2) 1 5: INC I1 6: BR (I1 < I2) 0

4. Defining a Virtual Machine and the Syntax of the Programming Language (3/4) Instructions EXCH : (Exchange) First, it compares A[I1] and A[I2]. And, if there is an inversion, it exchanges the values of A[I1] and A[I2]. If “0<=I1<=N-1 and 0<=I2<=N-1” is not satisfied, this instruction doesn’t do anything. INC operand1 : (Increase) It increases the value of the register specified in operand1. Operand1 can be I1 or I2. DEC operand1 : (Decrease) It decreases the value of the register specified in operand1. Operand1 can be I1 or I2. PUT operand1 : It copies the value of a register into another register. Operand1 can be “I2 ← I1” or “I1 ← I2”. BR operand1 operand2 : (Conditional branch) If the condition specified in operand1 is true, jump to the line specified in operand2. operand1 can be… I1 <= 0 I2 <= 0 I1 >= N-1 I2 >= N-1 I1 < I2 I1 > I2 I1 = I2 operand2 is the line number to jump to.

5. Defining a Virtual Machine and the Syntax of the Programming Language (4/4) A priori constraints that the programs must satisfy Use the instruction “EXCH” exactly one time

6. Defining the Cost Function (or Fitness Function) Weighted sum of Inversion = #( {(i,j) | i A[j], 0<=i,j<=N-1} ) Runtime = (how many instructions executed) Length = (how many lines)

7. Evaluation Implement the virtual machine, or an emulating program Execute the individual candidates(=programs) on this VM, and compute the cost function of each candidates. Here we generate T arrays, each array has N elements. Let each individual sort these T arrays. And we compute the average of the T cost values of each individual. If “runtime” exceed some threshold, stop the VM (unless it would run an infinite loop).

8. Defining the Evolutionary Operations 4 Mutation Methods Insert a new line When inserting a line including “BR”, the “operand2” should be in an adequate range. Don’t insert “EXCH” line. Delete a line Don’t delete “EXCH” line. Swap two lines Mutate a line Do not touch the instruction. Mutate the operands only. To mutate an individual candidate, randomly apply these 4 methods, O times.

9. Selection (how to generate the next generation)

10. Summary of the Procedure Make P initial individual candidates At first, each individual(=program) has only one line, i.e. “EXCH” To generate the initial population, apply the mutation methods I times to each individual. Evaluate the cost function Generate T arrays, each of them has N elements. Put these T arrays to each individual in current generation. And evaluate the cost function f for each individual. Selection Classify the individuals in the current generation as : best group(S candidates), worst group(S), ordinary group(P-2S) (according to the cost-function ranking) Make the next generation with : best group, mutated best group, mutated ordinary group (Here we apply the mutation method O times, in order to mutate an individual) Jump to “Evaluate the cost function” again, and repeat

11. Experimental Condition T = 20 How many sample arrays, needed to evaluate the cost function of a candidate? N = 10 How many elements in each array? I = 20 How many times we apply the mutation methods, when we generate the initial population? O = 5 How many times we apply the mutation methods, when we mutate an individual? W runtime = 0.0005 W inversion = 0.9945 W length = 0.005 P = 500 How many individuals in a generation? S = 120 How many individuals in the best group (preserved for the next generation)? = How many individuals in the worst group?

12. Experimental Results

13. After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

14. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

15. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

16. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177 I1 과 I2 를 N-1 까지 증가시키는 루프

17. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

18. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177 I1 을 계속 감소시키면서, A[I1] 과 A[I2] 가 inversion 된 경우 이 둘을 exchange 결국 A[0]~A[I2] 중 가장 큰 값이 A[I2] 로 들어 가게 된다

19. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

20. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

21. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

22. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

23. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177 I1 을 계속 감소시키면서, A[I1] 과 A[I2] 가 inversion 된 경우 이 둘을 exchange 결국 A[0]~A[I2] 중 가장 큰 값이 A[I2] 로 들어 가게 된다

24. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

25. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

26. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

27. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

28. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177 I1 을 계속 감소시키면서, A[I1] 과 A[I2] 가 inversion 된 경우 이 둘을 exchange 결국 A[0]~A[I2] 중 가장 큰 값이 A[I2] 로 들어 가게 된다

29. Experimental Results After the 2000th generation, the best individual doesn't change. The best individual 0: DEC I2 1: BR (I2 <= 0) 12 2: PUT I1 ← I2 3: DEC I1 4: EXCH 5: BR (I1 <= 0) 0 6: DEC I1 7: BR (I1 < I2) 4 8: INC I1 9: PUT I2 ← I1 10: BR (I1 >= N-1) 3 11: INC I1 12: BR (I1 > I2) 8 Runtime = 224, Inversion = 0, Length = 13 Cost Function = 0.177

30. Analysis of the Results selection sort (putting the correct value from right to left) Time complexity : O(N 2 ) Not the optimal solution, but not so bad Diversity preserved It is important to define the virtual machine and its instructions adequately Future works Using crossover Using simulated annealing Using general x86 machine instruction modular structure (make sub-routines and call them) Recursive call → O(N logN) algorithm Generalization → apply to other problems

31. Conclusion 주어진 문제를 해결하는 알고리즘을 찾기 위하여, EA(Evolutionary Algorithm) 를 이용하여 컴퓨터 프로그램을 진화 시키는 전략 (Genetic Programming) 적절한 알고리즘을 찾아내기 어려운 문제가 주어진 경우, 문제에 어느 정도 접근한 간단한 프로그램을 작성한 뒤에, 이 프로그램을 진화시켜 볼 수 있다. 그러면 문제를 더욱 잘 푸는 새로운 알고리즘을 찾을 수 있 을 것이다. 다항 시간 (polynomail time) 내에 푸는 알고리즘이 없는 NP class 나 그 밖의 문제들을 풀기 위해선, 휴리스틱 (heuristic) 방법을 사용하여 어느 정도 타당한 해를 구하는 방법을 많이 쓴다. 이때 사용하는 휴리스틱 알 고리즘을 잘 선택하는 것이 성능에 큰 영향을 미친다. 이런 경우 역시 EA 를 이용하여 더 좋은 휴리스틱 알고리즘을 찾아낼 수 있을 것이다. 최적화 (optimization) 가 아닌 다양성 (diversity) 을 목적으로 EA 를 사용한 다면, 기대하지 못했던 새로운 기능을 하는 프로그램이 창발 (emergence) 할 수도 있을 것이다.