1. 1 Evolution in Computation James A. Foster Initiative for Bioinformatics And Evolutionary Studies (IBEST), MRCI, CSDS, and Computer Science University of Idaho

2. 2 Outline of Talk Evolutionary computation (EC) Overview History Applications Optimization Automatic programming Hardware design Problems and promise Code bloat: survival of the fattest Robustness: why do evolved things endure? Hardness: when is EC not efficient? Randomness

3. 3 Two Abstractions Evolution: population based process in which individuals reproduce, inherit information via a noisy transmission process, interact with an environment which defines a fitness gradient on which selection operates. Computation: information transformations induced by finite algorithms in discrete, well-defined steps. Evolution and computation are abstractions, which can be: studied independently of their implementations, implemented in new ways

4. 4 EC Overview

5. 5 EC History ’50 A. Turing ’66 I. Rechenberg, H.P. Schwefel, P. Bienert: fluid dynammics ’66 L. Fogel, A. Owens, M. Walsh: machine intelligence ’75 J. Holland: modeling natural evolution ’85 J. Koza (and others): genetic programming

6. 6 Optimization Problems Minimize sorting network error Maximize “locomotion”* Optimizing a stock market portfolio Fuel optimal two stroke engine Drug design Disease diagnosis *Not done in my lab, but sweet stuff

7. 7 Case Study: Sorting Networks Robustness: an apparent intrinsic property of evolved systems - ability to degrade gracefully with the presence of local failures. (Robustness for free) Is this effect real, and how strong is it? What representations are best for developing robustness in circuits? How does redundancy influence the robustness of evolved circuits? How much is enough? Method: evolve sorting networks, measure performance degradation with point failures in circuits

8. 8 Sorting Nets: Example InputsOutput 0 1 1 0 1 0 1 1 0 0

9. 9 Sorting Nets: Example Inputs 0 Output 01 1 1 0 1 0 0 1 0 1 1 1 1 1 0 0 0 1 1 0

10. 10 CE (1,4)CE (0,3)CE (1,3)CE (1,2)CE (0,4)CE (0,2)CE (2,4)CE (3,4) SN: Linear Individuals 001 100 000 011 001 011 001 010 010 100 000 010 011 100 000 100

11. 11 CE (1,4)CE (0,3)CE (1,3)CE (2,4)CE (0,4)CE (2,3)CE (1,4)CE (1,2) CE (0,3)CE (3,4)CE (0,2)CE (1,3)CE (2,3)CE (1,2)CE (0,3)CE (0,1) SN: Crossover in Arrays CE (1,4)CE (0,3)CE (2,3)CE (1,4)CE (1,2) CE (0,3)CE (3,4) CE (0,2)CE (1,3)CE (2,3) CE (1,2)CE (0,3)CE (0,1)CE (1,3)CE (2,4)CE (0,4)

12. 12 Number of bits correctly sorted, biased exponentially toward low order bits: SN: Fitness Where C(x) i and N(x) i are outputs of wire i from correct circuit C and evolved circuit N (resp.) on input x, and  k is all k bit inputs.

13. 13 SN: Robustness Bitwise stability (bs): percentage of correctly sorted bits over all single point circuit failures BS higher in evolved circuits than hand- designed ones BS low for minimal circuits, high for circuits 5-10% larger than minimum, and decreases for larger circuits

14. 14 SN: Robustness Size Data Here is some data

15. 15 Case Study: Locomotion Karl Sims, Thinking Machines Inc., evolved “organism” descriptions and behaviors.

16. 16 Evolving Software & Hardware Sorting circuits Robot control Machine code Growing chips Image convolutions Feature extraction

17. 17 CE (1,4) CE (0,4)CE (0,3) CE (1,3)CE (1,2) CE (2,4)CE (0,2) CE (3,4) SN: Tree Representation

18. 18 SN: Tree Crossover 1. Randomly select one node on each tree 2. Swap Nodes and subtrees

19. 19 SN: Representation Effects Pass Through Stuck on Zero Stuck on One MSN Tree Linear * * * * *** ***** ** ***** All errors

20. 20 SN: Evolving Physical Circuits Cells ….. 64 0 1 1 0 0 0 1 1

21. 21 Evolving Circuits: More Results 17 electrical circuits evolved to date which infringe on patents or rediscover prior best known designs Filters, thermometers, amplifiers Arithmetic circuits Controllers (Higuchi’s prosthetic arm, missile guidance) Image convolutions and classification

22. 22 Case Study: Image Convolutions We evolved FPGA circuits to produce convolutions on laser diffraction patterns, pre-computed on special hardware.

23. 23 Case Study: Regression Problem: given target function, evolve symbolic functional representation to approximate it. Fitness: square of errors on several points. GP Demo by GP Demo by Hans U. Gerber, Swiss Federal Inst. Of Tech.

24. 24 EC: Limitations & Advantages Robustness: inherent advantage of evolution Survival of the fattest Hardness: limitations from complexity theory, VLSI design, information theory (Pseudo) randomness

25. 25 Survival of the Fattest Variable size genomes grow without correlation to fitness Protective neutral code Removal bias Combinatorics (in trees) Given insufficient selective pressure for parsimony Only parsimony pressure helps “Liposuction” does not Trees converge toward a fixed shape

26. 26 Concluding Thoughts Bring more biology into engineering and Computer Science—it works Bring more engineers and computer scientists into biology—it works There are soooooo many interesting questions, and soooooo much data And so very many interesting people Gee, this is fun!

27. 27 Acknowledgements John Dickinson Jim Frenzel Deb Frincke Larry & Millie Johnston Steve McGrew Holly Wichman IBEST BMDO DOD/OST NIH/NIGMS John Cavalieri Bill Danielson Joe Dumoulin Brad Harvey Kosuke Imamura Jason Masner Mark Meysenburg Bart Rylander Jackie Shoaf