Something about Building Block Hypothesis Ying-Shiuan You Taiwan Evolutionary Intelligence LAB 2009/10/31

Adaptive Capacity Some population based search algorithm called “adaptive.” – The avg. fitness of the populations “grows” as generation. This feature of GA is called “adaptive capacity.” – Building Block Hypothesis is currently dominant explanation. 2

The Schema Theorem If m(H,t+1) > m(H,t), the schema H grows. Lower-bound estimation of schema growth. Consider only destructive forces. Minimal, sequential, superior (ms 2 ) schemata grow. Identifies building blocks of a good solution. Holland (1975), Adaptation in Natural and Artificial Systems, The MIT Press

Building Block Hypothesis Goldberg’s words, “… we construct better and better strings from the best partial solutions of past samplings”(Goldberg, 1989, p. 41) “…a genetic algorithm seek near optimal performance through the juxtaposition of short, low- order, high-performance schemas”(Goldberg, 1989) 4 Goldberg, David E (1989), Genetic Algorithms in Search, Optimization and Machine Learning, Kluwer Academic Publishers, Boston, MA.

Two Landscape Features of BBH The presence of short, low-order, highly fit schemas, i.e. building block. The presence of “stepping stone” solutions which combine BBs to create even higher fitness schemas. 5 Stephanie Forrest & Melanie Mitchell (1993), Relative building-block fitness and building-block hypothesis.

Nearly Decomposable Problems We do not want to solve every problem. OneMax NIAH Too simple: simple heuristic, hill climbing Too difficult: enumeration Nearly decomposable problems Tain-Li Yu’s slide on 2006 GA course

Fully Deception Low-order estimates mislead GA. x* = 111: f 111 > f i, i ≠ 111. Require complementary schemata better than competitors. Tain-Li Yu’s slide on 2006 GA course

Trap Function Ackley, Local searcher would go to the wrong optima. In general: to be deceptive. Goldberg & Deb (1993). Analyzing Deception in Trap Functions. Tain-Li Yu’s slide on 2006 GA course

Theories Develop on m-k decomposable Convergence Time (Thierens & Goldberg, 1994) Population size – BB supply (Goldberg et al. 2001) – Decision making (Goldberg et al. 1992) – Decision making + supply (Harik et al. 1997) – Model building (Yu et al. 2007) 9 Reference: Thierens & Goldberg (1994), Convergence models of genetic algorithm selection schemes. Goldberg et al (2001), On the supply of building blocks. Goldberg et al (1992), Genetic algorithms, noise, and the sizing of populations. Harik et al (1997), The gambler's ruin problem, genetic algorithms, and the sizing of populations. Yu et al (2007), Population Sizing for entropy-based model building in genetic algorithms.

GA Design Theory Goldberg, – Know what GA processes: Building blocks (BBs). – Ensure BB growth. – Know BB challengers. – Ensure BB supply. – Ensure BB speed. – Ensure good BB decisions. – Ensure good BB mixing (exchange). Goldberg (2002), Design of Innovation.

Skepticism of the BBH Not realistic BB exist? – In real problem, it’s more likely existing BBs. No overlap BB? – It ‘s more likely exist! But too hard to analysis and design at this time. How about real-value? – Too hard to develop theories on real number. 11

Skepticism of the BBH (cont’d) Weak theoretical foundations. (Wright et al. 2003) “The various claims about Gas that are traditionally made under the name of the building block hypothesis have, to date, no basis in theory, in some cases, are simply incoherent.” 12 Reference: Wright et al. (2003), Implicit Parallelism.

Skepticism of the BBH (cont’d) Experiments side. (Forrest and Mitchell, 1993) “While the disruptive effects that we observed (hitchhiking, premature convergence, etc.) …, there is as yet no theorem associating them with the building-block structure of a given problem.” 13 Reference: Forrest & Mitchell (1993), Relative building-block fitness and building-block hypothesis.

Hitchhiking Once an instance of a high-fitness schema is discovered, the “unfit” material, especially that just next to the fit parts, spread along with the fit material. Slows discovery of good schemas in those positions. Sampling of the different regions is not independent. 14

Hitchhiking (cont’d) 15

Skepticism of the BBH (cont’d) Strong Assumption. (Burjorjee, 2009) Two strong assumption: – Abundant Basic Building Blocks – Heirarchical Synergism Too strong to accept this hypothesis. 16 Reference: Burjorjee (2009), The Fundamental Problem with the Building Block Hypothesis.