Collective Intelligence Week 14: Genetic Programming Old Dominion University Department of Computer Science CS 795/895 Spring 2009 Michael L. Nelson <mln@cs.odu.edu> 4/15/09

Genetic Programming vs. Genetic Algorithms Genetic algorithms: population = parameters for an algorithm Genetic programming: population = algorithm itself + its parameters

Programs As Trees def func(x,y) if x>3: return y + 5 else:

Trees in Python wrapper function; specifies the function, class fwrapper: def __init__(self,function,childcount,name): self.function=function self.childcount=childcount self.name=name class node: def __init__(self,fw,children): self.function=fw.function self.name=fw.name self.children=children def evaluate(self,inp): results=[n.evaluate(inp) for n in self.children] return self.function(results) class paramnode: def __init__(self,idx): self.idx=idx return inp[self.idx] class constnode: def __init__(self,v): self.v=v return self.v wrapper function; specifies the function, the # of parameters it takes and the name of the function functions with child nodes; evaluates child nodes and applies the function to their result returns the parameter passed to it returns a constant

Some Functions addw=fwrapper(lambda l:l[0]+l[1],2,'add') subw=fwrapper(lambda l:l[0]-l[1],2,'subtract') mulw=fwrapper(lambda l:l[0]*l[1],2,'multiply') def iffunc(l): if l[0]>0: return l[1] else: return l[2] ifw=fwrapper(iffunc,3,'if') def isgreater(l): if l[0]>l[1]: return 1 else: return 0 gtw=fwrapper(isgreater,2,'isgreater') flist=[addw,mulw,ifw,gtw,subw]

An Example Tree >>> import gp >>> exampletree=gp.exampletree() >>> exampletree.evaluate([2,3]) 1 >>> exampletree.evaluate([5,3]) 8 >>> exampletree.evaluate([10,2]) 7 >>> exampletree.evaluate([2,10]) >>> exampletree.display() if isgreater p0 3 add p1 5 subtract 2 def exampletree(): return node(ifw,[ node(gtw,[paramnode(0),constnode(3)]), node(addw,[paramnode(1),constnode(5)]), node(subw,[paramnode(1),constnode(2)]), ] )

Some Random Trees >>> random1=gp.makerandomtree(2) >>> random1.evaluate([7,1]) 1 >>> random1.evaluate([2,4]) 4 >>> random2=gp.makerandomtree(2) >>> random2.evaluate([7,1]) 9 >>> random2.evaluate([2,4]) >>> random1.display() p1 >>> random2.display() subtract multiply isgreater p0 7 -5 -20 Some Random Trees # pc = # of parameters for the tree # maxdepth = how deep the tree can go # fpr = probability the new node is a function node # ppr = probability the new node, if it’s not a # function node, is a parameter node # def makerandomtree(pc,maxdepth=4,fpr=0.5,ppr=0.6): if random()<fpr and maxdepth>0: f=choice(flist) children=[makerandomtree(pc,maxdepth-1,fpr,ppr) for i in range(f.childcount)] return node(f,children) elif random()<ppr: return paramnode(randint(0,pc-1)) else: return constnode(randint(0,10))

Solving a Real Problem def hiddenfunction(x,y): return x**2+2*y+3*x+5 def buildhiddenset(): rows=[] for i in range(200): x=randint(0,40) y=randint(0,40) rows.append([x,y,hiddenfunction(x,y)]) return rows

Testing Our Random Programs >>> hiddenset=gp.buildhiddenset() >>> gp.scorefunction(random1,hiddenset) 128596 >>> gp.scorefunction(random2,hiddenset) 148792 >>> random3=gp.makerandomtree(2) >>> random3.display() p1 >>> gp.scorefunction(random3,hiddenset) >>> random4=gp.makerandomtree(2) >>> random4.display() subtract 10 p0 >>> gp.scorefunction(random4,hiddenset) 134619 >>> random5=gp.makerandomtree(2) >>> random5.display() add 9 1 >>> gp.scorefunction(random5,hiddenset) 126998 >>> def scorefunction(tree,s): dif=0 for data in s: v=tree.evaluate([data[0],data[1]]) dif+=abs(v-data[2]) return dif

Mutation 1

Mutation 2 def mutate(t,pc,probchange=0.1): if random()<probchange: return makerandomtree(pc) else: result=deepcopy(t) if hasattr(t,"children"): # book has: if isinstance (t,node) result.children=[mutate(c,pc,probchange) for c in t.children] return result

Mutation Example >>> random5.display() add subtract 9 1 p1 >>> muttree=gp.mutate(random5,2) >>> muttree.display() if p0 isgreater 2 multiply 3 7 >>> >>> random1.display() p1 >>> muttree=gp.mutate(random1,2) >>> muttree.display() >>> random2.display() subtract multiply isgreater p0 7 4 >>> muttree=gp.mutate(random2,2) if

Crossover def crossover(t1,t2,probswap=0.7,top=1): if random()<probswap and not top: return deepcopy(t2) else: result=deepcopy(t1) if hasattr(t1,'children') and hasattr(t2,'children'): result.children=[crossover(c,choice(t2.children),probswap,0) for c in t1.children] return result

Crossover Example >>> random1=gp.makerandomtree(2) >>> random1.display() add if subtract multiply 8 4 p0 isgreater p1 10 7 >>> random2=gp.makerandomtree(2) >>> >>> random2.display() 5 >>> cross=gp.crossover(random1,random2) >>> cross.display() (cross is random1 unmodified) Crossover Example >>> random3=gp.makerandomtree(2) >>> random3.display() p1 >>> cross=gp.crossover(random1,random3) >>> cross.display() (cross is random1 unmodified) >>> random4=gp.makerandomtree(2) >>> random4.display() subtract multiply 10 2 >>> cross=gp.crossover(random1,random4) add >>>

Genetic Programming Code def evolve(pc,popsize,rankfunction,maxgen=500, mutationrate=0.1,breedingrate=0.4,pexp=0.7,pnew=0.05): # Returns a random number, tending towards lower numbers. The lower pexp # is, more lower numbers you will get def selectindex(): return int(log(random())/log(pexp)) # Create a random initial population population=[makerandomtree(pc) for i in range(popsize)] for i in range(maxgen): scores=rankfunction(population) print scores[0][0] if scores[0][0]==0: break # The two best always make it newpop=[scores[0][1],scores[1][1]] # Build the next generation while len(newpop)<popsize: if random()>pnew: newpop.append(mutate( crossover(scores[selectindex()][1], scores[selectindex()][1], probswap=breedingrate), pc,probchange=mutationrate)) else: # Add a random node to mix things up newpop.append(makerandomtree(pc)) population=newpop scores[0][1].display() return scores[0][1] pc = parameters for tree popsize = how many algorithms maxgen = how many iterations to run mutationrate = probability of a mutation breedingrate = probability of crossover pexp = rate of decline in accepting lower ranked algorithms (high = stringent) (listed as probexp on p. 266) pnew = probability of a new algorithm being introduced (listed as probnew on p. 266) cf. “Shuffling a Stacked Deck: The Case for Partially Randomized Ranking of Search Engine Results” http://oak.cs.ucla.edu/~cho/papers/cho-shuffle.pdf

Evolution Runs >>> rf=gp.getrankfunction(gp.buildhiddenset()) >>> gp.evolve(2,500,rf,mutationrate=0.2,breedingrate=0.1,pexp=0.7,pnew=0.1) 21166 5288 5282 5205 5122 5110 (see the rest at: http://mln-web.cs.odu.edu/~mln/cs895-s09/chapter11/evolve-1.txt ) >>> import gp 28328 16750 7046 5264 4183 2723 (see the rest at: http://mln-web.cs.odu.edu/~mln/cs895-s09/chapter11/evolve-2.txt )

Grid War idea: 1. use same trees as before modulo their result by 4 to get values of {0,1,2,3} 2. moves interpreted as: 3 1 rules: move in N/S/E/W directions only can’t move off the edge of the grid can’t make same move twice in a row capture the enemy by moving onto their square

Grid War, Two Players >>> p1=gp.makerandomtree(5) >>> p2=gp.makerandomtree(5) >>> gp.gridgame([p1,p2]) 1 >>> >>> p1.display() p4 >>> p2.display() 6 >>> p1=gp.makerandomtree(5) >>> p1.display() multiply add isgreater subtract p4 p0 7 p2 >>> p2=gp.makerandomtree(5) >>> p2.display() 10 p1 >>> gp.gridgame([p1,p2]) 1 >>> p2=gp.makerandomtree(5) >>> p2.display() if 7 p1 >>> gp.gridgame([p1,p2]) 1 isgreater 8 add p0 2 p2 >>> returns 0 if player 1 wins, 1 if player 2 wins, -1 if tie why is player 2 always winning? player 1 making the same move twice? why 5 parameters here?

Grid War Tournament >>> import gp >>> winner=gp.evolve(5,100,gp.tournament,maxgen=50) 8 32 94 96 88 [deletia] 68 72 58 78 76 52 subtract add p4 1 results at: http://mln-web.cs.odu.edu/~mln/cs895-s09/chapter11/ >>> winner=gp.evolve(5,100,gp.tournament,maxgen=20) 8 54 94 108 58 60 [deletia] 46 52 add if 3 subtract p4 p2 isgreater [even more deletia] >>> winner=gp.evolve(5,100,gp.tournament,maxgen=100) 28 30 44 86 96 78 106 112 94 76 98 80 84 68 [deletia]

Human (O) vs. Computer (X) # winner is from 100 iteration run on previous slide >>> gp.gridgame([winner,gp.humanplayer()]) . . . . . . X . . O . . Your last move was -1 2 3 1 Enter move: 0 . O X . Your last move was 0 Enter move: 3 # let the computer win so it won’t go home >>> gp.gridgame([winner,gp.humanplayer()]) . . . . . X . . . . . O Your last move was -1 2 3 1 Enter move: 3 . . X . Your last move was 3 # bounce back and forth until tie >>> gp.gridgame([winner,gp.humanplayer()]) . . . . . . . O . X . . Your last move was -1 2 3 1 Enter move: 2 . . O . . . X . Your last move was 2 Enter move: 3 [see the rest at: http://mln-web.cs.odu.edu/~mln/cs895-s09/chapter11/human-vs-computer-tie.txt ]

H vs. C, with “wrap around” on grid def gridgame(p): # Board size max=(3,3) # Remember the last move for each player lastmove=[-1,-1] # Remember the player's locations location=[[randint(0,max[0]),randint(0,max[1])]] # Put the second player a sufficient distance from the first location.append([(location[0][0]+2)%4,(location[0][1]+2)%4]) # Maximum of 50 moves before a tie for o in range(50): # For each player for i in range(2): locs=location[i][:]+location[1-i][:] locs.append(lastmove[i]) move=p[i].evaluate(locs)%4 # You lose if you move the same direction twice in a row if lastmove[i]==move: return 1-i lastmove[i]=move if move==0: location[i][0]-=1 # Wrap around board if location[i][0]<0: location[i][0]=max[0] if move==1: location[i][0]+=1 if location[i][0]>max[0]: location[i][0]=0 if move==2: location[i][1]-=1 if location[i][1]<0: location[i][1]=max[1] if move==3: location[i][1]+=1 if location[i][1]>max[1]: location[i][1]=0 # If you have captured the other player, you win if location[i]==location[1-i]: return i return -1 H vs. C, with “wrap around” on grid results at: http://mln-web.cs.odu.edu/~mln/cs895-s09/chapter11/gridwar-wraparound.txt