Research Associate Computing, Engineering & Physical Sciences University of Central Lancashire, UK Dr John Cartlidge 5th August 2008John Cartlidge: ALife XI, Winchester, UK 1 Dynamically adapting parasite virulence to combat coevolutionary disengagement

John Cartlidge: ALife XI, Winchester, UK2 5th August 2008 Synopsis Disengagement in coevolutionary systems Review Reduced Virulence (RV) Analysis of RV in Counting Ones domain Present Dynamic Virulence (DV), a novel method for adapting Virulence online Summary/Conclusions

John Cartlidge: ALife XI, Winchester, UK3 5th August 2008 Disengagement Competitive Coevolutionary Systems  Relative fitness assessment through self-play  Fitness varies as opponents vary in ability Relativity leads to Disengagement  Occurs when one population gets the “upper hand”  Can’t discriminate individuals  no selection pressure Occurs when competitors are badly matched  Suits of armour and nuclear weapons  There must be no outright winner

John Cartlidge: ALife XI, Winchester, UK4 5th August 2008 Reduced Virulence (RV) Cartlidge, J. & Bullock, S. (2002, 2004) Reward competitors that sometimes lose RV Fitness Transform f(x,v) = 2x ∕ v – x 2 ∕ v 2 virulence: 0.5 ≤ v ≤ 1.0 relative score: x

John Cartlidge: ALife XI, Winchester, UK5 5th August 2008 RV: An illustrative Example Selection only. No mutation. Linear fitness ranking Population B has an innovation (20) not found in A Trade-off between engagement and innovation loss V = 1 (standard)V = 0.75V = 0.5 Selection drives pop B to 20 causing disengagement Pop B drops genotype 20 and remains engaged at 19 Lots of innovation loss as populations move to 12

John Cartlidge: ALife XI, Winchester, UK6 5th August 2008 Symmetry Mutation introduces genetic novelty Symmetric system with unbiased mutation profile  Populations have equal chance of +/– mutation  Neither population has an advantage

John Cartlidge: ALife XI, Winchester, UK7 5th August 2008 Asymmetry Here population B has a favourable mutation bias  A finds it harder to discover +ve/beneficial genetic innovations Disengagement is exacerbated by asymmetry  In genetic representations, genotype-phenotype mappings, genetic operators, interaction rules, location in genotype space, etc.

John Cartlidge: ALife XI, Winchester, UK8 5th August 2008 Couting Ones Watson & Pollack, GECCO 2001 Two populations of binary strings Goal: evolve as many 1s as possible Asymmetrical bias controlled by varying mutation bias of one population (parasites) When is it best to reduce virulence?

John Cartlidge: ALife XI, Winchester, UK9 5th August 2008 Virulence ‘Sweet-Spot’ Low bias requires high virulence for both populations As bias increases, want progressively lower parasite V Parasite virulence Host virulence Parasite Bias / Asymmetry Maximums Engagement ‘Sweet-Spot’

John Cartlidge: ALife XI, Winchester, UK10 5th August 2008 Choosing RV Value Problem:  How do we know a priori what the asymmetry is likely to be?  Is asymmetry is likely to remain fixed? Solution:  Adapt virulence dynamically during runtime

John Cartlidge: ALife XI, Winchester, UK11 5th August 2008 Dynamic Virulence (DV) Reinforcement learning approach:  Value(t+1)  Value(t) + LearningRate [Target(t) – Value(t)] Each generation, t, update virulence, V t  ∆V t = ρ(1 − X t ∕φ)(1) X t : Mean relative score of population at time t φ: Target mean relative score of population ρ: Acceleration (rate of change of virulence)  Μ t = μΜ t-1 + (1−μ)∆V t (2) μ: Momentum, Μ 0 = V 0  if μ = 0, then  t, Μ t = ∆ V t  no momentum  if μ = 1, then  t, Μ t = V 0  fixed virulence  V t+1 = V t +Μ t (3) 0 ≤ φ, ρ, μ ≤ 1

John Cartlidge: ALife XI, Winchester, UK12 5th August 2008 Evolving φ, ρ, μ 30 runs. Mean value of parameter in population each generation. Bias fixed for each evaluation 15 runs. Mean value of parameter in population each generation. Bias varying during each evaluation

John Cartlidge: ALife XI, Winchester, UK13 5th August 2008 DV Performance Performance of DV in the Counting Ones domain DV Parameters: φ = 0.56; ρ = ; μ = 0.3  180/180 successful runs. 31/135,000 disengaged generations Compare with maximum virulence  79/180 successful runs. 68,900 disengaged generations Successful runs using fixed virulence (total 180 runs) Parasite Bias / Asymmetry Parasite Bias / Asymmetry Fixed Virulence Dynamic Virulence

John Cartlidge: ALife XI, Winchester, UK14 5th August 2008 DV in Action

John Cartlidge: ALife XI, Winchester, UK15 5th August 2008 Lessons for epidemiology? Can we use DV for modelling virulence in natural systems? Can we translate ideas of RV to the natural world for control of infectious diseases?  Rather than attack parasites and encourage an arms-race, creating ‘super-bugs’, can we take a reduced virulence approach?  E.g.: ‘Scientists create GM mosquitoes to fight malaria and save thousands of lives’ (Guardian 2005) ‘Plan to breed and sterilize millions of male insects’ Project ‘almost ready for testing in wild’

John Cartlidge: ALife XI, Winchester, UK16 5th August 2008 Summary / Conclusions Disengagement is problematic and is exacerbated by asymmetry Reducing virulence helps to promote engagement As asymmetry increases, virulence should fall Its hard to know a priori what virulence level to set DV is able to adapt virulence during evolution to find the best value DV has been shown to vastly outperform fixed virulence (and standard virulence) in the Counting Ones domain

John Cartlidge: ALife XI, Winchester, UK17 5th August 2008 Further Reading Cartlidge & Bullock (2002) CEC, p.1420, IEEE Press Cartlidge & Bullock (2003) ECAL, p.299, Springer Verlag Cartlidge & Bullock (2004) Evolutionary Comp., 12, p.193 Cartlidge (2004) PhD Thesis, University of Leeds Dr John Cartlidge, Research Associate University of Central Lancashire