1. Biologically Inspired Computation Some of the images in this lecture come from slides for a Course in Swarm Intelligence given at : Lecture 5: Introducing Swarm Intelligence contents: the behavior of natural swarms and flocks -- Reynold’s rules and swarm simulation.

2. But first: Finishing off encodings Information about mandatory reading Information about CW2

3. E.g. encoding a timetable I 4, 5, 13, 1, 1, 7, 13, 2 Generate any string of 8 numbers between 1 and 16, and we have a timetable! Fitness may be + + etc … Figure out an encoding, and a fitness function, and you can try to evolve solutions. montuewedthur 9:00E4, E5 E2E3, E7 11:00E8 2:00E6 4:00E1 Exam1 in 4 th slot Exam2 in 5 th slot Etc …

4. E.g. encoding a timetable II 4, 5, 13, 1, 1, 7, 13, 2 montuewedthur 9:00E4, E5 E3, E7 11:00E8 2:00E6, E2 4:00E1 Use the 4 th clash-free slot for exam1 Use the 5 th clash-free slot for exam2 (clashes with E4,E8) Use the 13 th clash-free slot for exam3 Etc …

5. So, a common approach is to build an encoding around an algorithm that builds a solution Don’t encode a candidate solution directly … instead encode parameters/features for a constructive algorithm that builds a candidate solution E2

6. e.g.: bin-packing – given collection of items, pack them into the fewest possible number of bins

8. Engineering Constructive Algorithms A typical constructive algorithm for bin-packing: Put the items in a specific sequence (e.g. smallest to largest) Initialise solution Repeat nitems times choose next item, place it in first bin it fits (create a new empty bin if necessary) Indirect Encodings often involve using a constructive algorithm,

9. Example using ‘First-Fit Ascending’ (FFA) constructive algorithm for bin-packing FFA

10. First-fit Ascending

16. Example using First-Fit Descending First-fit Descending

17. First-Fit Descending

22. Notes: In other problems, FFA gets better results than FFD There are many other constructive heuristics for bin packing, e.g. Using formulae that choose next item based on distribution of unplaced item sizes and empty spaces... There are constructive algorithms for most problems (e.g. Timetabling, scheduling, etc....) Often, ‘indirect encodings’ for EAs use constructive algorithms. Common approach: the encoding is permutation, and the solution is built using the items in that order READ THE FALKENAUER PAPER TO SEE GOOD EA ENCODING FOR BIN-PACKING

23. Encodings that use constructive algorithms The indirect encoding for timetabling, a few slides ago, is an example. The ‘underlying’ constructive algorithm is: Line up the exams in order e1, e2, … eN Repeat until all exams are scheduled: take the next exam in the list, and put it in the first place it can go without clashes This provides only a single solution, the same every time we run it. This may be very bad in terms of other factors, such as consecutive exams, time allowed for marking, etc. How did we modify it so that it could generate a big space of different solutions?

24. Encodings that use constructive algorithms Line up the exams in order e1, e2, … eN Repeat until all exams are scheduled: take the next exam in the list, and put it in the first place it can go without clashes

25. Encodings that use constructive algorithms Line up the exams in order e1, e2, … eN Repeat until all exams are scheduled: take the next exam in the list, and put it in the first place it can go without clashes

26. Encodings that use constructive algorithms Line up the exams in order e1, e2, … eN Repeat until all exams are scheduled: take the next exam in the list, and put it in the Nth place it can go without clashes The chromosome encodes each of the Ns. The original constructive algorithm corresponds to running the above on the chromosome “111111111….”. We could also engineer the original constructive algorithm into an encoding in a quite different way. How?

27. Encodings that use constructive algorithms Line up the exams in order e1, e2, … eN Repeat until all exams are scheduled: take the next exam in the list, and put it in the first place it can go without clashes

28. Encodings that use constructive algorithms Randomly permute the exams e1, …, eN Repeat until all exams are scheduled: take the next exam in the list, and put it in the first place it can go without clashes This is a fine constructive algorithm, which will provide a different solution depending on the permutation. It is easily used as an encoding: the chromosome provides the permutation.

29. Prim’s algorithm for building the minimal spanning tree (see an earlier lecture) is an example. Djikstra’s shortest path algorithm is also an example. In both of these cases, the optimal solution is guaranteed to be found, since MST and SP are easy problems. But usually we see constructive methods used to give very fast `OK’ solutions to hard problems. Other well known constructive algorithms

30. On engineering constructive methods Some Constructive Heuristics are deterministic. I.e. they give the same answer each time. Some are stochastic – I.e. they may give a different solution in different runs. Usually, if we have a deterministic constructive method such as FFD, we can engineer a stochastic version of it. E.g. instead of choosing the next-lightest item in each step, we might choose randomly between the lightest three unplaced items.

31. Bin packing example direct encoding: 2, 3, 2, 3, 1....  item 1 is in bin 2, item 2 is in bin 3, item 3 is in bin 2, etc... (often a bin will be over capacity, so fitness function will have to include penalties) Bin packing example indirect encoding: candidate solution is a perm of the items, e.g. 4, 2, 1, 3, 5...meaning: First place item 4 in the first available bin it fits in, then place item 2 in the first available... etc.

32. Direct vs Indirect Encodings Direct: straightforward genotype (encoding)  phenotype (actual solution) mapping Easy to estimate effects of mutation Fast interpretation of chromosome (hence speedier fitness evlaluation) Indirect/Hybrid: Easier to exploit domain knowledge – (e.g. use this in the constructive heuristic) Hence, possible to `encode away’ undesirable features Hence, can seriously cut down the size of the search space But, slow interpretation Neighbourhoods are highly rugged.

33. Example real-number Encoding (and: How EAs can innovate, rather than just optimize) D1, D2, D3, D4 D5 D6 D1 >= D2 >= D3, D4 <= D5 <= D6 Fixed at six diameters, five sections Design shape for a two-phase jet nozzle

34. 2, 1.8, 1.1, 1.3 1.3 1.5 The encoding enforces these constraints: D1 >= D2 >= D3, D4 <= D5 <= D6 Fixed at six diameters, five sections A simple encoding

35. Z1, Z2, D1, D2, D3 Dsmall…, Dn, Dn+1, … Middle section constrained to be smallest, That’s all Mutations can change diameters, add sections, and delete sections Num sections before smallest Num sections after smallest Section diameters A more complex encoding– bigger search space, slower, but potential for innovative solutions

36. About CW2

38. Pamela Hardaker, Benjamin N. Passow and David Elizondo. Walking State Detection from Electromyographic Signals towards the Control of Prosthetic Limbs UKCI 2013 got signals like this, but on her thigh just above the knee: standing walking running

39. Current knee-joint prospects need manual intervention to change between standing/walking/running modes (the wearer presses a button) Can we train a Neural Network to automatically detect when to change, on the basis of nerve signals from the last 30ms ?

40. About CW2 Snapshot of Pamela’s data Time signal state 0:04.2066.064453124standing 0:04.2075.693359374standing 0:04.2084.946289061standing 0:04.2094.609374999standing 0:04.2104.589843749standing 0:04.2114.482421874standing 0:04.2124.809570311standing 0:04.2135.102539061standing 0:04.2145.507812499standing 0:04.2155.922851562standing 0:04.2165.561523436standing 0:04.2175.610351561standing 0:04.2185.605468749standing 0:04.2195.551757811walking 0:04.2205.952148437walking 0:04.2216.542968749walking 0:04.2226.923828124walking 0:04.2236.845703124walking 0:04.2246.958007812walking 0:04.2256.484374999walking 0:04.2266.118164062walking 0:04.2275.424804686walking 0:04.2284.023437499walking 0:04.2293.696289061walking 0:04.2303.798828124walking 0:04.2313.579101561walking

41. About CW2 Snapshot of Pamela’s data Time signal state 0:04.2066.064453124standing 0:04.2075.693359374standing 0:04.2084.946289061standing 0:04.2094.609374999standing 0:04.2104.589843749standing 0:04.2114.482421874standing 0:04.2124.809570311standing 0:04.2135.102539061standing 0:04.2145.507812499standing 0:04.2155.922851562standing 0:04.2165.561523436standing 0:04.2175.610351561standing 0:04.2185.605468749standing 0:04.2195.551757811walking 0:04.2205.952148437walking 0:04.2216.542968749walking 0:04.2226.923828124walking 0:04.2236.845703124walking 0:04.2246.958007812walking 0:04.2256.484374999walking 0:04.2266.118164062walking 0:04.2275.424804686walking 0:04.2284.023437499walking 0:04.2293.696289061walking 0:04.2303.798828124walking 0:04.2313.579101561walking 1.37207031 1.254882812.39746 1.70410.448730.3381350.6748050 1.372070311.254882810.9716796852.397461.70411.372070.3554690.5324710.3901370 2.294921873 2.75391 0.517090.4382320.4863280 2.324218748 3.17383 0.4824220.5285640.5708010 2.324218748 2.0849609363.17383 2.690430.5079750.5187990.4667970 2.3242187482.0849609361.8603515613.173832.690432.377930.4847010.441650.4165040 2.0849609361.8603515611.6748046862.729492.504882.319340.3289390.260010.1035160 Max signal strength in last 30ms Max signal strength in last 20ms Max signal strength in last 10ms range of sig in last 30ms range … last 20ms range … last 10ms mean … last 30ms mean … last 20ms mean … last 10ms State: 0, 1 or 2

42. About CW2 Snapshot of Pamela’s data Time signal state 0:04.2066.064453124standing 0:04.2075.693359374standing 0:04.2084.946289061standing 0:04.2094.609374999standing 0:04.2104.589843749standing 0:04.2114.482421874standing 0:04.2124.809570311standing 0:04.2135.102539061standing 0:04.2145.507812499standing 0:04.2155.922851562standing 0:04.2165.561523436standing 0:04.2175.610351561standing 0:04.2185.605468749standing 0:04.2195.551757811walking 0:04.2205.952148437walking 0:04.2216.542968749walking 0:04.2226.923828124walking 0:04.2236.845703124walking 0:04.2246.958007812walking 0:04.2256.484374999walking 0:04.2266.118164062walking 0:04.2275.424804686walking 0:04.2284.023437499walking 0:04.2293.696289061walking 0:04.2303.798828124walking 0:04.2313.579101561walking 1.37207031 1.254882812.39746 1.70410.448730.3381350.6748050 1.372070311.254882810.9716796852.397461.70411.372070.3554690.5324710.3901370 2.294921873 2.75391 0.517090.4382320.4863280 2.324218748 3.17383 0.4824220.5285640.5708010 2.324218748 2.0849609363.17383 2.690430.5079750.5187990.4667970 2.3242187482.0849609361.8603515613.173832.690432.377930.4847010.441650.4165040 2.0849609361.8603515611.6748046862.729492.504882.319340.3289390.260010.1035160 Max signal strength in last 30ms Max signal strength in last 20ms Max signal strength in last 10ms range of sig in last 30ms range … last 20ms range … last 10ms mean … last 30ms mean … last 20ms mean … last 10ms State: 0, 1 or 2 CW2: evolve a neural network that predicts the state.

43. What you will do From me, you get: Working NN code that does the job already What you will do: Implement new mutation and crossover operators within my code, and test them on Pamela’s data Write a report comparing the performance of the different operators

44. Swarms, and how they might inspire us There are some interesting things that come to mind when we think of swarms (flocks, schools, etc …): A swarm sometimes seems to behave as if it is an individual organism. Ants or wasps on a hunt for food, or on the attack, behave as if with a single mind, co-ordinating different actions with different parts of the swarm. A swarm, of ants/bees/locusts/etc often exhibits behaviours that seem clearly more intelligent than any of the individual members of it. The way in which swarms in some species change direction is astoundingly well co-ordinated. The way in which swarms in some species avoid obstacles seems to be extremely well choreographed

45. Other puzzling things that swarms do Termites build huge nests – how?? Is an individual termite clever enough to do this? Bees build hives, with complex internal structure -- same question. How on earth can these things happen? HERE IS A LIVE EXAMPLE

47. Two simple rules: While continually wandering randomly: If you are empty-handed and encounter a disc, pick it up If you are carrying a disc and encounter another of the same colour, put yours down. Emergent order arises from simple local rules.

48. We sometimes exhibit swarm behaviour too

49. locustsants But we’re mainly interested in animals and insects Because we might learn something

50. Why does flocking/swarming occur so much in nature? Energy savings: Geese in V formation have around a 70% greater range than in flying individually. Individuals can fly around 25% faster (why?). Frightening and confusing predators; avoiding being “picked off” Helping to catch prey: e.g. tuna school in a crescent shaped flock with the concave part forward: This is thought to help channel their prey to the “focus”, and stop them from escaping

51. It also maybe helps with migration If we can assume that: –An individual has an idea, but not a perfect one, of where to go … e.g. by itself it may go a few degrees off course. –The “errors” of individuals are not correlated (i.e. they’re all wrong in a randomly different way) –An emergent result of the flocking is that the flock’s direction is the average of its members’ directions. Then: basic statistics can show that the error in the flocks direction is probably very small. About 1/sqrt(n) of the typical error of one of the n individuals.

52. So … Flocking occurs so much because it is clearly useful. But how do they do it so well? Individual ants are not clever enough to understand the benefits. It comes down to: simple behaviours of individuals in a group can have useful emergent properties. A theme we will continue to see a lot …

53. Another kinds of swarm behaviour is the dynamics and evolution of ideas as they get passed on and changed in social networks. A recommends to B, B recommends to C, …

54. The Adaptive Culture Model Robert Axelrod has a well-known theory, “Axelrod’s Culture Model”, which explains how ideas spread in societies. Kennedy and Eberhart (a computer scientist and a social scientist respectively) altered this into the “Adaptive Culture Model”, which works like this: If you think your neighbour is good, then be more like them. And that’s basically it. But notice the important words, neighbour: you change yourself under the influence of people nearby good: in some way your neighbour is more optimal than you, otherwise why be like them? more like: this is vague, so you have freedom in how you change This is actually a very good model for how culture and ideas spread quickly in societies. Everything from rumours to eating habits. I only hope this works with `green’ behaviour …

55. Back to computer science … From the CS viewpoint, the question is: How does this kind of, apparently organised, group behaviour emerge, without a central controller? Without (like we have) something like a brain in control of everything? The emergent behaviour that we see arises purely as a result of individuals in the swarm processing information in their (fairly) immediate neighbourhood. So, studying this in nature suggests how we can get co-ordinated behaviour from a group of individuals, without having to specify any overall controller. This is very useful, for example, for designing computer networks. If one main machine was in control of the network, and that machine crashed, … But so far that has not been a main success area for swarm inspiration …

56. Two main things that come from swarm inspiration: Simulations of natural flocks. For the entertainment and gaming industries, for example. Optimisation algorithms. Ants seem to find the shortest path to find food that may be quite distant from their nest. They do this via “stigmergy” – laying pheronomones on their path as they move. This has directly inspired the design of a very successful optimisation method, called Ant Colony Optimisation. Meanwhile, the adaptive culture model has led to a different, and also very successful, new optimisation algorithm, called Particle Swarm Optimisation

57. Craig Reynolds and “Boids” Craig Reynolds is a computer graphics researcher, who revolutionised animation in games and movies with his classic paper : Reynolds, C. W. (1987) Flocks, Herds, and Schools: A Distributed Behavioral Model, in Computer Graphics, 21(4) (SIGGRAPH '87 Conference Proceedings) pages 25-34. This paper is examinable reading, available on my teaching page. The story is: before this paper, animations of flocks, swarms, groups, and so on, behaved nothing at all like the real thing. Nobody knew how to make it realistic. (we still have that problem with fire, explosions, and realistic human movement, etc …) Reynold’s solved the problem by trying a very simple approach, which was inspired by a sensible view of how animals actually do it.

58. The problem We would like these to move like a realistic flock of starlings. (The heading of each one is suggested by where it’s pointing) But what’s wrong to start with?

59. The problem That’s better. Now what? Perhaps in the next timestep, they should all move the same small distance? They should all change their velocity in some way? What?

60. Reynold’s Rules Reynolds came up with three simple rules that solve this Problem, resulting in entirely realistic flocking behaviour. To explain them, we first need to consider the perceptual system of an individual (which Reynolds called a boid). For realistic movement, you need a realistic view of perception. E.g. a starling’s movement is not influenced at all by the flockmates that it cannot see – such as those out of its line of sight, or too far away.

61. A simple sensory system This picture is from Reynold’s boids page. The green boid can see a certain amount ahead, and is also aware of any flockmates within limits on either side (recall, birds tend to have eithers on the sides of their heads.) Two parameters, angle and distance, define the system. SO, this boid will only be influenced by those others it can sense according to these parameters.

62. Rule 1: Separation At each iteration, a boid makes an adjustment to its velocity according to the following rule: Avoid getting too close to local (the ones it is aware of) flockmates.

63. Rule 2: Alignment At each iteration, a boid makes an adjustment to match its velocity to the average of that of its local flockmates.

64. Rule 3: Cohesion At each iteration, a boid makes an adjustment to its velocity towards the centroid of its flockmates.

65. Notes: It’s not quite as simple as that to get realistic behaviour Need to define an appropriate distance for the perceptive range. What if this is too high, what if this is too small? Reynold’s found that he had to be careful about how the vectors from the three rules get combined. It is not ideal to simply add them. Opposing “shouts” from two rules may cancel out, leading to the third winning – in what scenarios might this be a problem? Note that the cohesion rule is interesting – it leads to “bifurcating” around obstacles – a follow-the-leader approach to flocking would not achieve that. The simple rules also realistically lead to “flash expansion” if started too close together.

66. Next time: Ant Colony Optimization