1. Learning School of Computing, University of Leeds, UK AI23 – 2004/05 – demo 2

2. part 1: what is learning? what would you say learning is?

3. Part 1 : what is learning? meaning of learning is subject to discussion recap some ideas high-level: experience alters behaviour low-level: weights (on neuron connections) change

4. example 1: Yamauchi/Beers alternate worlds one agent, one goal, one landmark two kinds of world: landmark-far/near a/b: landmark opposite to goal c/d: landmark between agent and goal agents task: reach goal (how? what if it knows the type of world it is in?)

5. example 1 [cont.] so, if world is known, a fixed strategy can be applied now, suppose a coin is tossed every 10 trials, and the kind of world is changed accordingly how can the problem be solved? The agent must learn to detect the kind of world it is in Yamauchi/Beers solution separately obtained (through artificial evolution) 3 distinct networks that solve subtasks: world detection, LF and LN goal-finding integrated networks: agent uses first trial in the 10-trial sequence to learn what world he is in; with that knowledge, he then switches to the right strategy for that world, for the next 9 trials. On average, 95% success

6. example 1 [cont.] is that learning??? can be seen as experience altering behaviour? no weights changing; rather, internal state of the agent is changed (by setting a world-type flag) – does it matter? the network is only learning one thing (the world the agent is in); can that still be called learning?

7. example 2: c.elegans 1-mm worm

8. example 2: c.elegans no evidence of synaptic plasticity in c.elegans, i.e. no mechanisms for changing the weights between neurons however, c.elegans exhibits various kinds of learning capabilities (behavioural plasticity) habituation / sensitisation, associative learning this would mean that changing weights on neuron connections is not the only way in which learning occurs in nature lots to discover and understand yet!

9. Part 2: different forms of learning activity: recall different forms of learning

10. forms of learning neural networks Gradient-descent algorithms for the McCulloch and Pitts neuron and for Feed-Forward Neural networks delta rule backpropagation feed-forward nets used in some demos in BEAST

11. forms of learning reinforcement learning agent interacts with environment and receives rewards (positive reinforcement) punishments (negative reinforcement) different to delta rule / backprop the agent is not given the correct answer, but only a good/bad signal; quantitative v. qualitative only desired results are needed to specify the problem, rather than intermediate actions; think of riding a bike, mazes, tic-tac-toe, backgammon [see demo, pendulum]

12. forms of learning conditioning Pavlovs experiments repeated pairing of two stimuli so that a previously neutral (conditioned) stimulus eventually elicits a response (conditioned response) similar to that originally elicited by nonneutral (unconditioned) stimulus notion of reward for artificial purposes

13. forms of learning Hebbian Learning form of learning in natural and artificial neural networks potentiation of effective synaptic connections and decay / depression of ineffective ones concept of simultaneous / concurrent / correlated activation

14. forms of learning winner takes all a form of competitive learning in natural and artificial neural networks neurons compete on activation over an input winner neuron gets reinforced Hebbian-like rule will be seen in this module

15. forms of learning evolutionary algorithms search algorithms inspired by natural evolution: population evolves, improving its fitness concepts of assessment (of an individual), selection, variation (of populations individuals over time) can be used as optimisation tools, even to "train" neural networks Yamauchi/Beer also the way we use them in BEAST will be seen in this module

16. forms of learning imitation a form of learning in nature and (recently) in robotics individuals learn by replication and repetition of behaviour observed in others [see demo, CogVis] work by CogVis lab @ SOC behaviour is adapted to their particulars [see demo, tennis]

17. forms of learning mimicry a form of evolutionary learning: species / groups learn by mimicking desirable genetic traits from other species / groups wasp-like insects work by J.Noble / D. Franks, SOC social learning learning is achieved via the communication of information within a social structure schools, books; birds, mammals

18. learning activity: where are the above used in nature and in bio-inspired algorithms?

19. thank you!