2. New Bulgarian University MindRACES, First Review Meeting, Lund, 11/01/2006 Anticipation by Analogy An Attempt to Integrate Analogical Reasoning with Perception, Selective Attention, Context, and Motor Control

3. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 2 Anticipation Mechanisms Explicit Anticipation: analogy-making Predictions based on one single example Implicit Anticipation: context & relevance Predicting relevance based on context – guiding attention in reasoning and perception Combining Explicit and Implicit Anticipation

4. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 3 Examples of Anticipation based on analogy-making and context Searching for your keys They are not at their usual place, so try to reconstruct what you have done with them (memory reconstruction), reminding of old episodes of key search and where you found them (analogy) Perceived elements (context) guide the reconstruction, reminding, and analogy

5. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 4 Examples of Anticipation based on analogy-making and context Searching for your car in the parking slot try to reconstruct where you have parked it (memory reconstruction), reminding of old episodes of car search and where you found it (analogy) reminding of old episodes of key search and where you found it (remote analogy) Perceived elements (context) guide the reconstruction, reminding, and analogy

6. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 5 Examples of Anticipation based on analogy-making and context Predicting the outcome of a game The same as the last outcome The same as the last failure The same as the last success The same as an special old case with this game The same as an old case with another game Perceived elements (context) guide the reminding and analogy

7. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 6 Examples of Anticipation based on analogy-making and context Predicting your partner’s or your rival’s next move What would I do in this situation (analogy with myself) What has this partner/rival done is analogous situation in the past (reminding of specific old case) What has another partner/rival done is analogous situation in the past (reminding of specific old case) Perceived elements (context) guide the reminding and analogy

8. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 7 Analogy-Making Analogy-making is the transfer of a system of relations from one domain (base) to another (target). Similarity based on structure, not overall similarity. Analogy is a very basic human ability.

9. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 8 Analogy-Making in water tpot in oven in milkw tpot on hplate corr-to

10. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 9 Rutherford’s Analogy Sun Nucleus ++ - -

11. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 10 Rutherford’s analogy The hydrogen atom is like our solar system. The Sun has a greater mass than the Earth and attracts it, causing the Earth to revolve around the Sun. The nucleus also has a greater mass then the electron and attracts it. Therefore it is plausible that the electron also revolves around the nucleus.

12. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 11 Main Implementation Tool - AMBR AMBR – a cognitive model of human analogy- making. The model is hybrid and integrates symbolic processing and connectionist spreading activation and constraint satisfaction at a micro level. The model is highly parallel and the behavior of the macro system emerges from the local interactions of micro agents.

13. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 12 Challenges to the pre-existing version of AMBR AMBR was a theoretical tool – it was never applied in realistic domain before. AMBR was developed for complex problem- solving, not for anticipation. AMBR was a model of the mind outside of a body – no interactions with the environment – no perception, no manipulation. AMBR was coded in LISP with no possibilities for communications with other software.

14. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 13 Scenario Implementation Selection of the scenarios to be used by NBU Developing simulation tools First simulation experiments

15. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 14 Scenarios studied by NBU Finding and Looking for an object (finding an object in a single room or in a maze of multiple rooms) Guards and thieves (collecting objects which are guarded by other agents)

16. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 15 Rooms layout

17. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 16 Looking for an Object (Scenario 1)

18. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 17 Guards and thieves (Scenario 3)

19. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 18 Developing Simulation Tools The AMBR model is being further developed and re-implemented in C#. The software for AIBO and Pioneer 3 is being mastered and tested. The simulation environment WEBOTS 5 is studied and simple simulation of the scenarios are being built. A middle tier is being implemented for communication between AMBR on one side and the robots and simulated environment on the other.

20. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 19 Overall System Architecture WORLD COMMUNI- CATION REASONING

21. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 20 World AIBO ERS7 Webots simulation

22. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 21 Communication World tier -> Reasoning tier  Collect information about the world using symbolic data from Webots  Report it to the Reasoning layer in suitable for AMBR form Reasoning tier -> World tier  Get the motion plan from AMBR: e.g “Go to the left cube”  Send commands for movement to Webots turning in place, walking forward

23. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 22 Reasoning Reasoning by analogy with previous episode (using the AMBR cognitive model) Describing AMBR in UML Implementation of the AMBR model in C# Project infrastructure (version control, unit testing, etc.)

24. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 23 Anticipation by Analogy ?

25. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 24 Past Episodes in Robot’s Memory Target situation

26. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 25 Results from the Simulation of Anticipation by Analogy

27. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 26 Results from the Simulation of Anticipation by Analogy

28. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 27 Results from the Simulation of Anticipation by Analogy

29. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 28 Results from the Simulation of Anticipation by Analogy

30. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 29 Results from the Simulation of Anticipation by Analogy

31. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 30 Results from the Simulation of Anticipation by Analogy

32. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 31 Simulation Result - Video

33. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 32 Challenges and Problems AMBR was developed as a model of complex analogies and therefore fitting and changes were required to produce anticipation:  Superficial features such as colors are typically ignored – colors are important in this domain;  Episodes are complex and differ significantly from each other – episodes are very similar in this domain.

34. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 33 Challenges and Problems AMBR was developed as an isolated reasoning model – needs to be integrated into a complete cognitive system:  Perceptual abilities need to be integrated that will encode the target situation – perception of objects, properties and relations – this is solved in the simulation environment, needs to be solved with real robots; integration of symbolic and sub-symbolic approach;  Selective attention needs to be modeled to limit the representation of the target and to focus on certain aspects of the situation;  Motor control – planning and motor control mechanisms

35. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 34 Challenges and Problems The simulation results need to be compared and possibly fitted to human data:  Some of the simulation data perfectly match human data;  Some differ significantly.

36. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 35 Comparing Simulation and Human Data: 100 Runs on each Target АBCD 12345678

37. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 36 Comparing Simulation and Human Data: 100 Runs on each Target Simulation dataHuman data

38. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 37 Integration with work of other partners Perception of objects, properties, relations – cooperation with IDSIA, LUCS, ISTC, OFAI Selective attention – integration of top- down and bottom-up mechanisms – cooperation with LUCS, IDSIA Emotions as regulators of the mechanisms of analogy-making, analogies as source of emotions – cooperation with ISTC, IST

39. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 38 Anticipation by Analogy: Putting things together Perception: target representation: IDSIA, LUCS Motor control: OFAI, IDSIA, LUCS Selective attention: LUCS, IDSIA, NBU Emotions (IST, ISTC) Analogical reasoning (NBU)

40. NBU MindRACES, First Review Meeting, Lund, 11/01/2006 39 Thank you for your attention! ?