New Bulgarian University MindRACES, First Review Meeting, Lund, 11/01/2006 Anticipation by Analogy An Attempt to Integrate Analogical Reasoning with Perception, Selective Attention, Context, and Motor Control

NBU MindRACES, First Review Meeting, Lund, 11/01/ Anticipation Mechanisms Explicit Anticipation: analogy-making Predictions based on one single example Implicit Anticipation: context & relevance Predicting relevance based on context – guiding attention in reasoning and perception Combining Explicit and Implicit Anticipation

NBU MindRACES, First Review Meeting, Lund, 11/01/ Examples of Anticipation based on analogy-making and context Searching for your keys They are not at their usual place, so try to reconstruct what you have done with them (memory reconstruction), reminding of old episodes of key search and where you found them (analogy) Perceived elements (context) guide the reconstruction, reminding, and analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Examples of Anticipation based on analogy-making and context Searching for your car in the parking slot try to reconstruct where you have parked it (memory reconstruction), reminding of old episodes of car search and where you found it (analogy) reminding of old episodes of key search and where you found it (remote analogy) Perceived elements (context) guide the reconstruction, reminding, and analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Examples of Anticipation based on analogy-making and context Predicting the outcome of a game The same as the last outcome The same as the last failure The same as the last success The same as an special old case with this game The same as an old case with another game Perceived elements (context) guide the reminding and analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Examples of Anticipation based on analogy-making and context Predicting your partner’s or your rival’s next move What would I do in this situation (analogy with myself) What has this partner/rival done is analogous situation in the past (reminding of specific old case) What has another partner/rival done is analogous situation in the past (reminding of specific old case) Perceived elements (context) guide the reminding and analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Analogy-Making Analogy-making is the transfer of a system of relations from one domain (base) to another (target). Similarity based on structure, not overall similarity. Analogy is a very basic human ability.

NBU MindRACES, First Review Meeting, Lund, 11/01/ Analogy-Making in water tpot in oven in milkw tpot on hplate corr-to

NBU MindRACES, First Review Meeting, Lund, 11/01/ Rutherford’s Analogy Sun Nucleus

NBU MindRACES, First Review Meeting, Lund, 11/01/ Rutherford’s analogy The hydrogen atom is like our solar system. The Sun has a greater mass than the Earth and attracts it, causing the Earth to revolve around the Sun. The nucleus also has a greater mass then the electron and attracts it. Therefore it is plausible that the electron also revolves around the nucleus.

NBU MindRACES, First Review Meeting, Lund, 11/01/ Main Implementation Tool - AMBR AMBR – a cognitive model of human analogy- making. The model is hybrid and integrates symbolic processing and connectionist spreading activation and constraint satisfaction at a micro level. The model is highly parallel and the behavior of the macro system emerges from the local interactions of micro agents.

NBU MindRACES, First Review Meeting, Lund, 11/01/ Challenges to the pre-existing version of AMBR AMBR was a theoretical tool – it was never applied in realistic domain before. AMBR was developed for complex problem- solving, not for anticipation. AMBR was a model of the mind outside of a body – no interactions with the environment – no perception, no manipulation. AMBR was coded in LISP with no possibilities for communications with other software.

NBU MindRACES, First Review Meeting, Lund, 11/01/ Scenario Implementation Selection of the scenarios to be used by NBU Developing simulation tools First simulation experiments

NBU MindRACES, First Review Meeting, Lund, 11/01/ Scenarios studied by NBU Finding and Looking for an object (finding an object in a single room or in a maze of multiple rooms) Guards and thieves (collecting objects which are guarded by other agents)

NBU MindRACES, First Review Meeting, Lund, 11/01/ Rooms layout

NBU MindRACES, First Review Meeting, Lund, 11/01/ Looking for an Object (Scenario 1)

NBU MindRACES, First Review Meeting, Lund, 11/01/ Guards and thieves (Scenario 3)

NBU MindRACES, First Review Meeting, Lund, 11/01/ Developing Simulation Tools The AMBR model is being further developed and re-implemented in C#. The software for AIBO and Pioneer 3 is being mastered and tested. The simulation environment WEBOTS 5 is studied and simple simulation of the scenarios are being built. A middle tier is being implemented for communication between AMBR on one side and the robots and simulated environment on the other.

NBU MindRACES, First Review Meeting, Lund, 11/01/ Overall System Architecture WORLD COMMUNI- CATION REASONING

NBU MindRACES, First Review Meeting, Lund, 11/01/ World AIBO ERS7 Webots simulation

NBU MindRACES, First Review Meeting, Lund, 11/01/ Communication World tier -> Reasoning tier  Collect information about the world using symbolic data from Webots  Report it to the Reasoning layer in suitable for AMBR form Reasoning tier -> World tier  Get the motion plan from AMBR: e.g “Go to the left cube”  Send commands for movement to Webots turning in place, walking forward

NBU MindRACES, First Review Meeting, Lund, 11/01/ Reasoning Reasoning by analogy with previous episode (using the AMBR cognitive model) Describing AMBR in UML Implementation of the AMBR model in C# Project infrastructure (version control, unit testing, etc.)

NBU MindRACES, First Review Meeting, Lund, 11/01/ Anticipation by Analogy ?

NBU MindRACES, First Review Meeting, Lund, 11/01/ Past Episodes in Robot’s Memory Target situation

NBU MindRACES, First Review Meeting, Lund, 11/01/ Results from the Simulation of Anticipation by Analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Results from the Simulation of Anticipation by Analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Results from the Simulation of Anticipation by Analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Results from the Simulation of Anticipation by Analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Results from the Simulation of Anticipation by Analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Results from the Simulation of Anticipation by Analogy

NBU MindRACES, First Review Meeting, Lund, 11/01/ Simulation Result - Video

NBU MindRACES, First Review Meeting, Lund, 11/01/ Challenges and Problems AMBR was developed as a model of complex analogies and therefore fitting and changes were required to produce anticipation:  Superficial features such as colors are typically ignored – colors are important in this domain;  Episodes are complex and differ significantly from each other – episodes are very similar in this domain.

NBU MindRACES, First Review Meeting, Lund, 11/01/ Challenges and Problems AMBR was developed as an isolated reasoning model – needs to be integrated into a complete cognitive system:  Perceptual abilities need to be integrated that will encode the target situation – perception of objects, properties and relations – this is solved in the simulation environment, needs to be solved with real robots; integration of symbolic and sub-symbolic approach;  Selective attention needs to be modeled to limit the representation of the target and to focus on certain aspects of the situation;  Motor control – planning and motor control mechanisms

NBU MindRACES, First Review Meeting, Lund, 11/01/ Challenges and Problems The simulation results need to be compared and possibly fitted to human data:  Some of the simulation data perfectly match human data;  Some differ significantly.

NBU MindRACES, First Review Meeting, Lund, 11/01/ Comparing Simulation and Human Data: 100 Runs on each Target АBCD

NBU MindRACES, First Review Meeting, Lund, 11/01/ Comparing Simulation and Human Data: 100 Runs on each Target Simulation dataHuman data

NBU MindRACES, First Review Meeting, Lund, 11/01/ Integration with work of other partners Perception of objects, properties, relations – cooperation with IDSIA, LUCS, ISTC, OFAI Selective attention – integration of top- down and bottom-up mechanisms – cooperation with LUCS, IDSIA Emotions as regulators of the mechanisms of analogy-making, analogies as source of emotions – cooperation with ISTC, IST

NBU MindRACES, First Review Meeting, Lund, 11/01/ Anticipation by Analogy: Putting things together Perception: target representation: IDSIA, LUCS Motor control: OFAI, IDSIA, LUCS Selective attention: LUCS, IDSIA, NBU Emotions (IST, ISTC) Analogical reasoning (NBU)

NBU MindRACES, First Review Meeting, Lund, 11/01/ Thank you for your attention! ?