N ew and E mergent W orld models T hrough I ndividual, E volutionary and S ocial Learning Main goal: to realize an evolving artificial society capable of exploring the environment and developing its own image of this environment through cooperation and interaction. Long-term target: to learn how to design agents that are able to adapt autonomously to, and then operate effectively in, environments whose features are not known in advance. Technical objectives: 1.To develop an artificial society with an emergent culture. 2.To create a powerful “emergence engine” as a combination of individual learning, evolutionary learning, and social learning. 3.To develop social learning mechanisms that allow sharing knowledge with other agents (incl. language evolution). System: scaleable, p2p infrastructure for large experiments: Agent complexity Population size World size and complexity Virtual time Logo IST LogFET Vrije Universiteit van Amsterdam NAPIER UNIVERSITY EDINBURGH Vrije Universiteit (NL) (Coordinator, agents, evolution): Prof.Dr. A.E. Eiben R. Griffioen C. Tzolov University of Surrey (UK) (Environments and challenges): Prof. N. Gilbert, S. Schuster Napier University (UK) (P2P infrastructure) Prof. B. Paechter Dr. B.G.W. Craenen Eötvös Loránd University (HU) (Individual learning, mining emergence) Dr. A. Lorincz A. Bontovics Dr. Gy. Hévizi Tilburg University (NL) (language, communication and co-operation) Dr. P. Vogt, Dr. F. Divina Learning mechanisms  Evolutionary learning: crossover and mutation on DQ-trees and crossover and mutation on attitude genes  Individual learning: reinforcement learning  Social learning: sharing knowledge bits directly (telepathy) sharing knowledge bits via evolved language learning / evolving language together Each time step every agent receives from the environment the objects it perceives. Agent may use this knowledge from the environment for decision-making and learning. At the end of each time step an agent produces an output to the environment, which is the action that it wants to perform. Agent environment interaction feedback MEMORYMEMORY PERCEPTION DECISION MAKING INDIVIDUAL / SOCIAL LEARNING GENETIC INFORMATION initial rules and values EVOLUTIONARY LEARNING Brain Genes ENVIRONMENT AGENT action visual, auditory information uses / creates Inside the Agent  physical (e.g., sex) vs. mental properties (e.g., controller)  inheritable vs. learnable properties  inheritable – subject to evolution attitude genes basic controller structure (decision tree)  learnable – subject to individual and social learning refined controller structure controller parameters, e.g., learned bias for action A concepts used in controller, e.g., “enemy” Controller = decision Q-tree Agent decisions by DQ-tree, also effected by attitude genes Agent Plant Token Road Environment Discrete time and space (grid world) with plants (energy) and tokens, locations with varying properties (e.g. road / obstacle). Results so far:  Evolutionary learning: learning food seeking behaviour learning to distinguish poisonous/edible food  Social learning: developing shared lexicon for built-in concepts  System: research platform downloadable (single machine version) distributed, p2p version beta FP6 FET Open