1. An Architecture for Empathic Agents

2. Abstract Architecture Planning + Coping Deliberated Actions Agent in the World Body Speech Facial expressions Effectors Sensors Filters World Agent model + Model of self (Emotions) + Model of others (“memories”) Appraisal Concerns Reactions objects agents actions properties Action tendencies Emotional Signals Agent Mind

3. Planning and Coping Module  The Planning and Coping module incorporates the action-selection mechanism of the agent  Conventional approaches require the programmer to anticipate every possible context and state and tune the mechanism to produce the right action  To overcome the problem complexity, we can adopt a learning approach

4. Planning and Coping Module  Hybrid approaches make use of both neural network and symbolic structures to learn sensory-motor correlations and abstract concepts through experience  We propose one way to deal with action sequencing, viewed as a type of motor reasoning, in a fully neural architecture

5. Basic Mechanisms - Bistables

10. “Stack”

11. Basic Mechanisms - Bistables “Stack”

12. Basic Mechanisms - Bistables

13. “Pop”

14. Basic Mechanisms - Bistables

15. “Pop”

16. Basic Mechanisms - Bistables

17. Schemas Schemas are functional units (intermediate between overall behavior and neural function) for analysis of cooperative competition in the brain  A perceptual schema embodies the process whereby the system determines whether a given domain of interaction is present in the environment.  Current plans are made up of motor schemas.

18. Simplified Architecture WORLD External Perceptions Perceptions Actions Planning and Coping Network Internal PerceptionsMotivational System

19. Neural Architecture The Planing network is composed of:  Nodes  Environmental states are expressed through the activation of state nodes  The agent’s needs are reflected in the activation of drive nodes  The agent’s actions are determined by the activation of action nodes  Links are one-way communication channels that enable the communication between nodes.

20. Splitting mechanism

21. A→B →F C→B →H

22. Splitting Mechanism

23. Internal State Internal perceptions are defined by a set of internal variables that evolve in time.  A general internal variable is in the range [v min, v max ] and evolves in time according to: where  v i is a variation caused by some external causes.

24. Drives Internal variables are homeostatic variables. Each internal variable has a comfort zone and two drives associated with it, whose activity measures the need of increase or decrease. Drive for increase : Drive for decrease :

25. Global Drive and Reward All the base nodes receive a global drive signal whose intensity is a function of the most pressing need All the base nodes also receive a global reward signal corresponding to the satisfaction of one of the needs

26. Learning - Detection of Goals A base action node learns the correlation between the success of the corresponding command and the global reward When such correlation is strong enough, the node splits, producing a specialized node that plays the role of a goal Action success + reward = GOAL

27. Context Learning Specialized nodes compute the context of certain events The context is defined by the ensemble configuration of the activities of the group of nodes linked to one node The context is learned on the occurrence of certain events  Event good is an event we wish to be able to predict  Event bad corresponds to the situation where the event good does not occur, contrary to what was expected The events good and bad are associated to two distinct weights that are used to compute the context value

28. Base Nodes Functioning Excitation of a node codes the detection of a perceptive event Action nodes can also have a call activity that will trigger the associated action The call activity is initiated at random, except when it is regulated by the specialized nodes, but the number of action nodes that can be simultaneously calling by means of spontaneous activity is limited The call activity is maintained during a random period of time

29. Specialized Nodes Functioning Specialized nodes are used primarily to organize the calls sent to the motor module These nodes can be seen as schemas that can be chained in a plan The bistable activity of specialized nodes implements a stack mechanism The specialized nodes make use of local notions of drive and reward  The local drive corresponds to the necessity of using the associated command  The local reward corresponds to the successful execution of that command

30. Specialized Nodes Functioning A Competition mechanism determines the nodes that can perform a call The behaviour of a specialized node depends on its bistable activity  If the node is off, the call is transmitted to the parent base node, where it will trigger action execution  If the node is on, the local drive is stocked and transmitted to the node’s subgoals The on is triggered by the event bad The off transition happens in two situations  After a given period of time  When the right context can now be obtained

31. Specialized Nodes Functioning Context learning is triggered when the events good or bad occur  The event good corresponds to occurrence of reward (action successful execution) in response to a call not stocked  The event bad corresponds to the case where the call of the node is not able of leading to reward Specialized nodes also learn the time needed for the execution of the associated command  The occurrence of the event good means that that time can be reduced  The occurrence of the event bad when the context is favourable means that that time should increase

32. Contexts There are two types of context  Excitation context evaluate the excitation of context nodes  Call context is only computed by action nodes and evaluate the call activity of the other action nodes Excitation context leads to the reinforcement of the weights corresponding to base nodes whose detection activity is predictive of the success of the command generated by the specialized node When a base node is strongly implicated in the context, it splits to create a subgoal of the specialized node

33. Contention Scheduling The local drive determines the dominant schemas and its computation favours the nodes whose excitation context is more coherent with the current activity The reward obtained by the subgoals during plan execution is propagated upwards in the hierarchy, in the direction of the goal A dominant specialized node inhibits the base nodes whose commands are incompatible with its own command according to the learned call context

34. Conclusions Several aspects of the simplified architecture have been tested successfully in a text-like world  Goal creation  Creation and chaining of subgoals  Inhibition mechanism

35. Conclusions The integration of the agents in a 3D world has raised some technical problems Goal creation was tested, other aspects need more work We have to design carefully the internal state of the agents and do some bootstrapping if we want the characters to exhibit the right behaviour

36. Future Work Representation of states, both external and internal (abstract space) Consider complex and high-level actions, like producing speech External representation of the agent’s state (body and facial expressions, speech)