How To Implement Social Policies. A Deliberative Agent Architecture Roberto Pedone Rosaria Conte IP-CNR Division "AI, Cognitive and Interaction Modelling" PSS (Project on Social Simulation) V.LE Marx 15, Roma. June 5-8, 2000

The Problem... Multiple agents in common environments face problems posed –by a finite world (e.g., resource scarcity), and therefore –by social interference. These are problems of social interdependence –Shared problems –Requiring a common solution = multi-agent plan (multiple actions for a unique goal) Necessity of common solutions adopted by the majority. How to obtain that interdependent but autonomous agents apply common solutions?

Autonomous agents? Self-sufficient: By definition, impossible! äSelf-interested: what does it mean? Selfish äHave own criteria to filter external inputs –Beliefs: they may not perceive/understand Problem (cognitive biases) As common (“) Solution (“) Complementarity (“) –Goals: they accept requests when these are useful for some of their goals BUT: difference between interests and goals...

How To Achieve Common Solutions Among Autonomous Agents ? Two approaches Bottom-up: Emergent, spontaneous processes among adaptive agents Top-down: Designed, incentives and sanctions modifying the preferences of rational agents –Acquisition of solutions –Enforcing mechanisms –Violation S p a >p b S

BUT... Evolutionary processes and Adaptive Agents: socially acceptable outcomes are doubtful! Rational agents require unrealistic conditions (specified, severe and certain sanctions).

Bottom UP. Adaptive Agents Analogy between biological evolution and social processes –Fitter individuals survive –Social processes spread and evolve through imitation of fitter agents. But –how can agents perceive the fitness of a strategy? “How do individuals get information about average fitness, or even observe the fitnesses of other individuals?” (Chattoe, 1998) -How to make sure that what propagates is what is socially desirable, without the intervention of some deus ex machina (the programmer)?

Top Down. Rational Agents Socially desirable effects are deliberately pursued through incentives and sanctions. Incentives and sanctions induce rational agents to act according to global interest. Rational agents take decisions by calculating the subjective expected value of actions according to their utility function. A rational decider will comply with the norm if utility of incentive (or of avoiding sanction) is higher than utility of transgression.

Effects Rational deciders will violate a norm n i as soon as one or more of the following conditions applies: -Sanctions are not imposed -Sanctions are not specified -The sanction for violating n i is lower than the value of transgression with equal probability of application of the sanction (1/2). -The sanction for violating an incompatible norm is higher. –The sanction for violating the norm is not or rarely applied. Fast decline in norm compliance is likely to follow from any of the above conditions. Then, what?

Top-Down and Bottom-UP Top down –agents acquire S –agents decide to accept S –have S even if they do not apply it Bottom up –infer S from others –communicate S to one another –control each other. Difference from previous approaches: –S is represented as a specific object in the mind –Can travel from one mind to the other. S Do you know that S? She believes S Should I? These guys believe S. Will they act accordingly?

Shut up, otherwise I knock you down! But How To Tell S? Sanction –may be sufficient (but unnecessary) for acceptance –but it is insufficient (and unnecessary) for recognition. Shut up, otherwise I knock you down! This guy is crazy, better to do as he likes!

Believed Obligations May be insufficient for acceptance But necessary & sufficient for recognition! Shut up! NO! You ought to! I know, but I don’t care!

This requires This requires a cognitive deliberative agent: –To communicate, infer, control: meta-level representation (beliefs about representations of S) –To decide whether to accept/reject a believed: meta- meta-level representation (decision about belief about representation of S). When the content of representation is a Norm, we have Deliberative Normative Agents. S BEL(S) GOAL(S) BEL(GOAL(S)) cognitive deliberative

Deliberative Normative Agents Are able to recognize the existence of norms Can decide to adopt a norm for different reasons (different meta-goals) Can deliberately follow the norm or violate it in specific cases Can react to violations of the norm by other agents Require more realistic conditions!

Deliberative Normative Agents  Can reason about norms  Can communicate norms Can negotiate about norms This implies Have norms as mental objects Hve different levels of representations and links among them

A Deliberative Agent Architecture With Norms (Castelfranchi et al., 1999) DESIRE I (Brazier et al., 1999)

DESIRE II Main Components Coomunication Agent Interaction Management, Perception World Interaction Management, Maintenance of Agent Information = agent models Maintenance of World Information = world models Maintenance of Society Information = social world models Own Process Control = mental states & processing: goal generation & dynamics, belief acceptance, decision-making.

DESIRE III

DESIRE IV Own Process Control

DESIRE V Own Process Control Components information Norm Management(NormsBelief) Strategy Management (Candidate Goals) Goal Management(Selected Goals) Plan Management(Plan Chosen) (Action)

DESIRE VI: Norms and goals non-adopted norms : useful for coordination (predict the behaviour of the other agents) adopted norms : impact on goal generation; among the possible ‘sources of goals’ -> normative goals impact on goal selection by providing criteria about how to select among existing goals; e.g., preference criteria.

DESIRE VII: Norms and plans Norms may generate plans Norms may select plans Norms may select actions E.g. the norm “be kind to colleagues” may lead to a preferred plan to reach a goal within an organisation.

To sum up Adaptive agents: fit = socially acceptable? Rational agents are good enough if sanctions are severe, effective, and certain. Otherwise, collapse in compliance...

With Deliberative Agents Acquisition of norms online Communication and negotiation (social monitoring and control) Flexibility: –agents follow the norm whenever possible –agents violate the norm (sometimes) –agents violate the norm always if possible But graceful degradation with uncertain sanctions!

Work in Progress DESIRE is used for simulation-based experiments on the role of deliberative agents in distributed social control –Market with heterogeneous, interdependent agents Make contracts in the morning (resource exchange under survival pressure) Deliver in the afternoon (norm of reciprocity) Violations can be found out and –The news spread through the group –The event denounced –Both. –Objectives: What are the effects of violation? (Under given enviornmental conditions, norms may be non adaptive…) When and why agents react to violation? What are the effects of reaction?

What To Do Next DESIRE is complex –Computationally: too few agents. Can simpler languages implement meta-level representations? –Mentally: too much deliberation Emotional/affective enforcement? NB -> E (moral sense, guilt) -> NG -> NA. Emotional shortcut others’ evaluations -> SE (shame) -> NA implicit normsimplicit n-goals. Affective computing! But integrated with meta-level representations.