1. Sisteme multi-agent Curs 2 Universitatea “Politehnica” din Bucuresti Adina Magda Florea

2. Modele arhitectura de agenti
Structura conceptuala a agentilor
Arhitecturi de agenti cognitivi
Arhitecturi de agenti reactivi
Arhitecturi stratificate

3. 1. Structura conceptuala a agentilor
1.1 Rationalitatea unui agent
Ce inseamna rationalitatea unui agent
Cum putem masura rationalitatea unui agent?
O masura a performantei 3

4. Un agent este situat in mediu
Perpece mediul prin sensori si actioneaza asupra lui prin efectori
Scop: proiectarea unui program – functie care realizeaza corespondenta sensori - efectori
Agent = architectura + program
Mediu
accesibil vs. inaccesibil
determinist vs. non-determinist
static vs. dinamic
discret vs. continu 4

5. 1.2 Modelare agent E = {e1, .., e, ..} P = {p1, .., p, ..}
A = {a1, .., a, ..}
Agent reactiv
see : E  P
action : P  A
env : E x A  E
(env : E x A  P(E))
Decision
component
Agent
action
Perception
Execution
component
component
see
action
Environment
env 5

6. Modelare agent Mai multi agenti reactivi see : E  P inter : P  I
env : E x A1 x … An  P(E)
inter : P  I
action : P x I  A
I = {i1,…,i,..}
Agent (A1)
Decision
component
Interaction
component
inter
action
Agent (A2)
Perception
Execution
component
component
Agent (A3)
see
action
Environment
env 6

7. Modelare agent Agenti cognitivi action : S x I Ai next : S x P  S
Agenti cu stare S = {s1,…,s,…}
action : S x I Ai
next : S x P  S
inter : S x P  I
see : E  P
env : E x A1 x … An  P(E) 7

8. Modelare agent Agenti cu stare si scopuri Agenti cu utilitate
goal : E  {0, 1}
Agenti cu utilitate
utility : E  R
Mediu nedeterminist
env : E x A  P(E)
Probabilitatea estimata de un agent ca rezultatul unei actiuni (a) executata in e sa fie noua stare e’ 8

9. Modelare agent Agenti cu utilitate
Utilitatea estimata (expected utility) a unei actiuni a intr-o stare e, dpv al agentului
Principiul utilitatii estimate maxime
Maximum Expected Utility (MEU) 9

10. Cum modelam? Iesirea din labirint Agent reactiv Agent cognitiv
Agent cognitiv cu utilitate
Probleme:
Ce actiuni selectez
Ce se face daca rezultatul actiunilor nu este cunoscut
Cum se iau in considerare schimbarile din mediu 10

11. 2. Arhitecturi de agenti cognitivi
2.1 Comportare rationala
IA si Teoria deciziei IA
Teoria deciziei
Problema 1 = deliberare/decizie vs. actiune/proactivitate
Problema 2 = limitarea resurselor 11

12. 12 General cognitive agent architecture Interactions Information about
itself
- what it knows
- what it believes
- what is able to do
- how it is able to do
- what it wants
environment and
other agents
- knowledge
- beliefs
Communication
Reasoner
Other
agents
Planner
Control
Scheduler&
Executor
Output
State
Input
General cognitive agent architecture
Environment 12

13. 2.2 Modele LPOI 13 (a)Reguli de deductie
Reprezentare simbolica + inferente – demonstrarea teoremelor pt a afla ce actiuni va face agentul
Abordare declarativa
(a)Reguli de deductie
Predicate At(x,y), Free(x,y), Wall(x,y), Exit(dir), Do(action)
Fapte si axiome despre mediu
At(0,0)
Wall(1,1)
x y Wall(x,y)  Free(x,y)
Reguli de deductie
At(x,y)  Free(x,y+1)  Exit(east)  Do(move_east)
Actualizare automata a starii curente si test pt starea scop
At(0,3) 13

14. Modele LPOI
(b) Utilizarea calcului situational = descrie schimbari utilizand formalismul logic
Situatie = starea rezultata prin executarea unei actiuni
Result(Action,State) = NewState
At(location, situation)
At((x,y), Si)  Free(x,y+1)  Exit(east) 
At((x,y+1), Result(move_east,Si))
Scop At((0,3), _) + actiuni care au condus la scop
means-end analysis 14

15. Avantaje LPOI
Dezavantaje
Avem nevoie de un alt model 15

16. Arhitecturi 2.3 BDI Specificatii de nivel inalt Means-end analysis
Beliefs (convingeri) = informatii pe care agentul le are despre lume
Desires (dorinte) = stari pe care agentul ar vrea sa le vada realizate
Intentions (intentii) = dorinte (sau actiuni) pe care agentul s-a angajat sa le indeplineasca
BDI – teoria rationamentului practic - Bratman, 1988
Rolul intentiilor 16

17. BDI
Componenta filozofica
Arhitectura software
IRMA - Intelligent Resource-bounded Machine Architecture
PRS - Procedural Reasoning System
Componenta logica
Rao & Georgeff, Wooldrige
(Int Ai  )   (Bel Ai ) 17

18. Arhitectura BDI 18 percepts Belief revision Beliefs B = brf(B, p)
Knowledge
B = brf(B, p)
Opportunity
analyzer
Deliberation process
Desires
D = options(B, D, I)
Intentions
Filter
Means-end
reasonner
I = filter(B, D, I)
Intentions structured
in partial plans
 = plan(B, I)
Library of plans
Plans
Executor 18
actions

19. Proprietati ale intentiilor conduc means-end analysis
limiteaza deliberare
persista
influenteaza convingerile
Bucla de control a agentului
B = B I = I D = D0
while true do
get next perceipt p
B = brf(B,p)
D = options(B, D, I)
I = filter(B, D, I)
 = plan(B, I)
execute()
end while 19

20. (Commitment strategies)
Strategii de angajare
(Commitment strategies)
Optiune aleasa de agent ca intentie – agentul s-a angajat pentru acea optiune
Persistenta intentiilor
Interbare: Cat timp se angajeaza un agent fata de o inetntie?
Angajare oarba (Blind commitment)
Angajare limitata (Single minded commitment)
Angajare deschisa (Open minded commitment) 20

21. 21 Bucla de control BDI angajare limitata B = B0 I = I0 D = D0
while true do
get next perceipt p
B = brf(B,p)
D = options(B, D, I)
I = filter(B, D, I)
 = plan(B, I)
while not (empty() or succeeded (I, B) or impossible(I, B)) do
 = head()
execute()
 = tail()
if not sound(, I, B) then
end while
Bucla de control BDI
angajare limitata
Dropping intentions that are impossible
or have succeeded
Reactivity, replan 21

22. 22 Bucla de control BDI angajare deschisa B = B0 I = I0 D = D0
while true do
get next perceipt p
B = brf(B,p)
D = options(B, D, I)
I = filter(B, D, I)
 = plan(B, I)
while not (empty() or succeeded (I, B) or impossible(I, B)) do
 = head()
execute()
 = tail()
end while
Bucla de control BDI
angajare deschisa
if reconsider(I, B) then
Replan 22

23. 3. Arhitecturi dea genti reactivi
Arhitectura de subsumare - Brooks, 1986
(1) Luarea deciziilor = {Task Accomplishing Behaviours}
Fiecare comportare (behaviour) = o functie ce realizeaza o actiune
TAB – automate finite
Implementare: situation  action
(2) Mai multe comportari pot fi activate in paralel 23

24. Arhitectura de subsumare
Un TAB este reprezentat de un modul de competenta (c.m.)
Fiecarte c.m. executa un task simplu – comportare concreta
c.m. opereaza in paralel
Nivele inferiroare fata de cele superioare
c.m. la nivele inferioare
c.m. la nivele superioare
 subsumtion architecture 24

25. 25 Sensors Effectors Competence Module (2) Explore environ Module (0)
Avoid obstacles
Effectors
Module (1)
Move around
Input
(percepts)
Output
(actions)
Competence
Module (1)
Move around
Module 1 can monitor and influence the inputs and outputs of Module 2
M1 = move around while avoiding obstacles  M0
M2 = explores the environment looking for distant objects of interests while moving around  M1
Incorporating the functionality of a subordinated c.m. by a higher module is performed using suppressors (modify input signals) and inhibitors (inhibit output)
Supressor node
Inhibitor node
Competence
Module (0)
Avoid obstacles 25

26. 26 Comportare (c, a) – conditie-actiune; descrie comportarea
R = { (c, a) | c  P, a  A} - multimea reguli de comportare
  R x R – relatie binara totala de inhibare
function action( p: P)
var fired: P(R), selected: A
begin
fired = {(c, a) | (c, a)  R and p  c}
for each (c, a)  fired do
if   (c', a')  fired such that (c', a')  (c, a) then return a
return null
end 26

27. 27 Comportare (1) Daca detectez obstacol atunci schimb directia
Ne aflam pe o planeta necunoscuta care contine aur. Mostre de teren trebuie aduse la nava. Nu se stie daca sunt aur sau nu. Exsita mai multi agenti autonomi care nu pot comunica intre ei. Nava transmite semnale radio: gradient al campului
Comportare
(1) Daca detectez obstacol atunci schimb directia
(2) Daca am mostre si sunt la baza atunci depune mostre
(3) Daca am mostre si nu sunt la baza atunci urmez campul de gradient
(4) Daca gasesc mostre atunci le iau
(5) Daca adevarat atunci ma misc in mediu
(1)  (2)  (3)  (4)  (5)
Care sunt premisele pt ca acest comportament sa functioneze? (distributie a aurului?)
Daca distributia este reala? 27

28. (1)  (2)  (3)  (4)  (5)  (6) 28 Agentii pot comunica indirect:
- Depun si culeg boabe radiocative
- Pot seziza aceste boabe radioactive
(1) Daca detectez obstacol atunci schimb directia
(2) Daca am mostre si sunt la baza atunci depune mostre
(3) Daca am mostre si nu sunt la baza atunci depun boaba radioactiva si urmez campul de gradient
(4) Daca gasesc mostre atunci le iau
(5) Daca gasesc boabe radioactive atunci iau una si urmez campul de gradient
(6) Daca adevarat atunci ma misc in mediu
(1)  (2)  (3)  (4)  (5)  (6) 28

29. 4. Arhitecturi stratificate
Comportare reactiva si pro-activa
Cel putin 2 straturi
Horizontal layering - i/o horizontal
Vertical layering - i/o vertical
Action
output
Action
output
Layer n …
Layer 2
Layer 1
Layer n …
Layer 2
Layer 1
Layer n …
Layer 2
Layer 1
Action
output
perceptual
input
Vertical
Horizontal
perceptual
input
perceptual
input 29

30. Horizontal layering Vertical layering n comportari, n niveluri
Comportarea globala poate fi inconsistenta
Interactiuni intre niveluri: mn (m = nr actiuni pe nivel)
Necesita un sistem de control
Vertical layering
Interactiuni intre niveluri m2(n-1)
Nu sunt tolerante la defecte (daca un nivel se defecteaza) 30

31. TouringMachine - biblioteca de planuri 31
Horizontal layering – 3 niveluri de realizare a actiunilor
Nivel reactiv - set de reguli situatie-actiune rules pt mediu
Nivel planificare
- comportare pro-activa
- biblioteca de planuri
Nivel modelare
- reprezinta mediul, agentul si ceilalti agenti
- stabileste scopuri
- scopurile sunt trimise nivelului/stratului inferior
Sistem de control 31

32. 32 Nivel modelare perceptii Subsistem Nivel planificare perceptie
actiune
Nivel reactiv
actiuni
Subsistem control 32

33. InteRRaP 33 Stratificata BDI Principii 2 niveluri
Atat controlul cat si BC sunt stratificate
Controlul este bottom-up
Fiecare nivel foloseste rezultatele nivelului inferior
Fiecare nivel de control este format din:
- modul recunoastere situatie / activare scop (SG)
- modul planificare (PS)
The prays and the predators (both are agents) move over a space represented in the form of a grid. The objective is for the predators to capture the pray animals by surrounding them as in the figure above. The following hypotheses are laid down:
1. The dimension of the environment is finite
2. The predator and pray animals move at fixed speeds and generally at the same speed.
3. The pray animals move in a random manner by making Brownian movements
4. The predators can use the corner and edges to block a pray animal's path
5. The predators have a limited perception of the world that surrounds them, which means that they can see the prey only if it is in one of the squares at a distance within their field of perception 33

34. 34 I n t e R a P Social KB Planning KB World KB actions percepts
Cooperative
planning layer SG
Social KB PS I n t e R a P
Local
planning layer SG PS
Planning KB
Behavior
based layer SG PS
World KB
World interface Sensors Effectors Communication
actions 34
percepts

35. BDI model in InteRRaP options filter SG plan PS 35 Beliefs
Social model
Mental model
World model
Situation
Cooperative situation
Local planning situation
Routine/emergency sit.
Goals
Cooperative goals
Local goals
Reactions
Sensors
filter SG
Options
Cooperative option
Local option
Reaction
Intentions
Cooperative intents
Local intentions
Response
Effectors
Operational primitive
Joint plans
Local plans
Behavior patterns
plan PS 35

36. 36 BDI Architectures First implementation of a BDI architecture: IRMA
[Bratman, Israel, Pollack, 1988] M.E. BRATMAN, D.J. ISRAEL et M. E. POLLACK. Plans and resource-bounded practical reasoning, Computational Intelligence, Vol. 4, No. 4, 1988, p
PRS
[Georgeff, Ingrand, 1989] M. P. GEORGEFF et F. F. INGRAND. Decision-making in an embedded reasoning system, dans Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (IJCAI-89), 1989, p
Successor of PRS: dMARS
[D'Inverno, 1997] M. D'INVERNO et al. A formal specification of dMARS, dans Intelligent Agents IV, A. Rao, M.P. Singh et M. Wooldrige (eds), LNAI Volume 1365, Springer-Verlag, 1997, p
Subsumption architecture
[Brooks, 1991] R. A. BROOKS. Intelligence without reasoning, dans Actes de 12th International Joint Conference on Artificial Intelligence (IJCAI-91), 1991, p 36

37. TuringMachine
[Ferguson, 1992] I. A. FERGUSON. TuringMachines: An Architecture for Dynamic, Rational, Mobile Agents, Thèse de doctorat, University of Cambridge, UK, 1992.
InteRRaP
[Muller, 1997] J. MULLER. A cooperation model for autonomous agents, dans Intelligent Agents III, LNAI Volume 1193, J.P. Muller, M. Wooldrige et N.R. Jennings (eds), Springer-Verlag, 1997, p
BDI Implementations
The Agent Oriented Software Group
Third generation BDI agent system using a component based approached. Implemented in Java
JASON 37