Sisteme multi-agent Curs 2 Universitatea “Politehnica” din Bucuresti 2005 - 2006 Adina Magda Florea adina@cs.pub.ro http://turing.cs.pub.ro/blia_06

Modele arhitectura de agenti Structura conceptuala a agentilor Arhitecturi de agenti cognitivi Arhitecturi de agenti reactivi Arhitecturi stratificate

1. Structura conceptuala a agentilor 1.1 Rationalitatea unui agent Ce inseamna rationalitatea unui agent Cum putem masura rationalitatea unui agent? O masura a performantei 3

Un agent este situat in mediu Perpece mediul prin sensori si actioneaza asupra lui prin efectori Scop: proiectarea unui program – functie care realizeaza corespondenta sensori - efectori Agent = architectura + program Mediu accesibil vs. inaccesibil determinist vs. non-determinist static vs. dinamic discret vs. continu 4

1.2 Modelare agent E = {e1, .., e, ..} P = {p1, .., p, ..} A = {a1, .., a, ..} Agent reactiv see : E  P action : P  A env : E x A  E (env : E x A  P(E)) Decision component Agent action Perception Execution component component see action Environment env 5

Modelare agent Mai multi agenti reactivi see : E  P inter : P  I env : E x A1 x … An  P(E) inter : P  I action : P x I  A I = {i1,…,i,..} Agent (A1) Decision component Interaction component inter action Agent (A2) Perception Execution component component Agent (A3) see action Environment env 6

Modelare agent Agenti cognitivi action : S x I Ai next : S x P  S Agenti cu stare S = {s1,…,s,…} action : S x I Ai next : S x P  S inter : S x P  I see : E  P env : E x A1 x … An  P(E) 7

Modelare agent Agenti cu stare si scopuri Agenti cu utilitate goal : E  {0, 1} Agenti cu utilitate utility : E  R Mediu nedeterminist env : E x A  P(E) Probabilitatea estimata de un agent ca rezultatul unei actiuni (a) executata in e sa fie noua stare e’ 8

Modelare agent Agenti cu utilitate Utilitatea estimata (expected utility) a unei actiuni a intr-o stare e, dpv al agentului Principiul utilitatii estimate maxime Maximum Expected Utility (MEU) 9

Cum modelam? Iesirea din labirint Agent reactiv Agent cognitiv Agent cognitiv cu utilitate Probleme: Ce actiuni selectez Ce se face daca rezultatul actiunilor nu este cunoscut Cum se iau in considerare schimbarile din mediu 10

2. Arhitecturi de agenti cognitivi 2.1 Comportare rationala IA si Teoria deciziei IA Teoria deciziei Problema 1 = deliberare/decizie vs. actiune/proactivitate Problema 2 = limitarea resurselor 11

12 General cognitive agent architecture Interactions Information about itself - what it knows - what it believes - what is able to do - how it is able to do - what it wants environment and other agents - knowledge - beliefs Communication Reasoner Other agents Planner Control Scheduler& Executor Output State Input General cognitive agent architecture Environment 12

2.2 Modele LPOI 13 (a)Reguli de deductie Reprezentare simbolica + inferente – demonstrarea teoremelor pt a afla ce actiuni va face agentul Abordare declarativa (a)Reguli de deductie Predicate At(x,y), Free(x,y), Wall(x,y), Exit(dir), Do(action) Fapte si axiome despre mediu At(0,0) Wall(1,1) x y Wall(x,y)  Free(x,y) Reguli de deductie At(x,y)  Free(x,y+1)  Exit(east)  Do(move_east) Actualizare automata a starii curente si test pt starea scop At(0,3) 13

Modele LPOI (b) Utilizarea calcului situational = descrie schimbari utilizand formalismul logic Situatie = starea rezultata prin executarea unei actiuni Result(Action,State) = NewState At(location, situation) At((x,y), Si)  Free(x,y+1)  Exit(east)  At((x,y+1), Result(move_east,Si)) Scop At((0,3), _) + actiuni care au condus la scop means-end analysis 14

Avantaje LPOI Dezavantaje Avem nevoie de un alt model 15

Arhitecturi 2.3 BDI Specificatii de nivel inalt Means-end analysis Beliefs (convingeri) = informatii pe care agentul le are despre lume Desires (dorinte) = stari pe care agentul ar vrea sa le vada realizate Intentions (intentii) = dorinte (sau actiuni) pe care agentul s-a angajat sa le indeplineasca BDI – teoria rationamentului practic - Bratman, 1988 Rolul intentiilor 16

BDI Componenta filozofica Arhitectura software IRMA - Intelligent Resource-bounded Machine Architecture PRS - Procedural Reasoning System Componenta logica Rao & Georgeff, Wooldrige (Int Ai  )   (Bel Ai ) 17

Arhitectura BDI 18 percepts Belief revision Beliefs B = brf(B, p) Knowledge B = brf(B, p) Opportunity analyzer Deliberation process Desires D = options(B, D, I) Intentions Filter Means-end reasonner I = filter(B, D, I) Intentions structured in partial plans  = plan(B, I) Library of plans Plans Executor 18 actions

Proprietati ale intentiilor conduc means-end analysis limiteaza deliberare persista influenteaza convingerile Bucla de control a agentului B = B0 I = I0 D = D0 while true do get next perceipt p B = brf(B,p) D = options(B, D, I) I = filter(B, D, I)  = plan(B, I) execute() end while 19

(Commitment strategies) Strategii de angajare (Commitment strategies) Optiune aleasa de agent ca intentie – agentul s-a angajat pentru acea optiune Persistenta intentiilor Interbare: Cat timp se angajeaza un agent fata de o inetntie? Angajare oarba (Blind commitment) Angajare limitata (Single minded commitment) Angajare deschisa (Open minded commitment) 20

21 Bucla de control BDI angajare limitata B = B0 I = I0 D = D0 while true do get next perceipt p B = brf(B,p) D = options(B, D, I) I = filter(B, D, I)  = plan(B, I) while not (empty() or succeeded (I, B) or impossible(I, B)) do  = head() execute()  = tail() if not sound(, I, B) then end while Bucla de control BDI angajare limitata Dropping intentions that are impossible or have succeeded Reactivity, replan 21

22 Bucla de control BDI angajare deschisa B = B0 I = I0 D = D0 while true do get next perceipt p B = brf(B,p) D = options(B, D, I) I = filter(B, D, I)  = plan(B, I) while not (empty() or succeeded (I, B) or impossible(I, B)) do  = head() execute()  = tail() end while Bucla de control BDI angajare deschisa if reconsider(I, B) then Replan 22

3. Arhitecturi dea genti reactivi Arhitectura de subsumare - Brooks, 1986 (1) Luarea deciziilor = {Task Accomplishing Behaviours} Fiecare comportare (behaviour) = o functie ce realizeaza o actiune TAB – automate finite Implementare: situation  action (2) Mai multe comportari pot fi activate in paralel 23

Arhitectura de subsumare Un TAB este reprezentat de un modul de competenta (c.m.) Fiecarte c.m. executa un task simplu – comportare concreta c.m. opereaza in paralel Nivele inferiroare fata de cele superioare c.m. la nivele inferioare c.m. la nivele superioare  subsumtion architecture 24

25 Sensors Effectors Competence Module (2) Explore environ Module (0) Avoid obstacles Effectors Module (1) Move around Input (percepts) Output (actions) Competence Module (1) Move around Module 1 can monitor and influence the inputs and outputs of Module 2 M1 = move around while avoiding obstacles  M0 M2 = explores the environment looking for distant objects of interests while moving around  M1 Incorporating the functionality of a subordinated c.m. by a higher module is performed using suppressors (modify input signals) and inhibitors (inhibit output) Supressor node Inhibitor node Competence Module (0) Avoid obstacles 25

26 Comportare (c, a) – conditie-actiune; descrie comportarea R = { (c, a) | c  P, a  A} - multimea reguli de comportare   R x R – relatie binara totala de inhibare function action( p: P) var fired: P(R), selected: A begin fired = {(c, a) | (c, a)  R and p  c} for each (c, a)  fired do if   (c', a')  fired such that (c', a')  (c, a) then return a return null end 26

27 Comportare (1) Daca detectez obstacol atunci schimb directia Ne aflam pe o planeta necunoscuta care contine aur. Mostre de teren trebuie aduse la nava. Nu se stie daca sunt aur sau nu. Exsita mai multi agenti autonomi care nu pot comunica intre ei. Nava transmite semnale radio: gradient al campului Comportare (1) Daca detectez obstacol atunci schimb directia (2) Daca am mostre si sunt la baza atunci depune mostre (3) Daca am mostre si nu sunt la baza atunci urmez campul de gradient (4) Daca gasesc mostre atunci le iau (5) Daca adevarat atunci ma misc in mediu (1)  (2)  (3)  (4)  (5) Care sunt premisele pt ca acest comportament sa functioneze? (distributie a aurului?) Daca distributia este reala? 27

(1)  (2)  (3)  (4)  (5)  (6) 28 Agentii pot comunica indirect: - Depun si culeg boabe radiocative - Pot seziza aceste boabe radioactive (1) Daca detectez obstacol atunci schimb directia (2) Daca am mostre si sunt la baza atunci depune mostre (3) Daca am mostre si nu sunt la baza atunci depun boaba radioactiva si urmez campul de gradient (4) Daca gasesc mostre atunci le iau (5) Daca gasesc boabe radioactive atunci iau una si urmez campul de gradient (6) Daca adevarat atunci ma misc in mediu (1)  (2)  (3)  (4)  (5)  (6) 28

4. Arhitecturi stratificate Comportare reactiva si pro-activa Cel putin 2 straturi Horizontal layering - i/o horizontal Vertical layering - i/o vertical Action output Action output Layer n … Layer 2 Layer 1 Layer n … Layer 2 Layer 1 Layer n … Layer 2 Layer 1 Action output perceptual input Vertical Horizontal perceptual input perceptual input 29

Horizontal layering Vertical layering n comportari, n niveluri Comportarea globala poate fi inconsistenta Interactiuni intre niveluri: mn (m = nr actiuni pe nivel) Necesita un sistem de control Vertical layering Interactiuni intre niveluri m2(n-1) Nu sunt tolerante la defecte (daca un nivel se defecteaza) 30

TouringMachine - biblioteca de planuri 31 Horizontal layering – 3 niveluri de realizare a actiunilor Nivel reactiv - set de reguli situatie-actiune rules pt mediu Nivel planificare - comportare pro-activa - biblioteca de planuri Nivel modelare - reprezinta mediul, agentul si ceilalti agenti - stabileste scopuri - scopurile sunt trimise nivelului/stratului inferior Sistem de control 31

32 Nivel modelare perceptii Subsistem Nivel planificare perceptie actiune Nivel reactiv actiuni Subsistem control 32

InteRRaP 33 Stratificata BDI Principii 2 niveluri Atat controlul cat si BC sunt stratificate Controlul este bottom-up Fiecare nivel foloseste rezultatele nivelului inferior Fiecare nivel de control este format din: - modul recunoastere situatie / activare scop (SG) - modul planificare (PS) The prays and the predators (both are agents) move over a space represented in the form of a grid. The objective is for the predators to capture the pray animals by surrounding them as in the figure above. The following hypotheses are laid down: 1. The dimension of the environment is finite 2. The predator and pray animals move at fixed speeds and generally at the same speed. 3. The pray animals move in a random manner by making Brownian movements 4. The predators can use the corner and edges to block a pray animal's path 5. The predators have a limited perception of the world that surrounds them, which means that they can see the prey only if it is in one of the squares at a distance within their field of perception 33

34 I n t e R a P Social KB Planning KB World KB actions percepts Cooperative planning layer SG Social KB PS I n t e R a P Local planning layer SG PS Planning KB Behavior based layer SG PS World KB World interface Sensors Effectors Communication actions 34 percepts

BDI model in InteRRaP options filter SG plan PS 35 Beliefs Social model Mental model World model Situation Cooperative situation Local planning situation Routine/emergency sit. Goals Cooperative goals Local goals Reactions Sensors filter SG Options Cooperative option Local option Reaction Intentions Cooperative intents Local intentions Response Effectors Operational primitive Joint plans Local plans Behavior patterns plan PS 35

36 BDI Architectures First implementation of a BDI architecture: IRMA [Bratman, Israel, Pollack, 1988] M.E. BRATMAN, D.J. ISRAEL et M. E. POLLACK. Plans and resource-bounded practical reasoning, Computational Intelligence, Vol. 4, No. 4, 1988, p.349-355. PRS [Georgeff, Ingrand, 1989] M. P. GEORGEFF et F. F. INGRAND. Decision-making in an embedded reasoning system, dans Proceedings of the Eleventh International Joint Conference on Artificial Intelligence (IJCAI-89), 1989, p.972-978. Successor of PRS: dMARS [D'Inverno, 1997] M. D'INVERNO et al. A formal specification of dMARS, dans Intelligent Agents IV, A. Rao, M.P. Singh et M. Wooldrige (eds), LNAI Volume 1365, Springer-Verlag, 1997, p.155-176.   Subsumption architecture [Brooks, 1991] R. A. BROOKS. Intelligence without reasoning, dans Actes de 12th International Joint Conference on Artificial Intelligence (IJCAI-91), 1991, p.569-595. 36

TuringMachine [Ferguson, 1992] I. A. FERGUSON. TuringMachines: An Architecture for Dynamic, Rational, Mobile Agents, Thèse de doctorat, University of Cambridge, UK, 1992. InteRRaP [Muller, 1997] J. MULLER. A cooperation model for autonomous agents, dans Intelligent Agents III, LNAI Volume 1193, J.P. Muller, M. Wooldrige et N.R. Jennings (eds), Springer-Verlag, 1997, p.245-260. BDI Implementations The Agent Oriented Software Group Third generation BDI agent system using a component based approached. Implemented in Java http://www.agent-software.com.au/shared/home/ JASON http://jason.sourceforge.net/ 37