1. 1 Computer Science Department California Polytechnic State University San Luis Obispo, CA, U.S.A. Franz J. Kurfess CPE/CSC 580: Intelligent Agents 1

2. 2 © Franz J. Kurfess Usage of the Slides ◆ these slides are intended for the students of my CPE/CSC 580 “Intelligent Agents” class at Cal Poly SLO ◆ if you want to use them outside of my class, please let me know (fkurfess@calpoly.edu)fkurfess@calpoly.edu ◆ some of them are based on other sources, which are identified and cited ◆ I usually select a subset for each quarter, either by hiding some slides, or creating a “Custom Show” (in PowerPoint) ◆ to view these, go to “Slide Show => Custom Shows”, select the respective quarter, and click on “Show” ◆ To print them, I suggest to use the “Handout” option ◆ 4, 6, or 9 per page works fine ◆ Black & White should be fine; there are few diagrams where color is important

3. 3 © Franz J. Kurfess Course Overview ❖ Introduction  Intelligent Agent, Multi-Agent Systems  Agent Examples ❖ Agent Architectures  Agent Hierarchy, Agent Design Principles ❖ Reasoning Agents  Knowledge, Reasoning, Planning ❖ Learning Agents  Observation, Analysis, Performance Improvement ❖ Multi-Agent Interactions  Agent Encounters, Resource Sharing, Agreements ❖ Communication  Speech Acts, Agent Communication Languages ❖ Collaboration  Distributed Problem Solving, Task and Result Sharing ❖ Agent Applications  Information Gathering, Workflow, Human Interaction, E-Commerce, Embodied Agents, Virtual Environments ❖ Conclusions and Outlook

4. 4 © Franz J. Kurfess Overview Agent Architectures ❖ Motivation ❖ Objectives ❖ Agent Design Principles ❖ Agent Hierarchy ❖ Intentional Systems ❖ Abstract Agent Architecture ❖ Reactive Agents ❖ Important Concepts and Terms ❖ Chapter Summary

5. 5 © Franz J. Kurfess Logistics ◆ Introductions ◆ Course Materials ◆ textbooks (see below) ◆ lecture notes ◆ PowerPoint Slides will be available on my Web page ◆ handouts ◆ Web page ◆ http://www.csc.calpoly.edu/~fkurfess http://www.csc.calpoly.edu/~fkurfess ◆ Term Project ◆ Lab and Homework Assignments ◆ Exams ◆ Grading

6. 6 © Franz J. Kurfess Textbooks ◆ Main Textbook ◆ Recommended for additional reading ❖ [Woolridge 2009] Michael Woolridge, An Introduction to Multi-Agent- Systems, Wiley, 2009. ❖ [Russell & Norvig 2009] Stuart Russell and Peter Norvig, Artificial Intelligence - A Modern Approach. 3 nd ed., Prentice Hall, 2009.

7. 7 © Franz J. Kurfess Bridge-In

8. 8 © Franz J. Kurfess Pre-Test

9. 9 © Franz J. Kurfess Motivation

10. 10 © Franz J. Kurfess Objectives

11. 11 © Franz J. Kurfess Objectives ❖ understand the notion of an agent  how agents are distinct from other software paradigms (e.g., objects),  characteristics of applications that lend themselves to an agent-oriented solution; ❖ understand the key issues associated with constructing agents  capable of intelligent autonomous action,  main approaches taken to developing such agents; ❖ understand the key issues and approaches to high-level communication  in multi-agent systems; ❖ understand the key issues in designing societies of agents  can effectively cooperate in order to solve problems; ❖ understand the main application areas of agent-based solutions; ❖ understand the main techniques for automated decision-making  in multi-agent systems  voting, forming coalitions, allocating scarce resources, and bargaining [Woolridge 2009]

12. 12 © Franz J. Kurfess Evaluation Criteria

13. 13 Agent Design Principles Autonomy Embodiment Belief, Desire, Intent Social Behavior

14. 14 [Woolridge 2009] Autonomous Agent An agent is  a computer system that is  capable of independent action on behalf of its user or owner SYSTEM ENVIRONMENT input output

15. 15 © Franz J. Kurfess Embodiment and Situatedness ❖ An embodied agent has a physical manifestation  often also called a robot  software agents typically are not embodied ❖ Agents are situated in an environment  often also referred to as context

16. 16 © Franz J. Kurfess Belief, Desire, Intention (BDI) ❖ software model developed for the design and programming of intelligent agentsintelligent agents ❖ implements the principal aspects of Michael Bratman's theory of human practical reasoningMichael Bratman theory of human practical reasoning http://en.wikipedia.org/wiki/Belief%E2%80%93desire%E2%80%93intention_software_model

17. 17 © Franz J. Kurfess Beliefs ❖ represent the informational state of the agent  beliefs about the world (including itself and other agents) ❖ beliefs can include inference rules  for the generation of new beliefs ❖ the term belief is used instead of knowledge  expresses the subjective nature  may change over time

18. 18 © Franz J. Kurfess Desires ❖ represent the motivational state of the agent  situations that the agent would like to achieve ❖ goals are desires adopted for active pursuit  sets of multiple goals should be consistent  sets of desires can be inconsistent

19. 19 © Franz J. Kurfess Intentions ❖ represent the deliberative state of the agent  the agent has chosen to do something ❖ intentions are desires to which the agent has committed  to some extent ❖ a plan is a sequences of actions to achieve an intention ❖ an event is a trigger for reactive activity by an agent

20. 20 [Woolridge 2009] Social Ability The real world is a multi-agent environment: we cannot go around attempting to achieve goals without taking others into account Some goals can only be achieved with the cooperation of others Similarly for many computer environments: witness the Internet Social ability in agents is the ability to interact with other agents (and possibly humans) via some kind of agent-communication language, and perhaps cooperate with others

21. 21 Agent Hierarchy Reflex Agent Model-Based Agent Goal/Utility-Based Agent Learning Agent Reasoning Agent

22. 22 © Franz J. Kurfess Reflex Agent Diagram 2 Sensors Actuators What the world is like nowWhat should I do now Condition-action rules Agent Environment

23. 23 © Franz J. Kurfess Model-Based Reflex Agent Diagram Sensors Actuators What the world is like nowWhat should I do now StateHow the world evolvesWhat my actions do Agent Environment Condition-action rules

24. 24 © Franz J. Kurfess Utility-Based Agent Diagram Sensors Actuators What the world is like nowWhat happens if I do an actionHow happy will I be thenWhat should I do now StateHow the world evolvesWhat my actions do Utility Agent Environment Goals

25. 25 © Franz J. Kurfess Learning Agent Diagram Sensors Actuators Agent Environment What the world is like now What happens if I do an action How happy will I be then What should I do now State How the world evolves What my actions do Utility Critic Learning Element Problem Generator Performance Standard

26. 26 Intentional Systems Agents as Intentional Systems The Need for Abstraction Representational Flexibility Post-Declarative Systems

27. 27 [Woolridge 2009] Agents as Intentional Systems When explaining human activity, it is often useful to make statements such as the following:  Janine took her umbrella because she believed it was going to rain.  Michael worked hard because he wanted to possess a PhD. Human behavior is predicted and explained through the attribution of attitudes,  such as believing and wanting, hoping, fearing,... The attitudes employed in such folk psychological descriptions are called the intentional notions

28. 28 [Woolridge 2009] Agents as Intentional Systems The philosopher Daniel Dennett coined the term intentional system  describes entities ‘whose behavior can be predicted by the method of attributing belief, desires and rational acumen’ different ‘grades’ of intentional systems:  first-order intentional system has beliefs and desires but no beliefs and desires about beliefs and desires.  second-order intentional system is more sophisticated; it has beliefs and desires about beliefs and desires also has other intentional states  together with beliefs and desires about those pther intentional states refers to states of others and its own

29. 29 [Woolridge 2009] Agents as Intentional Systems The answer seems to be that while the intentional stance description is consistent,... it does not buy us anything, since we essentially understand the mechanism sufficiently to have a simpler, mechanistic description of its behavior. (Yoav Shoham) Put crudely, the more we know about a system, the less we need to rely on animistic, intentional explanations of its behavior But with very complex systems, a mechanistic, explanation of its behavior may not be practicable As computer systems become ever more complex, we need more powerful abstractions and metaphors to explain their operation — low level explanations become impractical. The intentional stance is such an abstraction

30. 30 [Woolridge 2009] Intentional Systems as Abstraction the more we know about a system, the less we need to rely on animistic, intentional explanations of its behavior with very complex systems, a mechanistic, explanation of its behavior may not be practicable intentions can be used to describe complex systems at a higher level of abstraction  to express aspects like autonomy goals self-preservation social behavior

31. 31 [Woolridge 2009] Agents as Intentional Systems additional points in favor of this idea:  Characterizing Agents: provides a familiar, non-technical way of understanding & explaining agents  Nested Representations: offers the potential to specify systems that include representations of other systems widely accepted that such nested representations are essential for agents that must cooperate with other agents

32. 32 [Woolridge 2009] Post-Declarative Systems this view of agents leads to a kind of post-declarative programming:  In procedural programming, we say exactly what a system should do  In declarative programming, we state something that we want to achieve give the system general info about the relationships between objects, let a built-in control mechanism figure out what to do e.g., goal-directed theorem proving intentional agents  very abstract specification of the system  let the control mechanism figure out what to do knowing that it will act in accordance with some built-in theory of agency

33. 33 Abstract Agent Architecture Environment, States Actions, Runs State Transformations Agent as Function System

34. 34 [Woolridge 2009] Abstract Architecture for Agents Assume the environment may be in any of a finite set E of discrete, instantaneous states: Agents are assumed to have a repertoire of possible actions available to them, which transform the state of the environment: A run, r, of an agent in an environment is a sequence of interleaved environment states and actions:

35. 35 [Woolridge 2009] Abstract Architecture for Agents Let:  R be the set of all such possible finite sequences (over E and Ac)  R Ac be the subset of these that end with an action  R E be the subset of these that end with an environment state

36. 36 [Woolridge 2009] State Transformer Functions A state transformer function represents behavior of the environment: Note that environments are…  history dependent  non-deterministic If  (r)= ∅, then there are no possible successor states to r. In this case, we say that the system has ended its run Formally, we say an environment Env is a triple Env = 〈 E,e 0,  〉 where: E is a set of environment states, e 0 ∈ E is the initial state, and  is a state transformer function

37. 37 [Woolridge 2009] Agents Agent is a function which maps runs to actions: An agent makes a decision about what action to perform based on the history of the system that it has witnessed to date. Let AG be the set of all agents

38. 38 Computer Science Department California Polytechnic State University San Luis Obispo, CA, U.S.A. Franz J. Kurfess Systems 38

39. 39 Computer Science Department California Polytechnic State University San Luis Obispo, CA, U.S.A. Franz J. Kurfess Systems 39

40. 40 Reactive Agents Perception Agents with State Tasks Utility Functions

41. 41 [Woolridge 2009] Perception Now introduce perception system: Environment Agent seeaction

42. 42 [Woolridge 2009] Perception the see function is the agent’s ability to observe its environment, the action function represents the agent’s decision making process Output of the see function is a percept: see : E → Per  maps environment states to percepts action is now a function action : Per* → A  maps sequences of percepts to actions

43. 43 [Woolridge 2009] Agents with State We now consider agents that maintain state: Environment Agent action next state see

44. 44 [Woolridge 2009] Agents with State internal data structure  typically used to record information about the environment state and history. let I be the set of all internal states of the agent the perception function see for a state-based agent is unchanged: see : E → Per the action-selection function action is now defined as a mapping from internal states to actions: action : I → Ac An additional function next is introduced: next : I  Per → I  maps an internal state and percept to an internal state

45. 45 [Woolridge 2009] Agent Control Loop 1. Agent starts in some initial internal state i 0 2. Observes its environment state e, and generates a percept see(e) 3. Internal state of the agent is then updated via next function, becoming next(i 0, see(e)) 4. The action selected by the agent is action(next(i 0, see(e))) 5. Goto 2

46. 46 [Woolridge 2009] Tasks for Agents agents carry out tasks for users  tasks must be specified by users tell agents what to do without telling them how to do it

47. 47 [Woolridge 2009] Utility Functions over States associate utilities with individual states  the task of the agent is then to bring about states that maximize utility a task specification is a function u : E → ú  associates a real number with every environment state

48. 48 [Woolridge 2009] Utility Functions over States value of a run  minimum utility of state on run?  maximum utility of state on run?  sum of utilities of states on run?  average? disadvantage: difficult to specify a long term view when assigning utilities to individual states one possibility: a discount for states later on

49. 49 [Woolridge 2009] Utilities over Runs another possibility  assigns a utility not to individual states, but to runs themselves: u : R → ú  inherently long term view other variations  incorporate probabilities of different states emerging difficulties with utility-based approaches:  where do the numbers come from?  humans don’t think in terms of utilities  hard to formulate tasks in these terms

50. 50 © Franz J. Kurfess Post-Test

51. 51 © Franz J. Kurfess Evaluation ❖ Criteria

52. 52 © Franz J. Kurfess Summary Agent Architectures

53. 53 © Franz J. Kurfess Important Concepts and Terms ❖ agent ❖ agent society ❖ architecture ❖ deduction ❖ environment ❖ hybrid architecture ❖ intelligence ❖ intention ❖ multi-agent system ❖ reactivity ❖ subsumption

54. 54 © Franz J. Kurfess