1. DESC9180 Designing Virtual Worlds Week 9 Interactivity II – Agent Models for Dynamic Virtual Worlds 6pm – 9pm Tuesday, September 18 th, 2007 Kathryn Merrick and Owen Macindoe DESC9180 Designing Virtual Worlds University of Sydney, September 2007

2. Lecture Overview  What is an agent?  The role of agents in virtual worlds  Designing agents for dynamic virtual worlds  Demonstrations and examples DESC9180 Designing Virtual Worlds University of Sydney, September 2007

3. What is an Agent?  A software process that:  Monitors its environment using sensors  Reasons about sensor data using some characteristic reasoning process  Acts in its environment using effectors DESC9180 Designing Virtual Worlds University of Sydney, September 2007

4. Other Properties of an Agent (I)  Reflexive: Pre-programmed responses to the state of the virtual environment  Reactive: Reasoning within a fixed set of goals  Reflective: Hypothesises about possible goals DESC9180 Designing Virtual Worlds University of Sydney, September 2007

5. Other Properties of an Agent (I)  Autonomous: Includes reasoning processes to create new goals  Proactive: Anticipate the needs or intentions of others DESC9180 Designing Virtual Worlds University of Sydney, September 2007 Implementation complexity scripted behaviours reactive, reflective agents reflexive agents autonomous, proactive agents Behavioural complexity

6. The Role of Agents in Virtual Worlds (I)  Agents provide a structured approach to implementing dynamic virtual worlds  Believable characters  Incredible objects  A single agent program may be applicable in many scenarios DESC9180 Designing Virtual Worlds University of Sydney, September 2007

7. The Role of Agents in Virtual Worlds (II)  Enemies or opposition  Partners  Support characters DESC9180 Designing Virtual Worlds University of Sydney, September 2007

8. Designing Agents for Dynamic Virtual Worlds DESC9180 Designing Virtual Worlds University of Sydney, September 2007 effectors Current state of the virtual world or some part of it action A change to the virtual world state reasoning sensors AGENT  Agents generally monitor and act in only a small part of the world  There may be multiple agents

9. A LSL Approach to Agents DESC9180 Designing Virtual Worlds University of Sydney, September 2007 default { state_entry() { state sensation; } state sensation { state_entry() { // Gather information // about the world… state reasoning; } state reasoning { state_entry() { // Process information // about the world… state reasoning; } state activation { state_entry() { // Perform an action… state sensation; } Sensation Reasoning Activation

10. Monitoring the Virtual World  Agents generally monitor only a small part of the world:  Monitor properties that are necessary for deciding how to act  What to monitor?  Self  Other objects or scripts  Avatars or chat  Virtual land DESC9180 Designing Virtual Worlds University of Sydney, September 2007

11. Self-Monitoring Objects  llGet… functions are useful for this:  Shape and location properties  Physics properties  Meta-data  Ownership and permissions  Script properties DESC9180 Designing Virtual Worlds University of Sydney, September 2007

12. Monitoring Other Objects and Avatars  Monitoring other objects:  llSensor (see last week’s lecture)  llDetected… functions  Monitoring avatars  llSensor for generic properties  llListen for chat  touch_start for avatar touches  collision for collisions DESC9180 Designing Virtual Worlds University of Sydney, September 2007

13. Monitoring the Virtual Land  llGet... And llGround… functions for:  Time and sun position  Parcel and region flags  Camera position  Properties of the ground DESC9180 Designing Virtual Worlds University of Sydney, September 2007

14. Reasoning: Artificial Intelligence for Virtual Worlds DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Artificial Intelligence (AI) techniques used in virtual worlds lag behind the current state of the art in AI research:  AI techniques are complex  Behaviour may be unpredictable  But some techniques have been used with great success…

15. Rule-Based Systems DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Define a set of rules about states of the virtual world  Rules have the form:  If then  Simple to implement for a single agent  A lot of work when there are multiple agents requiring different rules

16. Flocking DESC9180 Designing Virtual Worlds University of Sydney, September 2007  A special example of rule-based reasoning  Used to control groups  crowds or animals  Three rules governing:  Separation  Alignment  Cohesion Craig Reynold’s flocking rules for boids

17. State Machines DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Divide the reasoning process into internal states  Each state defines rules about:  How to act while in that state  When to transition to a different internal state  One of the most popular control strategies for dynamic virtual worlds

18. DESC9180 Designing Virtual Worlds University of Sydney, September 2007 AI Implant

19. Fuzzy Logic DESC9180 Designing Virtual Worlds University of Sydney, September 2007  A way for agents draw a conclusion based on facts that may be vague, ambiguous, inaccurate, incomplete Classical rulesFuzzy logic

20. Fuzzy Rules DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Fuzzy logic can be used to create a rule-base with rules of the form:  If then  Eg: if number of avatars is large then overcrowding is likely  Can be used in rule based approaches and state machines

21. Supervised Learning DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Agents gradually modify their internal structure in response to examples  Examples of desirable behaviour provided by human controlled avatars  Learning agents can adapt  Learning can be slow or inaccurate

22. How a Supervised Learner Works DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Many variants of supervised learning exist  Agent monitors:  State of virtual world  Actions performed by avatars  Agent stores:  State-action pair  Weight indicating confidence  Agent acts autonomously when a recognised state is encountered State 1Action10.3 State 2Action 50.9 State 3Unknown State 4Action 30.8

23. Reinforcement Learning DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Learning by trial and error  Agents modify their internal structure in response to reward or punishment from:  Human controlled avatars  Internal function  Learning agents can develop better policies than programmed agents  Learning can take a long time

24. How a Reinforcement Learner Works DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Many variants exist  Action is initially random trial and error  Agent monitors:  State of virtual world  Reward or punishment  Agent stores:  State-action pair  Cumulative reward or punishment  Agent acts by choosing action with highest reward (or lowest punishment) State 1Action10.9 Action 20.5 State 2Action 10.3 Action 20.8

25. Evolutionary Algorithms DESC9180 Designing Virtual Worlds University of Sydney, September 2007  A society of characters evolve individuals with advantageous characteristics Karl Sims’ evolved virtual creatures

26. Acting in the Virtual World  Individual agents generally act in or change only a small part of the world  Multiple agents may control larger regions of the virtual environment  How to act?  Changes to self  Avatars  Changes to other objects  Environment eg: chat DESC9180 Designing Virtual Worlds University of Sydney, September 2007

27. Self-Changing Objects  llSet… functions are useful for this:  Shape and location properties  Physics properties  Meta-data  Ownership and permissions  Script properties DESC9180 Designing Virtual Worlds University of Sydney, September 2007

28. Changing Avatars  You can’t put scripts in avatars…  But you can:  Apply an animation to an avatar  Attach scripted objects to an avatar to control:  Avatar location  Other avatar attachments DESC9180 Designing Virtual Worlds University of Sydney, September 2007

29. Changing Other Objects  You can’t directly change an object from a script inside a different object…  But you can:  Send messages to an object telling it how to change itself (remember the light switch example from last week) DESC9180 Designing Virtual Worlds University of Sydney, September 2007

30. Tutorial: Advanced LSL techniques DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Download the agent framework LSL script: http://www.it.usyd.edu.au/~kkas0686/desc9180 readings/Agent%20Framework.txt  Copy a sheep from the truck in Furness (236, 10, 55)  Write your own agent scripts to create some intelligent sheep (see tutorial sheet)

31. Next Week (s) DESC9180 Designing Virtual Worlds University of Sydney, September 2007  Next week (25/9) is the mid semester break  No lecture or tutorial  The following week (2/10) will be our paper review session  Please send me your slides in advance so I can have them installed, ready for class