1. Learning Agents MSE 2400 EaLiCaRA Spring 2015 Dr. Tom Way

2. Learning Agent An Agent that observes its performance and adapts its decision-making to improve its performance in the future. MSE 2400 Evolution & Learning2

3. Agent Something that does something Computational Agent – a computer that does something MSE 2400 Evolution & Learning3

4. Simple Reflex Agent MSE 2400 Evolution & Learning4

5. Simple Reflex Agent The action to be selected only depends on the most recent percept, not a percept sequence As a result, these agents are stateless devices which do not have memory of past world states MSE 2400 Evolution & Learning5

6. Model-based Reflex Agent MSE 2400 Evolution & Learning6

7. Model-based Reflex Agent Have internal state which is used to keep track of past states of the world (i.e., percept sequences may determine action) Can assist an agent deal with at least some of the on observed aspects of the current state MSE 2400 Evolution & Learning7

8. Goal-based Agent MSE 2400 Evolution & Learning8

9. Goal-based Agent Agent can act differently depending on what the final state should look like Example: automated taxi driver will act differently depending on where the passenger wants to go MSE 2400 Evolution & Learning9

10. Utility-based Agent MSE 2400 Evolution & Learning10

11. Utility-based Agent An agent's utility function is an internalization of the performance measure (which is external) Performance and utility may differ if the environment is not completely observable or deterministic MSE 2400 Evolution & Learning11

12. Learning Agent (in general) MSE 2400 Evolution & Learning12

13. Learning Agent Parts (1) Environment – world around the agent Sensors – data input, senses Critic – evaluates the input from sensors Feedback – refined input, extracted info Learning element – stores knowledge Learning goals – tells what to learn MSE 2400 Evolution & Learning13

14. Learning Agent Parts (2) Problem generator – test what is known Performance element – considers all that is known so far, refines what is known Changes – new information Knowledge – improved ideas & concepts Actuators – probes environment, triggers gathering of input in new ways MSE 2400 Evolution & Learning14

15. Intelligent Agents should… accommodate new problem solving rules incrementally adapt online and in real time be able to analyze itself in terms of behavior, error and success. learn and improve through interaction with the environment (embodiment) learn quickly from large amounts of data have memory-based exemplar storage and retrieval capacities have parameters to represent short and long term memory, age, forgetting, etc. MSE 2400 Evolution & Learning15

16. Classes of Intelligent Agents (1) MSE 2400 Evolution & Learning16 Decision Agents – for decision making Input Agents - that process and make sense of sensor inputs (neural networks) Processing Agents - solve a problem like speech recognition Spatial Agents - relate to physical world

17. Classes of Intelligent Agents (2) MSE 2400 Evolution & Learning17 World Agents - incorporate a combination of all the other classes of agents to allow autonomous behaviors Believable agents - exhibits a personality via the use of an artificial character for the interaction

18. Classes of Intelligent Agents (3) MSE 2400 Evolution & Learning18 Physical Agents - entity which percepts through sensors and acts through actuators. Temporal Agents - uses time based stored information to offer instructions to a computer program or human being and uses feedback to adjust its next behaviors.

19. How Learning Agents Acquire Knowledge Supervised Learning –Agent told by teacher what is best action for a given situation, then generalizes concept F(x) Inductive Learning –Given some outputs of F(x), agent builds h(x) that approximates F on all examples seen so far is SUPPOSED to be a good approximation for as yet unseen examples MSE 2400 Evolution & Learning19

20. How Learning Agents Acquire Concepts (1) Incremental Learning: update hypothesis model only when new examples are encountered Feedback Learning: agent gets feedback on quality of actions it chooses given the h(x) it learned so far. MSE 2400 Evolution & Learning20

21. How Learning Agents Acquire Concepts (2) Reinforcement Learning: rewards / punishments prod agent into learning Credit Assignment Problem: agent doesn’t always know what the best (as opposed to just good) actions are, nor which rewards are due to which actions. MSE 2400 Evolution & Learning21

22. Examples Eliza - https://apps.facebook.com/eliza-chatbot/https://apps.facebook.com/eliza-chatbot/ Amy - http://help2.talktalk.co.uk/http://help2.talktalk.co.uk/ Ask the Candidates - http://askthecandidates2012.com/http://askthecandidates2012.com/ Mike - http://www.rong-chang.com/tutor_mike.htmhttp://www.rong-chang.com/tutor_mike.htm iEinstein - http://www.pandorabots.com/pandora/talk?botid=ea77c0 200e365cfb http://www.pandorabots.com/pandora/talk?botid=ea77c0 200e365cfb Talking Robot - http://talkingrobot.orghttp://talkingrobot.org Chatbots - http://www.chatbots.org/http://www.chatbots.org/ MSE 2400 Evolution & Learning22