1. AI: Artificial Intelligence
Lecture 3
AI: Artificial Intelligence

2. Agents
An agent is anything that can be viewed as perceiving (getting input) its environment through sensors and acting upon that environment through actuators (tools used for actions).

4. Human Vs Robotic Vs Software Agent
A human agent has eyes, ears, and other organs for sensors and hands, legs, vocal tract, and so on for actuators.
A robotic agent might have cameras and infrared range finders for sensors and various motors for actuators.
A software agent receives keystrokes, file contents, and network packets as sensory inputs and acts on the environment by displaying on the screen, writing files, and sending network packets.

5. Perception
A  percept  is the input that an intelligent agent is perceiving at any given moment.
An percept sequence agent’s percept sequence is the complete history of everything the agent has ever perceived.
In general, an agent’s choice of action at any given instant can depend on the entire percept sequence observed to date, but not on anything it hasn’t perceived.

6. Example: Vacuum-Agent
Percepts: Location and status, e.g., [A,Dirty]
Actions: Left, Right, Suck, NOp
function Vacuum-Agent([location,status]) returns an action
if status = Dirty then return Suck
else if location = A then return Right
else if location = B then return Left
A percept is the input that an intelligent agent  is perceiving at any given moment.

7. Rational agents
For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and the agent’s built-in knowledge Performance measure (utility function): An objective criterion for success of an agent's behavior. (devised on environment states, not on agents states).
ExpectedUtility(action) = sum_outcomes Utility(outcome) * P(outcome|action)

8. Specifying the task environment
Problem specification: Performance measure, Environment, Actuators, Sensors (PEAS)
Example: Autonomous Taxi
Performance measure
Safe, fast, legal, comfortable trip, maximize profits
Environment
Roads, other traffic, pedestrians, customers
Actuators
Steering wheel, accelerator, brake, signal, horn
Sensors
Cameras, speedometer, GPS, engine sensors, keyboard

9. Another PEAS example: Spam filter
Performance measure
Minimizing false positives, false negatives
Environment
A user’s account, server
Actuators
Mark as spam, delete, etc.
Sensors
Incoming messages, other information about user’s account

10. Medical Diagnosis System
Performance measure
Healthy Patients, Minimize cost
Environment
Patients, Hospital, Staff
Actuators
Skin Display (questions, tests, diagnosis, treatments)
Sensors
Keyboard(entry of symptoms, patients answers)

11. Environment types
Fully observable vs. partially observable Deterministic vs. stochastic Episodic vs. sequential Static vs. dynamic Discrete vs. continuous Single agent vs. multi-agent Known vs. unknown
A stochastic process is simply a collection of random variables indexed by time. It will be useful to consider separately the cases of discrete time and continuous time. 

12. Fully observable vs. partially observable
Do the agent's sensors give it access to the complete state of the environment?
vs.

13. Fully Observable (accessible)
If an agent’s sensors give it access to the complete state of the environment at each point in time, then we say that the task environment is fully observable.
the sensors detect all aspects that are relevant to the choice of action; relevance, in turn, depends on the performance measure.
Fully observable environments are convenient because the agent need not maintain any internal state to keep track of the world.

14. Partially Observable (inaccessible)
An environment might be partially observable because of noisy and inaccurate sensors or because parts of the state are simply missing from the sensor data
Examples:
chess – the board is fully observable
Autonomous Taxi – what is around the next bend is not observable (yet).

15. Deterministic vs. stochastic
Is the next state of the environment completely determined by the current state and the agent’s action?
Strategic: the environment is deterministic except for the actions of other agents
vs.

16. Deterministic Vs Stochastic
Deterministic the next state of the environment is completely predictable from the current state and the action executed by the agent
Stochastic the next state has some uncertainty associated with it,
Uncertainty could come from randomness, lack of a good environment model, or lack of complete sensor coverage
Stochastic= variable (temperature)
Deterministic= constant (time table of an inst)

17. Episodic vs. sequential
Is the agent’s experience divided into unconnected episodes, or is it a coherent sequence of observations and actions?
vs.
Episodic: The agent's experience is divided into atomic “episodes,” and the choice of action in each episode depends only on the episode itself

18. Episodic Vs Sequential
Episodic The agent's experience is divided into atomic "episodes" (each episode consists of the agent perceiving and then performing a single action)
Sequential if current decisions affect future decisions, or rely on previous ones
Examples of episodic are expert advice systems – an episode is a single question and answer
Packman is an example of Sequential
Many are both – a number of episodes containing a number of sequential steps to a conclusion

19. Static vs. dynamic
Is the world changing while the agent is thinking?
Semidynamic: the environment does not change with the passage of time, but the agent's performance score does
vs.

20. Static Vs Dynamic Chess with clock is semidynamic
Static if nothing (other than the agent) in the environment changes
Dynamic if the environment may change over time.
If the environment itself does not change with the passage of time but the agent’s performance score does, then we say the environment is semidynamic.
CrossWord Puzzles are Static, aging, integers
Taxi Driving is dynamic
Chess with clock is semidynamic

21. Discrete vs. continuous
Does the environment provide a fixed number of distinct percepts, actions, and environment states?
Time can also evolve in a discrete or continuous fashion
vs.

22. Discrete vs. continuous
Discrete time moves in fixed steps, usually with one measurement per step (and perhaps one action, but could be no action). E.g. a game of chess
Continuous, Signals constantly coming into sensors, actions continually changing. E.g. driving a car

23. Single-agent vs. multi-agent
Is an agent operating by itself in the environment?
vs.

24. Single Vs Multi Agent
An agent operating by itself in an environment is single agent.
Multi agent is when other agents are present
A strict definition of an other agent is anything that changes from step to step, and can sense and act.
Other players in a football team (or opposing team), wind and waves in a sailing agent, other cars in a taxi driver

25. Known vs. unknown
Are the rules of the environment (transitions and rewards) known to the agent?
Strictly speaking, not a property of the environment, but of the agent’s state of knowledge
vs.

26. known environment, the outcomes (or outcome probabilities if the environment is stochastic) for all actions are given.
if the environment is unknown, the agent will have to learn how it works in order to make good decisions.
The distinction between known and unknown environments is not the same as the one between fully and partially observable environments. It is quite possible for a known environment to be partially observable—for example, in solitaire card games, I know the rules but am still unable to see the cards that have not yet been turned over. Conversely, an unknown environment can be fully observable—in a new video game, the screen may show the entire game state but I still don’t know what the buttons do until I try them.

27. Examples of different environments
Word jumble solver
Chess with a clock
Scrabble
Autonomous driving
Fully
Fully
Partially
Partially
Observable Deterministic Episodic Static Discrete Single agent
Deterministic
Strategic
Stochastic
Stochastic
Episodic
Sequential
Sequential
Sequential
Static
Semidynamic
Static
Dynamic
Discrete
Discrete
Discrete
Continuous
Single
Multi
Multi
Multi