Done Done Course Overview What is AI? What are the Major Challenges? What are the Main Techniques? Where are we failing, and why? Step back and look at the Science Step back and look at the History of AI What are the Major Schools of Thought? What of the Future? Done

Course Overview What are the Main Techniques? (How do we do it?) What is AI? What are the Major Challenges? What are the Main Techniques? (How do we do it?) Where are we failing, and why? Step back and look at the Science Step back and look at the History of AI What are the Major Schools of Thought? What of the Future?

Course Overview What are the Main Techniques? (How do we do it?) What is AI? What are the Major Challenges? What are the Main Techniques? (How do we do it?) Where are we failing, and why? Step back and look at the Science Step back and look at the History of AI What are the Major Schools of Thought? What of the Future? Search Logics (knowledge representation and reasoning) Planning Bayesian belief networks Neural networks Evolutionary computation Reinforcement learning Language parsing and speech techniques Statistical methods (language, learning)

Course Overview What are the Main Techniques? (How do we do it?) What is AI? What are the Major Challenges? What are the Main Techniques? (How do we do it?) Where are we failing, and why? Step back and look at the Science Step back and look at the History of AI What are the Major Schools of Thought? What of the Future? Search Logics (knowledge representation and reasoning) Planning Bayesian belief networks Neural networks Evolutionary computation Reinforcement learning Language parsing and speech techniques Statistical methods (language, learning)

Planning We have done a sort of planning already Consider the “search” applied to general problem solving The sequence of moves with the “Jugs” was a plan Fill 3 litre Pour 3 litre into 4 litre Fill 3 litre … The sequence of moves in a game is a plan Why not apply same techniques for general planning? Try going to the shop to buy milk and a light bulb We need: Initial situation Goal situation Actions that can be done + cost of action Constraints

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap Eat bread

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap Eat bread Drink juice

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap Eat bread Drink juice Make a phone call

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap Eat bread Drink juice Make a phone call Browse the Web Go out the door

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap Eat bread Drink juice Make a phone call Browse the Web Go out the door Read a book

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap Eat bread Drink juice Make a phone call Browse the Web Go out the door Read a book Take another nap

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap Eat bread Drink juice Make a phone call Browse the Web Go out the door Read a book Take another nap Eat bread

At home, no milk, broken bulb Planning Initial situation: At home, no milk, broken bulb Read a book Take a nap Eat bread Drink juice Make a phone call Browse the Web Go out the door Read a book Take another nap Eat bread Go to Uni Go to friend Go to beach Go to food shop Go to clothes shop

Planning Why not apply same techniques for general planning? Try going to the shop to buy milk and light bulb We need: Initial situation Goal situation Actions that can be done + cost of action Constraints Problem is not tightly constrained (like jugs, or game) too many silly (irrelevant) actions We know they’re silly because of commonsense Solution (3 parts):

Planning Solution: Represent states, actions with logic sentences Start state is not just a node, but a description (NOT have(milk)) AND (NOT have (light_bulb)) AND my_location(home) Same for goal state Action is not node to node, where node is complete state buy(X)  achieves  have(X) Action does not affect other aspects Allow planner to add actions in any order Not necessary to work from the top, searching E.g. add subgoal “buy(milk)” before leaving house Do important or obvious parts first Note: state representation important here Divide and conquer Most things in the world are independent Can solve subgoals separately (compare with jugs/games)

STRIPS Planning (STRIPS = Stanford Research Institute Problem Solver) Initial state: (NOT have(milk)) AND (NOT have (light_bulb)) AND my_location(home) Goal state: have(milk) AND have (light_bulb) AND my_location(home) Actions: STRIPS operators Op Example: go(X) Precondition Must be true before action can be performed Example: my_location(Y) AND path (Y,X) Effect How action changes state, ADD facts and DELETE facts Example: ADD: my_location(X) DELETE: my_location(Y)

STRIPS Planning (STRIPS = Stanford Research Institute Problem Solver) go(food_shop) my_location(home) AND path (home,food_shop) ADD: my_location(food_shop) DELETE: my_location(home)

STRIPS Planning (STRIPS = Stanford Research Institute Problem Solver) go(food_shop) my_location(home) AND path (home,food_shop) ADD: my_location(food_shop) DELETE: my_location(home) go(X) my_location(Y) AND path (Y,X) ADD: my_location(X) DELETE: my_location(Y)

How to Search for a Plan? We could search forward from our initial state We saw that this would search loads of silly actions We could search backwards from our goal state Works better, but still searching silly actions No heuristic to find actions that get closer to initial state Need a heuristic… Means-ends analysis Find actions that reduce the difference between initial and goal states Newell and Simon’s General Problem Solver Generates heuristics from a table “Table of differences” identifies operators (actions) to reduce types of differences Needs a lot of human input

Newell & Simon " Human Problem Solving" 1972. “I want to take my son to nursery school. What’s the difference between what I have and what I want? One of distance. What changes distance? My automobile. My automobile doesn’t work. What is needed to make it work? A new battery. What has new batteries? An auto repair shop. I want the repair shop to put in a new battery; but the shop doesn’t know I need one. What is the difficulty? One of communication. What allows communication? A telephone...” Slavery of machine and implications Newell & Simon " Human Problem Solving" 1972.

How to Search for a Plan? We could search forward from our initial state We saw that this would search loads of silly actions We could search backwards from our goal state Works better, but still searching silly actions No heuristic to find actions that get closer to initial state Need a heuristic… Means-ends analysis Find actions that reduce the difference between initial and goal states Newell and Simon’s General Problem Solver Generates heuristics from a table “Table of differences” identifies operators (actions) to reduce types of differences Needs a lot of human input

How to Search for a Plan? Modern Approach: Search space of plans, not states Nodes can be partial bits of plans Search what action to add Backtrack if stuck Least commitment Leave choices to be worked out later if possible Variable values e.g. shop: buy(milk,X) Partial Ordering e.g. socks example What is a plan? Actions you will take Fix variable values in a step Ordering among actions Causal links

Start Left Sock Right Sock Left Shoe Right Shoe Finish

How to Search for a Plan? Modern Approach: Search space of plans, not states Nodes can be partial bits of plans Search what action to add Backtrack if stuck Least commitment Leave choices to be worked out later if possible Variable values e.g. shop: buy(milk,X) Partial Ordering e.g. socks example What is a plan? Actions you will take Fix variable values in a step Ordering among actions Causal links go(X) go(Y) X= food_shop Y= hardware_shop go(X) before go(Y) go(X) causes buy(milk,X)

How to Search for a Plan? go(X) go(Y) Y= hardware_shop go(X) causes buy(milk,X) X= food_shop go(X) go(Y) Y= hardware_shop go(X) causes buy(milk,X) X= food_shop go(X) before go(Y)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(food_shop) buy(milk) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? Black arrow indicates Causal Link INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) Black arrow indicates Causal Link (this implies Causal link protected and Ordering in this way too) my_location(food_shop) buy(milk) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) go(home,food_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) my_location(home) go(home,food_shop) go(home,hardware_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) my_location(home) go(home,food_shop) go(home,hardware_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) my_location(home) go(home,food_shop) go(home,hardware_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) my_location(home) clash go(home,food_shop) go(home,hardware_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) BACKTRACK Try a different way to achieve my_location(hardware_shop) my_location(home) go(home,food_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) Try a different way to achieve my_location(hardware_shop) my_location(home) my_location(food_shop) go(home,food_shop) go(food_shop,hardware_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) my_location(food_shop) go(home,food_shop) go(food_shop,hardware_shop) Clash! my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) Need a constraint on ordering my_location(home) my_location(food_shop) go(home,food_shop) go(food_shop,hardware_shop) Clash! my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? Red arrow indicates Ordering INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) my_location(food_shop) go(home,food_shop) go(food_shop,hardware_shop) my_location(food_shop) my_location(hardware_shop) Red arrow indicates Ordering buy(milk) buy(bulb) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) my_location(food_shop) go(home,food_shop) go(food_shop,hardware_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) go(hardware_shop,home) GOAL: have(milk) have (light_bulb) my_location(home)

How to Search for a Plan? INITIAL STATE: (NOT have(milk)) (NOT have (light_bulb)) my_location(home) my_location(home) my_location(food_shop) go(home,food_shop) go(food_shop,hardware_shop) my_location(food_shop) my_location(hardware_shop) buy(milk) buy(bulb) go(hardware_shop,home) GOAL: have(milk) have (light_bulb) my_location(home)

Industrial Planners Applications Other issues Assembly, Integration, Verification of spacecraft (European Space Agency) Space missions Job Shop scheduling (Hitachi) Other issues Hierarchical Top level: prepare booster, capsule, cargo, launch Low level: insert nuts, fasten bolts Conditional effects Depends on state Time e.g. window when machine is available Resources Budget Number of Workers Number of machines / robots Changing/uncertain world Conditional planning Action/execution monitoring

Planning – Recap… Problem solving was already a type of planning Why not use it for general planning? Other way: What about general planning for problem solving? Solution: Represent states, actions with logic sentences Allow planner to add actions in any order Divide and conquer Search… Forwards, Backwards, Heuristic? Search space of plans, not states What is a plan? Actions you will take Fix variable values in a step Ordering among actions Causal links Least commitment Variable values Partial Ordering Real world planning: Hierarchical, Conditional effects, Time, Resources, Changing/uncertain world

These are all in fact types of “Machine Learning” Course Overview What is AI? What are the Major Challenges? What are the Main Techniques? (How do we do it?) Where are we failing, and why? Step back and look at the Science Step back and look at the History of AI What are the Major Schools of Thought? What of the Future? Search Logics (knowledge representation and reasoning) Planning Bayesian belief networks Neural networks Evolutionary computation Reinforcement learning These are all in fact types of “Machine Learning”

These are all in fact types of “Machine Learning” Course Overview What is AI? What are the Major Challenges? What are the Main Techniques? (How do we do it?) Where are we failing, and why? Step back and look at the Science Step back and look at the History of AI What are the Major Schools of Thought? What of the Future? Search Logics (knowledge representation and reasoning) Planning Bayesian belief networks Neural networks Evolutionary computation Reinforcement learning These are all in fact types of “Machine Learning”

Biological Inspiration Artificial Neural Network (ANN) loosely based on biological neuron Each unit is simple, but many connected in a complex network If enough inputs are received Neuron gets “excited” Passes on a signal, or “fires” ANN different to biological: ANN outputs a single value Biological neuron sends out a complex series of spikes Biological neurons not fully understood Image from Purves et al., Life: The Science of Biology, 4th Edition, by Sinauer Associates and WH Freeman

Neural Net example: ALVINN Autonomous vehicle controlled by Artificial Neural Network Drives up to 70mph on public highways Note: most images are from the online slides for Tom Mitchell’s book “Machine Learning”

Neural Net example: ALVINN Autonomous vehicle controlled by Artificial Neural Network Drives up to 70mph on public highways

Neural Net example: ALVINN Sharp left Straight ahead Sharp right 30 output units 4 hidden units 1 input pixel Input is 30x32 pixels = 960 values

Neural Net example: ALVINN Sharp left Straight ahead Sharp right 30 output units 4 hidden units Learning means adjusting weight values 1 input pixel Input is 30x32 pixels = 960 values

Neural Net example: ALVINN Sharp left Straight ahead Sharp right 30 output units 4 hidden units 1 input pixel Input is 30x32 pixels = 960 values

Neural Net example: ALVINN

Neural Net example: ALVINN This shows one hidden node Input is 30x32 array of pixel values = 960 values Note: no special visual processing Size/colour corresponds to weight on link

Neural Net example: ALVINN Output is array of 30 values This corresponds to steering instructions E.g. hard left, hard right This shows one hidden node Input is 30x32 array of pixel values = 960 values Note: no special visual processing Size/colour corresponds to weight on link

The Perceptron add weight1 input1 weight2 output input2 weight3 (threshold) weight4 input3 input4

The Perceptron student first last year male works hard Lives in halls First this year 1 Richard 2 Alan 3 Alison 4 Jeff 5 Gail 6 Simon Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.2 _ Male output _ 0.2 Threshold = 0.5 0.2 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 1 Richard Note: example from Alison Cawsey

The Perceptron add First last year _ 0.15 0.15 _ Male output _ 0.2 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 1 Richard Note: example from Alison Cawsey

The Perceptron add First last year _ 0.15 0.15 _ Male output _ 0.2 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 2 Alan 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.2 _ Male output _ 0.25 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 2 Alan 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.2 _ Male output _ 0.25 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 3 Alison 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.2 _ Male output _ 0.25 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 4 Jeff 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.2 _ Male output _ 0.25 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 5 Gail 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.2 _ Male output _ 0.25 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 6 Simon 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.15 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 6 Simon 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.15 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 1 Richard Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.15 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 2 Alan 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.15 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 3 Alison 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.15 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 4 Jeff 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.2 0.15 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 5 Gail 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.25 0.15 _ Male output _ 0.25 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 5 Gail 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.25 0.15 _ Male output _ 0.25 Threshold = 0.5 0.15 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 6 Simon 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.25 0.10 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking _ Lives in halls student first last year male works hard Lives in halls First this year 6 Simon 1 Note: example from Alison Cawsey

The Perceptron add First last year _ 0.25 0.10 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking Apply idea in many applications _ Lives in halls Finished

The Perceptron add First last year _ 0.25 0.10 _ Male output _ 0.20 Threshold = 0.5 0.10 _ hardworking Apply idea in many applications _ Lives in halls Finished Ready to try unseen examples

The Perceptron add First last year 0.25 _ Male output 0.20 Threshold hardworking Lives in halls 0.10 Threshold = 0.5 0.20 Simple perceptron works ok for this example But sometimes will never find weights that fit everything In our example: Important: Getting a first last year, Being hardworking Not so important: Male, Living in halls Suppose there was an “exclusive or” Important: (male) OR (live in halls), but not both Can’t capture this relationship

The Perceptron If no weights fit all the examples… Could we find a good approximation? (i.e. won’t be correct 100% of the time) Our current training method looks at output 0 or 1 whenever it meets the examples that don’t fit: It will make the weights jump up and down It will never settle down to a best approximation What if we don’t “threshold” the output? Look at how big the error is rather than 0 or 1 Can add up the error over all examples Tells you how good current weights are