1. ECE457 Applied Artificial Intelligence Spring 2008 Lecture #3
Informed Search
ECE457 Applied Artificial Intelligence
Spring 2008
Lecture #3

2. Outline Heuristics Informed search techniques More on heuristics
Iterative improvement
Russell & Norvig, chapter 4
Skip “Genetic algorithms” pages (will be covered in Lecture 12)
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 2

3. Recall: Uninformed Search
Travel blindly until they reach Bucharest
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 3

4. An Idea…
It would be better if the agent knew whether or not the city it is travelling to gets it closer to Bucharest
Of course, the agent doesn’t know the exact distance or path to Bucharest (it wouldn’t need to search otherwise!)
The agent must estimate the distance to Bucharest
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 4

5. Heuristic Function More generally:
We want the search algorithm to be able to estimate the path cost from the current node to the goal
This estimate is called a heuristic function
Cannot be done based on problem formulation
Need to add additional information
Informed search
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 5

6. Heuristic Function Heuristic function h(n) Path cost g(n)
h(n): estimated cost from node n to goal
h(n1) < h(n2) means it’s probably cheaper to get to the goal from n1
h(ngoal) = 0
Path cost g(n)
Evaluation function f(n)
f(n) = g(n) Uniform Cost
f(n) = h(n) Greedy Best-First
f(n) = g(n) + h(n) A*
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 6

7. Greedy Best-First Search
f(n) = h(n)
Always expand the node closest to the goal and ignore path cost
Complete only if m is finite
Rarely true in practice
Not optimal
Can go down a long path of cheap actions
Time complexity = O(bm)
Space complexity = O(bm)
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 7

8. Greedy Best-First Search
Upper-bound case: goal is last node of the tree
Number of nodes generated: b nodes for each node of m levels (entire tree)
Time and space complexity: all generated nodes O(bm)
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 8

9. A* Search f(n) = g(n) + h(n) Best-first search Complete
Optimal, given admissible heuristic
Never overestimates the cost to the goal
Optimally efficient
No other optimal algorithm will expand less nodes
Time complexity = O(bC*/є+1)
Space complexity = O(bC*/є+1)
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 9

10. A* Search
Upper-bound case: heuristic is the trivial h(n) = 0 A* becomes Uniform Cost Search
Goal has path cost C*, all other actions have minimum cost of є
Depth explored before taking action C*: C*/є
Depth of fringe nodes: C*/є + 1
Space & time complexity: all generated nodes: O(bC*/є+1) C* є є є є є є є є є є є є є є є
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 10

11. A* Search Using a good heuristic can reduce time complexity
Can go down to O(bm)
However, space complexity will always be exponential
A* runs out of memory before running out of time
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 11

12. Iterative Deepening A* Search
Like Iterative Deepening Search, but cut-off limit is f-value instead of depth
Next iteration limit is the smallest f-value of any node that exceeded the cut-off of current iteration
Properties
Complete and optimal like A*
Space complexity of depth-first search (because it’s possible to delete nodes and paths from memory when we explore down to the cut-off limit)
Performs poorly if small action cost (small step in each iteration)
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 12

13. Simplified Memory-Bounded A*
Uses all available memory
When memory limit reached, delete worst leaf node (highest f-value)
If equality, delete oldest leaf node
SMA memory problem
If the entire optimal path fills the memory and there is only one non-goal leaf node
SMA cannot continue expanding
Goal is not reachable
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 13

14. Simplified Memory-Bounded A*
Space complexity known and controlled by system designer
Complete if shallowest goal depth less than memory size
Shallowest goal is reachable
Optimal if optimal goal is reachable
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 14

15. Example: Greedy Search
Arad 366
Zerind 374
Sibiu 253
Timisoara 329
Fagaras 176
Rimnicu 193
Arad 366
Zerind 374
Sibiu 253
Timisoara 329
Fagaras 176
Rimnicu 193
Bucharest 0
Arad 366
Zerind 374
Sibiu 253
Timisoara 329
Arad 366
Zerind 374
Sibiu 253
Timisoara 329
Fagaras 176
Rimnicu 193
Bucharest 0
Arad 366
h(n) = straight-line distance
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 15

16. Example: A* Search h(n) = straight-line distance Arad 366 Zerind 449
Sibiu 393
Timisoara 447
Fagaras 415
Rimnicu 413
Pitesti 417
Craiova 526 Bucharest 450
Arad 366
Zerind 449
Sibiu 393
Timisoara 447
Fagaras 415
Rimnicu 413
Pitesti 417
Craiova 526 Bucharest 418
Arad 366
Zerind 449
Sibiu 393
Timisoara 447
Fagaras 415
Rimnicu 413
Pitesti 417
Craiova 526 Bucharest 418
Arad 366
Zerind 449
Sibiu 393
Timisoara 447
Fagaras 415
Rimnicu 413
Pitesti 417
Craiova 526
Arad 366
Zerind 449
Sibiu 393
Timisoara 447
Arad 366
Arad 366
Zerind 449
Sibiu 393
Timisoara 447
Fagaras 415
Rimnicu 413
h(n) = straight-line distance
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 16

17. Heuristic Function Properties
Admissible
Never overestimate the cost
Consistency / Monotonicity
h(np) ≤ h(nc) + cost(np,nc) h(np) + g(np) ≤ h(nc) + cost(np,nc) + g(np) h(np) + g(np) ≤ h(nc) + g(nc) f(np) ≤ f(nc)
f(n) never decreases as we get closer to the goal
Domination
h1(n) ≥ h2(n) for all n
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 17

18. Creating Heuristic Functions
Found by relaxing the problem
Straight-line distance to Bucharest
Eliminate constraint of traveling on roads
8-puzzle
Move each square that’s out of place (7)
Move by the number of squares to get to place (12)
Move some tiles in place
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 18

19. Creating Heuristic Functions
Block world
Move a block on the table or on another block
If there’s nothing on top of it
Possible heuristics for this game
+1 for each block in the wrong position
+1 for each block on top of the wrong block
+1 for every block in the support structure of each block with incorrect support
4 - 1 for every block with the correct support structure
Partial solving (get to A-B-?-?) B C D A A B C D
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 19

20. Creating Heuristic Functions
State h1 4 3 h2 2 1 h3 6 h4 h5
½(h3+h4) 5
2.5 B C D A B C D A B C D A B C D A
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 20

21. Creating Heuristic Functions
State h1 2 1 h2 h3 h4 h5
½(h3+h4)
1.5
0.5 A B C D A B C D A B C D A B C D
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 21

22. Path to the Goal Sometimes the path to the goal is irrelevant
Only the solution matters
n-queen puzzle
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 22

23. Different Search Problem
No longer minimizing path cost
Improve quality of state
Minimize state cost
Maximize state payoff
Iterative improvement
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 23

24. Example: Iterative Improvement
Minimize cost: number of attacks
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 24

25. Example: Travelling Salesman
Tree search method
Start with home city
Visit next city until optimal round trip
Iterative improvement method
Start with random round trip
Swap cities until optimal round trip
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 25

26. Graphic Visualisation
State value / state plot: state space
“State” axis can be states or specific properties
Neighbouring states on the axis are states linked by actions or with similar property values
State values are computed using a heuristic and do not include path cost
Value
State
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 26

27. Graphic Visualisation
State value / state plot: state space
Global maximum
Global minimum
Value
State
Local maxima
Local minima
Plateau
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 27

28. Graphic Visualisation
If state payoff is a complex mathematical function depending on one state property
-1x * x2 + sin2(x)/x + (1000-x)*cos(5x)/5x – x/10
State space: x  [10, 80]
Max: x = 74 payoff =
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 28

29. Graphic Visualisation
More complex state spaces can have several dimensions
Example: States are X-Y coordinates, state value is Z coordinate
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 29

30. Graphic Visualisation
Each state is a point on the map
Each state’s value is the distance to the CN Tower
Locations in water always have the worst value because we can’t swim
2D state space
X-Y coordinates of the agent
Z coordinate for state value
Red = minimum distance
Blue = maximum distance
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 30

31. Hill Climbing (Gradient Descent)
Simple but efficient local optimization strategy
Always take the action that most improves the state
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 31

32. Hill Climbing (Gradient Descent)
Generate random initial state
Each iteration
Generate and evaluate neighbours at step size
Move to neighbour with greatest increase/decrease (i.e. take one step)
End when there are no better neighbours
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 32

33. Example: Travelling to Toronto
Trying to get to downtown Toronto
Take steps toward the CN Tower
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 33

34. Hill Climbing (Gradient Descent)
Advantages
Fast
No search tree
Disadvantages
Gets stuck in local optimum
Does not allow worse moves
Solution dependant on initial state
Selecting step size
Common improvements
Random restarts
Intelligently-chosen initial state
Decreasing step size
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 34

35. Simulated Annealing
Problem with hill climbing: local best move doesn’t lead to optimal goal
Solution: allow bad moves
Simulated annealing is a popular way of doing that
Stochastic search method
Simulates annealing process in metallurgy
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 35

36. Annealing Tempering technique in metallurgy
Weakness and defects come from atoms of crystals freezing in the wrong place (local optimum)
Heating to unstuck the atoms (escape local optimum)
Slow cooling to allow atoms to get to better place (global optimum)
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 36

37. Simulated Annealing Annealing Simulated Annealing Atoms moving
Agent modifying state
towards minimum-energy location in crystal
towards state with global optimal value
while avoiding bad position.
while avoiding local optimum.
Atoms are more likely to move out of a bad position
Agents are more likely to accept bad moves
if the metal’s temperature is high.
if the “temperature” control parameter has a high value.
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 37

38. Simulated Annealing Annealing Simulated Annealing
The metal’s temperature starts hot,
The “temperature” control parameter starts with a high value,
then it cools off
then it decreases
continuously
incrementally
over time
with each iteration of the search
until the metal is room temperature
until it reaches a pre-set threshold.
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 38

39. Simulated Annealing Allow some bad moves
Bad enough to get out of local optimum
Not so bad as to get out of global optimum
Probability of accepting bad moves given
Badness of the move (i.e. variation in state value V)
Temperature T
P = e-V/T
Stochastic search technique
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 39

40. Simulated Annealing Generate random initial state and high temperature
Each iteration
Generate and evaluate a random neighbour
If neighbour better than current state
Accept
Else (if neighbour worse than current state)
Accept with probability e-V/T
Reduce temperature
End when temperature less than threshold
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 40

41. Simulated Annealing Advantages Disadvantage Avoids local optima
Very good at finding high-quality solutions
Very good for hard problems with complex state value functions
Disadvantage
Can be very slow in practice
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 41

42. Simulated Annealing Application
Traveling-wave tube (TWT)
Uses focused electron beam to amplify electromagnetic communication waves
Produces high-power radio frequency (RF) signals
Critical components in deep-space probes and communication satellites
Power efficiency becomes a key issue
TWT research group at NASA working for over 30 years on improving power efficiency
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 42

43. Simulated Annealing Application
Optimizing TWT efficiency
Synchronize electron velocity and phase velocity of RF wave
Using “phase velocity tapper” to control and decrease RF wave phase velocity
Improving tapper design improves synchronization, improves efficiency of TWT
Tapper with simulated annealing algorithm to optimize synchronization
Doubled TWT efficiency
More flexible then past tappers
Maximize overall power efficiency
Maximize efficiency over various bandwidth
Maximize efficiency while minimize signal distortion
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 43

44. Assumptions Goal-based agent Environment Fully observable
Deterministic
Sequential
Static
Discrete
Single agent
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 44

45. Assumptions Updated Utility-based agent Environment Fully observable
Deterministic
Sequential
Static
Discrete / Continuous
Single agent
ECE457 Applied Artificial Intelligence R. Khoury (2008) Page 45