1. More on Search: A* and Optimization
CPSC 315 – Programming Studio
Spring 2013
Project 2, Lecture 4
Adapted from slides of
Yoonsuck Choe

2. A* Algorithm Avoid expanding paths that are already expensive.
f(n) = g(n) + h(n)
g(n) = current path cost from start to node n
h(n) = estimate of remaining distance to goal
h(n) should never overestimate the actual cost of the best solution through that node.
The better h is, the better the algorithm works
Should be fast to compute

3. A* Then apply a best-first search
Add node with lowest overall estimate f
Update the states reachable from that node, if this offers a better path
If g(n) is lower than path previously found
Value of f will only increase as paths evaluate

4. 5 3 7 1 7 4 4 2 5 6 10 2

5. g=5
h=9
f=14 5 3 7 1 7
g=4
h=7
f=11 4 4 2 5 6 10 2
g=2
h=14
f=16

6. g=5
h=9
f=14 5 3 7 1 7
g=4
h=7
f=11 4 4 2 5 6 10 2
g=2
h=14
f=16
g=14
h=1
f=15

7. g=5
h=9
f=14
g=8
h=4
f=12 5 3 7 1 7
g=4
h=7
f=11
g=12
h=4
f=16 4 4 2 5 6 10 2
g=2
h=14
f=16
g=14
h=1
f=15

8. g=5
h=9
f=14
g=8
h=4
f=12
g=15
h=2
f=18 5 3 7 1 7
g=4
h=7
f=11
g=9
h=4
f=13 4 4 2 5 6 10 2
g=2
h=14
f=16
g=14
h=1
f=15

9. Improving Results and Optimization
Assume a state with many variables
Assume some function that you want to maximize/minimize the value of
Searching entire space is too complicated
Can’t evaluate every possible combination of variables
Function might be difficult to evaluate analytically

10. Iterative improvement
Start with a complete valid state
Gradually work to improve to better and better states
Sometimes, try to achieve an optimum, though not always possible
Sometimes states are discrete, sometimes continuous

11. Simple Example
One dimension (typically use more):
function
value x

12. Simple Example Start at a valid state, try to maximize function value

13. Simple Example
Move to better state
function
value x

14. Simple Example
Try to find maximum
function
value x

15. Hill-Climbing Choose Random Starting State Repeat
From current state, generate n random
steps in random directions
Choose the one that gives the best new
value
While some new better state found
(i.e. exit if none of the n steps were better)

16. Simple Example
Random Starting Point
function
value x

17. Simple Example
Three random steps
function
value x

18. Simple Example
Choose Best One for new position
function
value x

19. Simple Example
Repeat
function
value x

20. Simple Example
Repeat
function
value x

21. Simple Example
Repeat
function
value x

22. Simple Example
Repeat
function
value x

23. Simple Example
No Improvement, so stop.
function
value x

24. Problems With Hill Climbing
Random Steps are Wasteful
Addressed by other methods
Local maxima, plateaus, ridges
Can try random restart locations
Can keep the n best choices (this is also called “beam search”)
Comparing to game trees:
Basically looks at some number of available next moves and chooses the one that looks the best at the moment
Beam search: follow only the best-looking n moves

25. Gradient Descent (or Ascent)
Simple modification to Hill Climbing
Generallly assumes a continuous state space
Idea is to take more intelligent steps
Look at local gradient: the direction of largest change
Take step in that direction
Step size should be proportional to gradient
Tends to yield much faster convergence to maximum

26. Gradient Ascent
Random Starting Point
function
value x

27. Gradient Ascent Take step in direction of largest increase
(obvious in 1D, must be computed
in higher dimensions)
function
value x

28. Gradient Ascent
Repeat
function
value x

29. Gradient Ascent
Next step is actually lower, so stop
function
value x

30. Gradient Ascent
Could reduce step size to “hone in”
function
value x

31. Gradient Ascent
Converge to (local) maximum
function
value x

32. Dealing with Local Minima
Can use various modifications of hill climbing and gradient descent
Random starting positions – choose one
Random steps when maximum reached
Conjugate Gradient Descent/Ascent
Choose gradient direction – look for max in that direction
Then from that point go in a different direction
Simulated Annealing

33. Simulated Annealing
Annealing: heat up metal and let cool to make harder
By heating, you give atoms freedom to move around
Cooling “hardens” the metal in a stronger state
Idea is like hill-climbing, but you can take steps down as well as up.
The probability of allowing “down” steps goes down with time

34. Simulated Annealing Heuristic/goal/fitness function E (energy)
Generate a move (randomly) and compute
DE = Enew-Eold
If DE <= 0, then accept the move
If DE > 0, accept the move with probability:
Set
T is “Temperature”

35. Simulated Annealing Compare P(DE) with a random number from 0 to 1.
If it’s below, then accept
Temperature decreased over time
When T is higher, downward moves are more likely accepted
T=0 means equivalent to hill climbing
When DE is smaller, downward moves are more likely accepted

36. “Cooling Schedule” Speed at which temperature is reduced has an effect
Too fast and the optima are not found
Too slow and time is wasted

37. Simulated Annealing Random Starting Point T = Very High function valuex

38. Simulated Annealing
T = Very
High
Random Step
function
value x

39. Simulated Annealing Even though E is lower, accept T = Very High
function
value x

40. Simulated Annealing Next Step; accept since higher E T = Very High
function
value x

41. Simulated Annealing Next Step; accept since higher E T = Very High
function
value x

42. Simulated Annealing Next Step; accept even though lower T = High
function
value x

43. Simulated Annealing Next Step; accept even though lower T = High
function
value x

44. Simulated Annealing Next Step; accept since higher T = Medium function
value x

45. Simulated Annealing Next Step; lower, but reject (T is falling)
T = Medium
Next Step; lower, but reject (T is falling)
function
value x

46. Simulated Annealing Next Step; Accept since E is higher T = Medium
function
value x

47. Simulated Annealing Next Step; Accept since E change small T = Low
function
value x

48. Simulated Annealing Next Step; Accept since E larger T = Low function
value x

49. Simulated Annealing Next Step; Reject since E lower and T low T = Low
function
value x

50. Simulated Annealing Eventually converge to Maximum T = Low function
value x

51. Other Optimization Approach: Genetic Algorithms
State = “Chromosome”
Genes are the variables
Optimization Function = “Fitness”
Create “Generations” of solutions
A set of several valid solutions
Most fit solutions carry on
Generate next generation by:
Mutating genes of previous generation
“Breeding” – Pick two (or more) “parents” and create children by combining their genes

52. More on Optimization Lots of other variants/approaches
Range from heuristic methods to formal methods
A critical problem, constantly being studied