1. Logistics Problem Set 1 Mailing List Due 10/13 at start of class
Note start Robocode work
Mailing List
Sign up via course web page
© Daniel S. Weld

2. Robocode in PS1
© Daniel S. Weld

3. Tank fighting simulator
Human players code tanks in Java
~4000 tank control programs online
Directional “radar” sensor
Must be pointed at enemy to see (but noisefree obs)
Actuators
Moving, turning takes time
Gun must cool before firing
No terrain effects
Walled combat area
Real Time!
Adapted from Jacob Eisenstein presentation
© Daniel S. Weld

4. Outputs Inputs Self Enemy Forward / Backward Turn robot Turn radar
Position
Velocity
Heading
Energy
Gun Heat
Enemy
Distance
Bearing
Heading
Energy
Velocity
Outputs
Forward / Backward
Turn robot
Turn radar
Turn gun
Fire
Gun heat must be zero
Variable power
Adapted from Jacob Eisenstein presentation
© Daniel S. Weld

5. Knowledge Representation
573 Topics
Perception
NLP
Multi-agent
Robotics
Reinforcement
Learning
MDPs
Supervised
Learning
Planning
Uncertainty
Search
Knowledge Representation
Problem Spaces
Agency
© Daniel S. Weld

6. Search Problem spaces Blind
Depth-first, breadth-first, iterative-deepening, iterative broadening
Informed
Best-first, Dijkstra's, A*, IDA*, SMA*, DFB&B, Beam, hill climb,limited discrep, AO*, LAO*, RTDP
Local search
Heuristics
Evaluation, construction, learning
Pattern databases
Online methods
Techniques from operations research
Constraint satisfaction
Adversary search
© Daniel S. Weld

7. Last Time Agents Problem Spaces Blind Search DFS BFS
Iterative Deepening
Iterative Broadening
© Daniel S. Weld

8. Speed BFS Nodes Time Iter. Deep. Nodes Time 8 Puzzle 2x2x2 Rubik’s
Assuming 10M nodes/sec & sufficient memory
BFS
Nodes Time
Iter. Deep.
Nodes Time
8 Puzzle
2x2x2 Rubik’s
15 Puzzle
3x3x3 Rubik’s
24 Puzzle
sec
sec
days
k yrs
B yrs
sec
sec
k yrs
k yrs
yrs
1Mx 8x
Why the difference? Rubik has higher branch factor puzzle has greater depth
 # of duplicates
Adapted from Richard Korf presentation
© Daniel S. Weld

9. Model-Based Diagnosis 2 w 3 A +
y=12  D 2 v B 3 +
z=12
[10]  E 2 x 3 C
Model-based vs.
Library of fault models
Decision tree
Use model of correct behavior
Hypothesis
Generation
Testing
Discrimination
© Daniel S. Weld

10. Improving Generation Enumerate components (or sets of them)
Only if connected to discrepancy
Devices often have inputs & outputs
Output of suspect must be connected
Not every input influences an output
Information from multiple discrepancies
Constraint reasoning or
a=1
b=0
c=1
[0]
© Daniel S. Weld

11. Search for Robocode Controllers
Nodes?
Operators?
Start state?
Goal?
© Daniel S. Weld

12. Best-first Search Generalization of breadth first search
Priority queue of nodes to be explored
Cost function f(n) applied to each node
Add initial state to priority queue
While queue not empty
Node = head(queue)
If goal?(node) then return node
Add children of node to queue
© Daniel S. Weld

13. Old Friends Breadth first = best first
With f(n) = depth(n)
Dijkstra’s Algorithm = best first
With f(n) = g(n)
Where g(n) = sum of edge costs from start to n
Space bound (stores all generated nodes)
© Daniel S. Weld

14. { A* Search Hart, Nilsson & Rafael 1968
Best first search with f(n) = g(n) + h(n)
Where g(n) = sum of edge costs from start to n
And h(n) = estimate of lowest cost path n-->goal
If h(n) is admissible then search will find optimal
Underestimates cost
of any solution which
can reached from node {
Space bound since must maintain queue
© Daniel S. Weld

15. Iterative-Deepening A*
Like iterative-deepening depth-first, but...
Depth bound modified to be an f-limit
Start with limit = h(start)
Prune any node if f(node) > f-limit
Next f-limit=min-cost of any node pruned
FL=21 e
FL=15 d a f b c
© Daniel S. Weld

16. IDA* Analysis Complete & Optimal (ala A*)
Space usage  depth of solution
Each iteration is DFS - no priority queue!
# nodes expanded relative to A*
Depends on # unique values of heuristic function
In 8 puzzle: few values  close to # A* expands
In traveling salesman: each f value is unique
 1+2+…+n = O(n2) where n=nodes A* expands
if n is too big for main memory, n2 is too long to wait!
Generates duplicate nodes in cyclic graphs
© Daniel S. Weld

17. SMA* Problem with IDA*: f-limit increases slowly
Must do iterative search again and again
Storing little state between each iteration
Just one number: next highest contour level
SMA*
Uses all available memory to store state
Keeps bounded-size priority queue
When needs memory, drops node with highest f-cost
In ancestor of forgotten nodes, retains cost of best path which has been dropped. This focuses regeneration.
Duplicates minimal work
Optimal in a number of nice ways
© Daniel S. Weld

18. Depth-First Branch & Bound
Single DF search
 uses linear space
Keep track of best solution so far
If f(n) = g(n)+h(n)  cost(best-soln)
Then prune n
Requires
Finite search tree, or
Good upper bound on solution cost
Generates duplicate nodes in cyclic graphs
Adapted from Richard Korf presentation
© Daniel S. Weld

19. Beam Search Idea Evaluation
Best first but only keep N best items on priority queue
Evaluation
Complete?
Time Complexity?
Space Complexity? No
O(b^d)
O(b + N)
© Daniel S. Weld

20. Hill Climbing Idea Problems? Local maxima Plateaus Diagonal ridges
“Gradient ascent”
Idea
Always choose best child; no backtracking
Beam search with |queue| = 1
Problems?
Local maxima
Plateaus
Diagonal ridges
© Daniel S. Weld

21. Randomizing Hill Climbing
Randomly disobeying heuristic
Random restarts
[Add something on heavy tailed distribution]
© Daniel S. Weld

22. Simulated Annealing Objective: avoid local minima Technique:
For the most part use hill climbing
When no improvement possible
Choose random neighbor
Let  be the decrease in quality
Move to neighbor with probability e /T
Reduce “temperature” (T) over time
Pros & cons
Optimal?
If T decreased slowly enough, will reach optimal state
Widely used
See also
WalkSAT
temp
© Daniel S. Weld

23. Limited Discrepancy Search
Discrepancy bound indicates how often to violate heuristic
Iteratively increase... a d b e g c h f
Assume that heuristic says go left
© Daniel S. Weld

24. Genetic Algorithms Start with random population Produce new generation
Representation serialized
States are ranked with “fitness function”
Produce new generation
Select random pair(s):
probability ~ fitness
Randomly choose “crossover point”
Offspring mix halves
Randomly mutate bits
Crossover Mutation  
© Daniel S. Weld

25. Properties
Importance of careful reprsentation
© Daniel S. Weld