1. Logistics Problem Set 1 Office Hours Reading Mailing List Dan’s Travel
Due 10/13 at start of class
Office Hours
Monday 3:30pm – o for another time
Reading
R&N ch 6 (skip 5 for now)
Mailing List
Last reminder to sign up via course web page
Dan’s Travel
Off until Mon morning 
If problems on PS, make best assumption
© Daniel S. Weld

2. Knowledge Representation
573 Topics
Perception
NLP
Multi-agent
Robotics
Reinforcement
Learning
MDPs
Supervised
Learning
Planning
Uncertainty
Search
Knowledge Representation
Problem Spaces
Agency
© Daniel S. Weld

3. Search Problem spaces Blind
Depth-first, breadth-first, iterative-deepening, iterative broadening
Informed
Best-first, Dijkstra's, A*, IDA*, SMA*, DFB&B, Beam, hill climb,limited discrep, AO*, LAO*, RTDP
Local search
Heuristics
Evaluation, construction, learning
Pattern databases
Online methods
Techniques from operations research
Constraint satisfaction
Adversary search
© Daniel S. Weld

4. Combinatorial Optimization
Nonlinear programs
Convex
Programs
Integer
Programming
(NPC)
Linear
Programs
(poly time)
Local optimality  global optimality
Kuhn Tucker conditions for optimality
Flow &
Matching
(fast!)
© Daniel S. Weld

5. Genetic Algorithms Start with random population Produce new generation
Representation serialized
States are ranked with “fitness function”
Produce new generation
Select random pair(s):
probability ~ fitness
Randomly choose “crossover point”
Offspring mix halves
Randomly mutate bits
Selection Crossover Mutation  
© Daniel S. Weld

6. Properties Randomized, parallel, beam search
Using fitness function
Importance of careful representation
© Daniel S. Weld

7. © Daniel S. Weld

8. } } Experiment Fitness Initialized to 300 random examples
Given 166 random problem instances
Fitness = number of these problems solved
Initialized to 300 random examples
Results after 10 generations
The following program discovered
Solves all 166
(EQ (DU (MT CS) (NOT CS) (DU (MS NN) (NOT NN))) } }
Move all blocks to table
Build correct stack
© Daniel S. Weld

9. © Daniel S. Weld

10. © Daniel S. Weld

11. Koza Block Stacking Learn a program which stacks blocks
Initial blocks can be in any orientation
Program should make a tower spelling “Universal”
Clever Representation
CS = name of block on top of current stack
TB = name of top block st it and lower blocks OK
NN = name next block needed above TB
Imagine if blocks described using x,y coords!
© Daniel S. Weld

12. Available Actions (MS x) (MT x) (EQ x y) (NOT x) (DU x y)
if x is on table, move it to top of stack
(MT x)
If x is somewhere in the stack, move to table the topmost block
(EQ x y)
Returns T if x equals y
(NOT x)
(DU x y)
Do x until y returns T
Any problems?
© Daniel S. Weld

13. Eisenstein’s Representation
onRammed
onHit
Gun
Input
onScan
Base
Other actuators
Each AFSM is a REX-like program
Fixed-length encoding
64 operations per AFSM
~2000 bits per genome
Adapted from Jacob Eisenstein presentation
© Daniel S. Weld

14. Training Scaled fitness Mutation pegged to diversity
Typical parameters
individuals
10% copy, 88% crossover, 2% elitism
This takes a LONG TIME!!!
Sample from ~25 starting positions
Up to 50,000 battles per generation
seconds per battle
20 minutes to 3 hours per generation
Adapted from Jacob Eisenstein presentation
© Daniel S. Weld

15. Results Fixed starting position, one opponent
GP crushes all opposition
Beats “showcase” tank
Randomized starting positions
Wins 80% of battles against “learning” tank
Wins 50% against “showcase” tank
Multiple opponents
Beats 4 out of 5 “learning” tanks
Both…
Unsuccessful
Adapted from Jacob Eisenstein presentation
© Daniel S. Weld

16. Example Program Function Input 1 Input 2 Output 1. Random ignore
2. Divide
Const_1
Const_2
3. Greater Than
Line 1
Line 2
4. Normalize Angle
Enemy bearing
5. Absolute Value
Line 4
6. Less Than
Const_90
7. And
Line 6
Line 3
8. Multiply
Const_10
9. Less Than
Enemy distance
Line 8
10. And
Line 9
Line 7
11. Multiply
Line 10
12. Output
Turn gun left
Line 11
0.87
0.5 1
-50 50 1 1
100
Adapted from Jacob Eisenstein presentation
© Daniel S. Weld

17. Functions Greater than, less than, equal + - * / % Absolute value
Random number
Constant
And, or, not
Normalize relative angle
Adapted from Jacob Eisenstein presentation
© Daniel S. Weld

18. Search          Problem spaces Blind
Depth-first, breadth-first, iterative-deepening, iterative broadening
Informed
Best-first, Dijkstra's, A*, IDA*, SMA*, DFB&B, Beam, hill climb, limited discrep,AO*, LAO*, RTDP
Local search
Heuristics
Evaluation, construction via relaxation
Pattern databases
Online methods
Techniques from operations research
Constraint satisfaction
Adversary search        
© Daniel S. Weld

19. Admissable Heuristics
f(x) = g(x) + h(x)
g: cost so far
h: underestimate of remaining costs
Where do heuristics come from?
© Daniel S. Weld

20. Relaxed Problems
Derive admissible heuristic from exact cost of a solution to a relaxed version of problem
For transportation planning, relax requirement that car has to stay on road  Euclidean dist
For blocks world, distance = # move operations heuristic = number of misplaced blocks
What is relaxed problem?
# out of place = 2, true distance to goal = 3
Cost of optimal soln to relaxed problem  cost of optimal soln for real problem
© Daniel S. Weld

21. Simplifying Integrals
vertex = formula
goal = closed form formula without integrals
arcs = mathematical transformations
heuristic = number of integrals still in formula
what is being relaxed?
© Daniel S. Weld

22. Heuristics for eight puzzle
8 3
7 8 
start
goal
What can we relax?
© Daniel S. Weld

23. Importance of Heuristics
8 5
h1 = number of tiles in wrong place
h2 =  distances of tiles from correct loc
D IDS A*(h1) A*(h2)
© Daniel S. Weld

24. Need More Power! Performance of Manhattan Distance Heuristic
8 Puzzle < 1 second
15 Puzzle 1 minute
24 Puzzle years
Need even better heuristics!
Adapted from Richard Korf presentation
© Daniel S. Weld

25. Subgoal Interactions Manhattan distance assumes Underestimates because
Each tile can be moved independently of others
Underestimates because
Doesn’t consider interactions between tiles
Adapted from Richard Korf presentation
© Daniel S. Weld

26. Pattern Databases Pick any subset of tiles Precompute a table
[Culberson & Schaeffer 1996]
Pick any subset of tiles
E.g., 3, 7, 11, 12, 13, 14, 15
Precompute a table
Optimal cost of solving just these tiles
For all possible configurations
57 Million in this case
Use breadth first search back from goal state
State = position of just these tiles (& blank)
Adapted from Richard Korf presentation
© Daniel S. Weld

27. Using a Pattern Database
As each state is generated
Use position of chosen tiles as index into DB
Use lookup value as heuristic, h(n)
Admissible?
Adapted from Richard Korf presentation
© Daniel S. Weld

28. Combining Multiple Databases
Can choose another set of tiles
Precompute multiple tables
How combine table values?
E.g. Optimal solutions to Rubik’s cube
First found w/ IDA* using pattern DB heuristics
Multiple DBs were used (dif subsets of cubies)
Most problems solved optimally in 1 day
Compare with 574,000 years for IDDFS
Adapted from Richard Korf presentation
© Daniel S. Weld

29. Drawbacks of Standard Pattern DBs
Since we can only take max
Diminishing returns on additional DBs
Would like to be able to add values
Adapted from Richard Korf presentation
© Daniel S. Weld

30. Disjoint Pattern DBs Partition tiles into disjoint sets During search
Partition tiles into disjoint sets
For each set, precompute table
E.g. 8 tile DB has 519 million entries
And 7 tile DB has 58 million
During search
Look up heuristic values for each set
Can add values without overestimating!
Manhattan distance is a special case of this idea where each set is a single tile
Adapted from Richard Korf presentation
© Daniel S. Weld

31. Performance 15 Puzzle: 2000x speedup vs Manhattan dist
IDA* with the two DBs shown previously solves 15 Puzzles optimally in 30 milliseconds
24 Puzzle: 12 million x speedup vs Manhattan
IDA* can solve random instances in 2 days.
Requires 4 DBs as shown
Each DB has 128 million entries
Without PDBs: years
Adapted from Richard Korf presentation
© Daniel S. Weld