1. Assignment 2 Observations
Mausam
© Mausam

2. The SAT Encoding Induced Subgraph Isomorphism Variables
NP complete problem
Applications: similarities between chemical compounds; protein-interaction networks, social networks, circuit design..
Variables
X(i,j) denotes that ith node (in smaller graph) mapped to jth node (in larger graph)
Self edge: If node i has a self-edge and node j doesn’t
remove X(i,j)
Self-edge: If node j has a self-edge and node i doesn’t
© Mausam

3. The SAT encoding Overall – O(nm) vars, O(n2m2) clauses
Each node maps to exactly one other node
X(i,1)  X(i,2)  X(i,3)…  X(i,n)
Pairs of  X(i,1)   X(i,2);  X(i,1)   X(i,3) …
Each node in larger graph maps to at most one node
Pairs of X(1,j)X(2,j); X(1,j)X(3,j); …
Disallowed pairs of mappings
If edge(i1,i2) and no edge(j1,j2): X(i1,j1)X(i2,j2);
If no edge(i1,i2) and edge(j1,j2): X(i1,j1)X(i2,j2);
If edge(i1,i2) and edge(j2,j1): X(i1,j1)X(i2,j2);
Overall – O(nm) vars, O(n2m2) clauses
© Mausam

4. Fun optimizations Degree trick: Can B ever map to P/S/R?
Yes. A mapping is: M(A) = S,
M(B) = Q, M(C) = R
The edges from P to other nodes
don‘t matter since no node in G
got mapped to P.
Degree trick: Can B ever map to P/S/R?
No: Indegree of B is greater than indegree of P/S/R
Remove X(B,P), X(B,S), X(B,R)
Common neighbor constraint: Can A-U and C-T?
They both don’t have an edge
They both have outdegrees 1
No: because AC have common neighbor 1; UT 0
X(A,U)X(C,T) … A C P R T B Q
© Mausam

5. Observations People solved ~100+80 node problems…
For an NP hard problem that’s not too bad, is it?
Paresh’s observations
adding redundant clauses helps miniSAT for tough problems.
© Mausam

6. Best Competitors
Kim Wallmark
Alan Ludwig
Haroon Barri
© Mausam

7. Assignment 3 Discussion
Mausam
© Mausam

8. BlackJack: State Space
MinSum: the minimum sum of the player's hand so far.
NumAces: the number of Aces in the player's hand so far.
dMinSum: the minimum sum of the dealer's hand so far.
dNumAces: the number of Aces in the dealer's hand so far.
isTwoCards: a Boolean that represents that the player's hand has just two cards so far.
isBlackJack: a Boolean that represents that the player got a BlackJack.
pair: has value between 0 and 10. Zero indicates that the player doesn't have a pair. Any other value i indicates that the player has two cards of value i.
turn: a Boolean to indicate whether it is player's turn or dealer's.
© Mausam

9. BlackJack: Action Space
Stand: Turn = 0
Hit: get a new card with prob… update minSum, numAces, isTwoCard, pair
Double: get a new card, update minSum, numAces, etc.;
Turn = 1.
Allowed when isTwoCard = 1
Default: run default dealer policy.
Allowed with turn = 1
Split: tricky!
Allowed with pair > 0; isTwoCard = 1
© Mausam

10. Equations
© Mausam

11. BlackJack Split (say only three cards)
Q((c,c), SPLIT)
2p1p1V(c,1)
p1p2V(c,1) + p1p2V(c,2)
p1p3V(c,1) p1p3V(c,3)
p2p1V(c,1) + p2p1V(c,2)
2p2p2V(c,2)
p2p3V(c,3) + p2p3V(c,3)
2p3p3V(c,3)
(c,c)
[(c,1) & (c,1)] p1p1
[(c,1) & (c,2)] p1p2
[(c,1) & (c,3)] p1p3
[(c,2) & (c,1)] p2p1
[(c,2) & (c,2)] p2p2
[(c,2) & (c,3)] p2p3
[(c,3) & (c,1)] p3p1
[(c,3) & (c,2)] p3p2
[(c,3) & (c,3)] p3p3
2p1(p1+p2+p3) = 2p1
© Mausam

12. Equations
© Mausam

13. Value of Terminal States
If isTwoCards = 1, minSum=11, numAces = 1
Return 1.5
If minSum>21
Return -1
If minSum < 21 and dMinSum > 21
Return 1 …
© Mausam

14. Can you solve it with Expectimax?No
Because there is possibility of infinite loop
Values are well-formed
Loop of size n becomes less probable as n increases
(see beginning example from lecture on MDPs)
EXCEPTION: Q(1010, split) for p> 0.5
Solution: use Value Iteration/Policy Iteration
© Mausam

15. Optimizations
Dealer’s policy is fixed. So, run the Markov chain ahead of time to generate a table
Pr(dealer gets 17|first card=c)
Pr(dealer gets 18|first card=c)
Pr(dealer gets 19|first card=c)
Pr(dealer gets 20|first card=c)
Pr(dealer gets 21|first card=c)
Pr(dealer busts|first card=c)
Remove dMinSum, dNumAces, turn.
Add dFirstCard
Remove Default action
Change the equations for double/stand based on computed probabilities
© Mausam

16. Comments numAces – not required. Convert to hasAce
Only first ace is relevant. After that each ace = 1
Handle infinite splits by allowing say, 10 splits
Should work fine in practice
Modification: have the bet amount in state space
Needn’t multiply by 2 in equations for doubling
Need to use this info when computing terminal rewards
© Mausam

17. Difference in solutions?
In real BJ: Dealer and player BJ is a push
In our BJ: player wins
???
In real BJ: there is a 4/6/8 card deck
In our BJ: infinite cards
You can easily modify code to handle 1st case
2nd case: Monte carlo sampling!
© Mausam

18. Should you play our BlackJack?
Expected reward before game starts
SUM[Pr(game)*expected reward of the game]
Deal three cards… and use V(s) to compute exp reward
My expected reward: $0.058
Yes! We should play the game
© Mausam

19. Discussion
Mausam
© Mausam

20. Followup Courses 546 Machine Learning/Data Mining
515 Probabilistic Graphical Models
517 Natural Language Processing
??? AI/Statistics for Big Data
571 Robotics
579 Intelligent control
574 Special Topics in AI
528 Computational Neuroscience
510 Human computer Interaction
© Mausam

21. Final Exam 50% objective 25% short answer questions
True/false
Multiple choice
Fill in the blanks
Match the following…
25% short answer questions
1 word – 5 line responses…
opinions/strengths/weaknesses/etc…
25% Problem solving
Problem modeling; short calculations…
© Mausam

22. Why do an exam? Goal: revise and analyze the whole course
Expected preparation time
~1 day
Go over lecture slides
Read parts of the notes that are unclear…
© Mausam

23. What worked Programming Assignments Written Assignments(?) Classes
You hated them
You loved them
You learned through them
Written Assignments(?)
Forced us to be regular
Classes
I absolutely enjoyed the interaction!
s/bulletin boards
Videos(?)
© Mausam

24. What didn’t work Programming environment Packed Workload
People didn’t know unix
People didn’t know Java!
Packed
Once someone got behind, it got tough to recover
Workload
© Mausam

25. Thanks
To all of you
who contributed to helping each other through newsgroup and other questions
who asked terrific questions in class
who gave regular feedback on class, assignments, grading, organization
© Mausam