1

Assignment 2 Observations

Mausam

© Mausam

2

The SAT Encoding

• Induced Subgraph Isomorphism– NP complete problem– Applications: similarities between chemical compounds; protein-

interaction networks, social networks, circuit design..

• Variables– X(i,j) denotes that ith node (in smaller graph) mapped to jth node

(in larger graph)– Self edge: If node i has a self-edge and node j doesn’t

• remove X(i,j)

– Self-edge: If node j has a self-edge and node i doesn’t• remove X(i,j)

© Mausam

3

The SAT encoding• Each node maps to exactly one other node– X(i,1) X(i,2) X(i,3)… X(i,n)– Pairs of X(i,1) X(i,2); X(i,1) X(i,3) …

• Each node in larger graph maps to at most one node– Pairs of X(1,j)X(2,j); X(1,j)X(3,j); …

• Disallowed pairs of mappings– If edge(i1,i2) and no edge(j1,j2): X(i1,j1)X(i2,j2); – If no edge(i1,i2) and edge(j1,j2): X(i1,j1)X(i2,j2); – If edge(i1,i2) and edge(j2,j1): X(i1,j1)X(i2,j2);

• Overall – O(nm) vars, O(n2m2) clauses© Mausam

4

Fun optimizations

• Degree trick: Can B ever map to P/S/R?– No: Indegree of B is greater than indegree of P/S/R– Remove X(B,P), X(B,S), X(B,R)

• Common neighbor constraint: Can A-U and C-T?– They both don’t have an edge– They both have outdegrees 1– No: because AC have common neighbor 1; UT 0– X(A,U)X(C,T)

• …

A C

B

P R

Q

SYes. A mapping is: M(A) = S, M(B) = Q, M(C) = R

The edges from P to other nodesdon‘t matter since no node in Ggot mapped to P.

T

U

© Mausam

5

Observations

• People solved ~100+80 node problems…– For an NP hard problem that’s not too bad, is it?

• Paresh’s observations– adding redundant clauses helps miniSAT for tough

problems.

© Mausam

6

Best Competitors

• Kim Wallmark

• Alan Ludwig

• Haroon Barri

© Mausam

7

Assignment 3 Discussion

Mausam

© Mausam

8

BlackJack: State Space

• MinSum: the minimum sum of the player's hand so far.• NumAces: the number of Aces in the player's hand so far.• dMinSum: the minimum sum of the dealer's hand so far.• dNumAces: the number of Aces in the dealer's hand so far.• isTwoCards: a Boolean that represents that the player's hand has

just two cards so far.• isBlackJack: a Boolean that represents that the player got a

BlackJack.• pair: has value between 0 and 10. Zero indicates that the player

doesn't have a pair. Any other value i indicates that the player has two cards of value i.

• turn: a Boolean to indicate whether it is player's turn or dealer's.© Mausam

9

BlackJack: Action Space

• Stand: Turn = 0• Hit: get a new card with prob… update minSum,

numAces, isTwoCard, pair• Double: get a new card, update minSum, numAces,

etc.; – Turn = 1. – Allowed when isTwoCard = 1

• Default: run default dealer policy. – Allowed with turn = 1

• Split: tricky! – Allowed with pair > 0; isTwoCard = 1

© Mausam

10

Equations

© Mausam

'

)'(),|'(),(s

sVHITssPHITsQ

'

)'(),|'(),(s

sVSTANDssPSTANDsQ

'

)'(),|'(),(s

sVDEFssPDEFsQ

'

)'(),|'(2),(s

sVDOUBssPDOUBsQ

11

BlackJack Split (say only three cards)• (c,c)– [(c,1) & (c,1)] p1p1

– [(c,1) & (c,2)] p1p2– [(c,1) & (c,3)] p1p3– [(c,2) & (c,1)] p2p1– [(c,2) & (c,2)] p2p2– [(c,2) & (c,3)] p2p3– [(c,3) & (c,1)] p3p1– [(c,3) & (c,2)] p3p2– [(c,3) & (c,3)] p3p3© Mausam

• Q((c,c), SPLIT)– 2p1p1V(c,1) – p1p2V(c,1) + p1p2V(c,2)– p1p3V(c,1) + p1p3V(c,3)– p2p1V(c,1) + p2p1V(c,2) – 2p2p2V(c,2)– p2p3V(c,3) + p2p3V(c,3)– p1p3V(c,1) + p1p3V(c,3)– p2p3V(c,3) + p2p3V(c,3)– 2p3p3V(c,3)

2p1(p1+p2+p3) = 2p1

12

Equations

© Mausam

'

)'(),|'(),(s

sVHITssPHITsQ

'

)'(),|'(),(s

sVSTANDssPSTANDsQ

'

)'(),|'(),(s

sVDEFssPDEFsQ

'

)'(),|'(2),(s

sVDOUBssPDOUBsQ

'

,...)',()'(2),...),,((c

ccVcPSPLITccQ

13

Value of Terminal States• If isTwoCards = 1, minSum=11, numAces = 1– Return 1.5

• If minSum>21– Return -1

• If minSum < 21 and dMinSum > 21– Return 1

• …• …

© Mausam

14

Can you solve it with Expectimax?

• No– Because there is possibility of infinite loop

• Values are well-formed– Loop of size n becomes less probable as n increases– (see beginning example from lecture on MDPs)– EXCEPTION: Q(1010, split) for p> 0.5

• Solution: use Value Iteration/Policy Iteration© Mausam

15

Optimizations• Dealer’s policy is fixed. So, run the Markov chain ahead of

time to generate a table– Pr(dealer gets 17|first card=c)– Pr(dealer gets 18|first card=c)– Pr(dealer gets 19|first card=c)– Pr(dealer gets 20|first card=c)– Pr(dealer gets 21|first card=c)– Pr(dealer busts|first card=c)

• Remove dMinSum, dNumAces, turn.• Add dFirstCard• Remove Default action• Change the equations for double/stand based on computed

probabilities© Mausam

16

Comments

• numAces – not required. Convert to hasAce– Only first ace is relevant. After that each ace = 1

• Handle infinite splits by allowing say, 10 splits– Should work fine in practice

• Modification: have the bet amount in state space– Needn’t multiply by 2 in equations for doubling– Need to use this info when computing terminal rewards

© Mausam

17

Difference in solutions?

• In real BJ: Dealer and player BJ is a push– In our BJ: player wins

• ???• In real BJ: there is a 4/6/8 card deck– In our BJ: infinite cards

• You can easily modify code to handle 1st case– 2nd case: Monte carlo sampling!

© Mausam

18

Should you play our BlackJack?

• Expected reward before game starts– SUM[Pr(game)*expected reward of the game]

• Deal three cards… and use V(s) to compute exp reward

• My expected reward: $0.058– Yes! We should play the game

© Mausam

19

Discussion

Mausam

© Mausam

20

Followup Courses• 546 Machine Learning/Data Mining

• 515 Probabilistic Graphical Models

• 517 Natural Language Processing

• ??? AI/Statistics for Big Data

• 571 Robotics

• 579 Intelligent control

• 574 Special Topics in AI

• 528 Computational Neuroscience

• 510 Human computer Interaction© Mausam

21

Final Exam• 50% objective– True/false– Multiple choice– Fill in the blanks– Match the following…

• 25% short answer questions– 1 word – 5 line responses…– opinions/strengths/weaknesses/etc…

• 25% Problem solving– Problem modeling; short calculations…

© Mausam

22

Why do an exam?

• Goal: revise and analyze the whole course

• Expected preparation time– ~1 day– Go over lecture slides– Read parts of the notes that are unclear…

© Mausam

23

What worked• Programming Assignments

– You hated them– You loved them– You learned through them

• Written Assignments(?)– Forced us to be regular

• Classes– I absolutely enjoyed the interaction!

• Emails/bulletin boards

• Videos(?)

© Mausam

24

What didn’t work

• Programming environment– People didn’t know unix– People didn’t know Java!

• Packed– Once someone got behind, it got tough to recover

• Workload

© Mausam

25

Thanks

• To all of you– who contributed to helping each other through

newsgroup and other questions

– who asked terrific questions in class

– who gave regular feedback on class, assignments, grading, organization

© Mausam