CSE391 -20051

Search

When you can’t use A*

• Hill-climbing

• Simulated Annealing

• Other strategies

• 2 person- games

CSE391 -20052

Search

Hill-climbing

• Just keep current state

• Generate successors

• If best successor is better than current state, move to it.

• Otherwise, you’re stuck – local maxima.

CSE391 -20053

Search

N-queens

• NxN board

• N queens

• Place the queens on the board so that all queens are safe. No queen is on the same row, column or diagonal as another queen.

CSE391 -20054

Search

8 queens

• Initial state: no queens

• Successor function: add a queen to any empty square

• Goal: are all queens safe?

• 64 x 63 x 62 ….= 3 x 1014 possible sequences

CSE391 -20055

Search

8-queens – almost a solution, 4 steps

CSE391 -20056

Search

8 queens

• Initial state: no queens

• Successor function: add a queen to a safe empty square

• Goal: are all queens safe?

• 2057 possible sequences

CSE391 -20057

Search

Applying hill climbing to 8 queens

• Initial state: 8 queens placed randomly

• Successor function: move queen in its column (8 x 7 = 56 successors) 88

• Goal state: all queens are safe

• Works 14% of the time, gets stuck 86%

CSE391 -20058

Search

8-queens – a random start state

CSE391 -20059

Search

8-queens – h = # of pairs of queens attacking each other18 12 14 13 13 12 14 14

14 16 13 15 12 14 12 16

14 12 18 13 15 12 14 14

15 14 14 13 16 13 16

14 17 15 14 16 16

17 16 18 15 15

18 14 15 15 14 16

14 14 13 17 12 14 12 18

CSE391 -200510

Search

8-queens – almost a solution, 4 steps

Almost a solution - 5 moves

CSE391 -200511

Search

Applying hill climbing to 8 queens

• Initial state: 8 queens placed randomly

• Successor function: move queen in its column (8 x 7 = 56 successors) 88

or allow up to 100 sideways moves…

• Goal state: all queens are safe

• Works 94% of the time, gets stuck 6%

CSE391 -200512

Search

Hill-climbing

• Local maxima: a peak which is higher than each of its neighbors but lower than the global maximum.

• Ridge: a sequence of local maxima

• Plateau: function value is flat– Flat local maxima, no uphill exit– Shoulder, can find uphill path

CSE391 -200513

Search

Hill-climbing

• Just keep current state

• Generate successors

• If best successor is better than current state, move to it.

• Otherwise, you’re stuck – local maxima.

• Random restart.

CSE391 -200514

Search

Simulated annealing

• Modeled after “annealing” in metallurgy– Heated metal cools slowly so that crystals can form

• Closer to a purely random walk• Starts by picking random moves (high temp)• As the “temperature” cools, shifts to preferring

“best” moves.• Applications:

– VLSI layouts, factory scheduling, airline scheduling

CSE391 -200515

Search

Other strategies

• Local beam search– Same as hill-climbing, but keeps k nodes in

memory instead of just 1.

• Genetic algorithms– Generate a successor by combining two

parent states, instead of just modifying one.

– Apply random mutations– Evaluate with “fitness function”

CSE391 -200516

Search

2 person games – more complex

• Opponent introduces non-determinism– Minimax, alpha-beta

• Space and time limitations can

introduce inaccessibility

CSE391 -200517

Search

Problem Formulation

• Initial state

• Operators

• Terminal test (goal state)

• Utility function (payoff)– Minimax: back up from terminal state,

high values for max, low values for min

CSE391 -200518

Search

Tic-Tac-Toe

• Initial state– Representation? matrix

• Successor functions– Placing x’s and o’s

• Goal states– Explicit

• Utility function - Minimax

x o x

o x

xo x

o x

x x x o

o

CSE391 -200519

Search

Straightforward Mimimax

• Utility function (payoff)– Minimax: back up from terminal state,

high values for max, low values for min

Search20

CSE391 -2005

x

x

x

o x

x o x

x o

x x o

x o x

x x o

x o

x

x x o x x

o

x o

x

o x x

o x x

o x x

o

x

x o

x o

x x o

x x o

o

x x x o

o

x x o

o

x x o

x o

x x o x

oTic-Tac-Toe

xx o

x o

x

x o x

x x o

o x

0

0 0 0

10

1

01

0

1

1

CSE391 -200521

Search

A More “Intelligent” Minimax

• Backing up wins and losses – Requires entire search tree– Often infeasible

• Can use heuristic estimates, e(p), instead– pick “best next move” based on limited search– After opponent’s move, extend search further

and estimate again

CSE391 -200522

Search

Tic-Tac-Toe Evaluation Function

• If p is a win for MAX then e(p) =• If p is a win for MIN then e(p) = -• Else h(n) =

(X’s # of complete open rows,columns, diagonals

- O’s # of complete open rows, columns,diagonals)

e(p) = 6 - 4 = 2

o x

Search23

CSE391 -2005

Heuristic Minimax for Tic-Tac-Toe

x

o x

o x

x o

ox

x o

x

x o

x

o

x

o

xo

x o

x

o x

o x

6-5=1 5-5=0 6-5=1 5-5=0

4-5=-1

5-6=-1 5-5=0 5-6=-1 6-6=0 4-6=-2

5-4=1 6-4=2

-1

-2

1

1

Search24

CSE391 -2005

o x

o x x

o x

x

o x

x

x o x

x o

x o

x o x

o

x oo x

x o x o

x o o

xx o x

o

ox x o

o ox x

o o x x

oo x x

o xo x

o ox x

ox x o

ox xo

ox x o

o o x

x

o o x

x

o x

x o

o x

x o

o o x

x

o x o

x

4-2=2 3-2=1 5-2=3 3-2=1 4-2=2 3-2=1

4-3=1 3-3=0 5-3=2 3-3= 0 4-3=1 4-3=1

4-2=2 4-2=2 5-2=3 3-2=1 4-2=2 4-2=2

4-3=1 4-3=1 3-3= 0

1

0

1

0

1

Minimax - 2

Search25

CSE391 -2005

Minimax-3

o o xx

2-1=1 3-1=2 2-1=1 3-1=2

- = - 2-2=0 2-2=0 3-2=1

-=- 2-1=1 2-1=1 2-1=1

1

-

-

1

x o o xx

o o xx x

x o o o xx

x o o xX o

x o o xx o

x o o x ox

o o x ox x

o o xx x o

o o o xx x

o o o xx x

o o o x xx

o o x xx o

o o x xx o

o o x xx

o o o x xx

- -

CSE391 -200526

Search

Alpha/Beta

• The value of a MAX node = current largest final backed-up value of its successors

• The value of a MIN node = current smallest final backed-up value of its successors

CSE391 -200527

Search

Alpha/Beta Pruning

• Stop search below any MIN node where

• Stop search below any MAX node where

ancestors MAX its ofany of value value

ancestors MIN its ofany of value value

Search28

CSE391 -2005

Alpha/Beta for Tic-Tac-Toe

x

o x

o x

x o

ox

x o

x

x o

x

o

x

o

xo

x o

x

o x

o x

6-5=1 5-5=0 6-5=1 5-5=0

4-5=-1

5-6=-1 5-5=0 5-6=-1 6-6=0 4-6=-2

5-4=1 6-4=2

MIN=-1, BETA=-1

MIN=-2, BETA=-1

MIN=1, BETA=1

MAX=1, ALPHA= -1

X XX X

ALPHA= 1

Search29

CSE391 -2005

Alpha/Beta Pruning

o o xx

2-1=1 3-1=2 2-1=1 3-1=2

- = - 2-2=0 2-2=0 3-2=1

x o o xx

o o xx x

x o o o xx

x o o xX o

x o o xx o

x o o x ox

o o x ox x

o o xx x o

o o o xx x

o o o xx x

XX X

o o x xx

o o o x xx

X XX

- = -

MIN= -, BETA= -

MIN= -BETA= -MIN=1, BETA=1

MAX=1 ALPHA=1