State Space State Space

Heuristic SearchHeuristic Search

Three AlgorithmsThree Algorithms BacktrackBacktrack Depth FirstDepth First Breadth FirstBreadth First

All work if we have well-defined:All work if we have well-defined: Goal stateGoal state Start stateStart state State transition rulesState transition rules

But could take a long timeBut could take a long time

HeuristicHeuristic

An informed guess that guides An informed guess that guides search through a state spacesearch through a state space

Can result in a suboptimal solutionCan result in a suboptimal solution

Blocks World: A Stack of BlocksBlocks World: A Stack of Blocks

StartStart GoalGoalAA DDBB CCCC BBDD AATwo rules:Two rules:clear(X) clear(X) on(X, table) on(X, table)clear(X) ^ clear(Y) clear(X) ^ clear(Y) on(X,Y) on(X,Y)

All Three Algorithms will find a All Three Algorithms will find a solutionsolution

Partial Look At Search SpacePartial Look At Search Space

AABBCCDD

BBCCDADA

CC BCBC B A DB A D ADAD

Etc.

Heuristic 1Heuristic 1

1.1. For each block that is resting where For each block that is resting where it should, subtract 1it should, subtract 1

2.2. For each block that is not resting For each block that is not resting where it should, add 1where it should, add 1

Hill ClimbingHill Climbing

1.1. At every level, generate all childrenAt every level, generate all children2.2. Continue down path with lowest scoreContinue down path with lowest score

Define three functions:Define three functions:f(n) = g(n) + h(n)f(n) = g(n) + h(n)Where:Where:

h(n) is the heuristic estimate for n--guides the h(n) is the heuristic estimate for n--guides the searchsearchg(n) is path length from start to current node—g(n) is path length from start to current node—ensures that we choose node closest to root ensures that we choose node closest to root when more than 1 have same h valuewhen more than 1 have same h value

Problem: heuristic is localProblem: heuristic is local

Given Given CC andand CBCBBB DADAAADD

At level nAt level nThe f(n) of each structure is the sameThe f(n) of each structure is the samef(n) = g(n) = (1+1-1-1) = g(n)f(n) = g(n) = (1+1-1-1) = g(n)

But which is actually betterBut which is actually better

The vertical structure must be The vertical structure must be undone entirelyundone entirely

CC B BBB A A A A AA D C D C DCB DCB ABCD B ABCD B CCDD ACD ACD BB

ADADDDCCBBAA

For a total of 6 movesFor a total of 6 moves

ButBut

CBCB CC DD

DADA BB CC

ADAD BB

AA

2 moves2 moves

TaskTask

Design a global heuristic that takes the Design a global heuristic that takes the entire structure into accountentire structure into account

1.1. Subtract 1 for each block that has Subtract 1 for each block that has correct support structurecorrect support structure

2.2. Add 1 for each block in an incorrect Add 1 for each block in an incorrect support structuresupport structure

f(n) = g(n) + (3+2+1+0) =g(n) + 6f(n) = g(n) + (3+2+1+0) =g(n) + 6C C B B AA goal Dgoal DDD CC

BBAA

CB f(n) = g(n) + (1 + 0 -1+ 0) = g(n)CB f(n) = g(n) + (1 + 0 -1+ 0) = g(n)DADA

So the heuristic correctly chose the second So the heuristic correctly chose the second structurestructure

Leads to a New Algorithm: Best Leads to a New Algorithm: Best FirstFirst

The Road Not Taken

Best firstBest first Keeps nodes on open in a priority queue Keeps nodes on open in a priority queue

ordered by h(n) so that if it goes down a ordered by h(n) so that if it goes down a bad path that at first looks good, it can bad path that at first looks good, it can retry a new pathretry a new path

Contains algorithm for generating h(n)Contains algorithm for generating h(n) Nodes could contain backward pointers so Nodes could contain backward pointers so

that path back to root can be recoveredthat path back to root can be recovered

list best_first(Start)list best_first(Start){{ open = [start], closed = [];open = [start], closed = []; while (!open.isEmpty())while (!open.isEmpty()) {{ cs = open.serve();cs = open.serve(); if (cs == goal)if (cs == goal) return path;return path; generate children of cs;generate children of cs; for each childfor each child {{ case:case: {{ child is on open;child is on open; //node has been reached by a shorter path//node has been reached by a shorter path if (g(child) < g(child) on open)if (g(child) < g(child) on open) g(child on open) = g(child);g(child on open) = g(child); break;break;

child is on closed;child is on closed; if (g(child < g(child on closed))if (g(child < g(child on closed)) { { //node has been reached by a shorter path and is more attractive//node has been reached by a shorter path and is more attractive remove state from closed;remove state from closed; open.enqueue(child);open.enqueue(child); }} break;break;

default:default: {{ f(child) = g(child) + h(child);//child has been examined yetf(child) = g(child) + h(child);//child has been examined yet open.enqueue(child);open.enqueue(child); }} }} }} closed.enqueue(cs);//all cs’ children have been examined.closed.enqueue(cs);//all cs’ children have been examined. open.reorder);//reorder queue because case statement may have affected orderingopen.reorder);//reorder queue because case statement may have affected ordering }} return([]); //failurereturn([]); //failure}}

State Space of a Hypothetical State Space of a Hypothetical SearchSearch

Next SlideNext Slide Goal: PGoal: P States with attached evaluations are those States with attached evaluations are those

generated by best-firstgenerated by best-first States expanded by best-first are States expanded by best-first are

indicated in boldindicated in bold Just before evaluating O, closed contains Just before evaluating O, closed contains

HCBAHCBA Just before evaluating O, open containsJust before evaluating O, open contains

OPGEFDOPGEFD

AdmissibilityAdmissibility

A search algorithm is admissible if it A search algorithm is admissible if it finds the optimal path whenever one finds the optimal path whenever one existsexists

BF is admissibleBF is admissible DF is not admissibleDF is not admissible Suppose on the previous slide Node S Suppose on the previous slide Node S

were replaced by a Pwere replaced by a P The optimal path is ACHPThe optimal path is ACHP But DF discovers ABEKPBut DF discovers ABEKP

Algorithm AAlgorithm A

UsesUses Best FirstBest First f(n) = g(n) + h(n)f(n) = g(n) + h(n)

f*f*

f*(n) = g*(n) + h*(n)f*(n) = g*(n) + h*(n)WhereWhere g*(n) is the cost of the shortest path g*(n) is the cost of the shortest path

from start to nfrom start to n h*(n) is the cost of the shortest path h*(n) is the cost of the shortest path

from n to goalfrom n to goal

So, f*(n) is the actual cost of the So, f*(n) is the actual cost of the optimal pathoptimal path

Consider g*(n)Consider g*(n)

g(n) – actual cost to ng(n) – actual cost to n g*(n) – shortest path from start to ng*(n) – shortest path from start to n

So g(n) >= g*(n)So g(n) >= g*(n)

When g*(n) = g(n), the search has When g*(n) = g(n), the search has discovered the optimal path to ndiscovered the optimal path to n

Consider h*(n)Consider h*(n)

We can’t know it unless exhaustive We can’t know it unless exhaustive search is possible and we’ve already search is possible and we’ve already searched the state spacesearched the state space

But we can know sometimes if a But we can know sometimes if a given h-1(n) is bounded above by given h-1(n) is bounded above by some h-2(n)some h-2(n)

8-puzzle Example8-puzzle Example

283283 123 123164 -> 8 4164 -> 8 47 57 5 765 765

h-1(n) h-1(n) number of tiles not in goal position number of tiles not in goal position= 5 (1,2,6,8, B)= 5 (1,2,6,8, B)

H-2(n) H-2(n) number of moves required to move them number of moves required to move them to goalto goal(T1 = 1, T2 = 1, T6 = 1, T8 = 2, TB = 1)(T1 = 1, T2 = 1, T6 = 1, T8 = 2, TB = 1)= 6= 6

So h-1(n) <= h-2(n)So h-1(n) <= h-2(n)

h-2(n) cannot exceed h*(n) because h-2(n) cannot exceed h*(n) because each tile has to be moved a certain each tile has to be moved a certain distance to reach goal no matter distance to reach goal no matter what. h-2(n) could equal h*(n)what. h-2(n) could equal h*(n)

h-1(n) is certainly <= h-2(n) which h-1(n) is certainly <= h-2(n) which requires moving each incorrect tile at requires moving each incorrect tile at least as far as h-1(n) least as far as h-1(n)

So, h-1(n) <= h-2(n) <=h*(n)So, h-1(n) <= h-2(n) <=h*(n)

Leads to a DefinitionLeads to a Definition

A*A*

If algorithm A uses a heuristic that If algorithm A uses a heuristic that returns a value h(n) <= h*(n) for all returns a value h(n) <= h*(n) for all n, then it is called A*n, then it is called A*

ClaimClaim

All A* algorithms are admissibleAll A* algorithms are admissible

Suppose:Suppose:

1.1. h(n) = 0 and so <= h*(n)h(n) = 0 and so <= h*(n)Search will be controlled by g(n)Search will be controlled by g(n)

If g(n) = 0, search will be randomIf g(n) = 0, search will be random

If g(n) is the actual cost to n, f(n) becomes BF If g(n) is the actual cost to n, f(n) becomes BF because the sole reason for examining a because the sole reason for examining a node is its distance from start.node is its distance from start.

We already know that this terminates in an We already know that this terminates in an optimal solutionoptimal solution

2.2. h(n) = h*(n)h(n) = h*(n)Then the algorithm will go directly to the goal since h*(n) Then the algorithm will go directly to the goal since h*(n)

computes the shortest path to the goalcomputes the shortest path to the goal

Therefore, if our algorithm is between these two extremes, Therefore, if our algorithm is between these two extremes, our search will always result in an optimal solutionour search will always result in an optimal solution

Call h(n) = 0, h’(n)Call h(n) = 0, h’(n)So, for any h such thatSo, for any h such thath’(n) <= h(n) <= h*(n) we will always find an optimal h’(n) <= h(n) <= h*(n) we will always find an optimal

solutionsolutionThe closer our algorithm is to h’(n), the more extraneous The closer our algorithm is to h’(n), the more extraneous

nodes we’ll have to examine along the waynodes we’ll have to examine along the way

InformednessInformedness

For any two A* heuristics, h-a, h-bFor any two A* heuristics, h-a, h-b

If h-a(n) <= h-b(n), h-b(n) is more If h-a(n) <= h-b(n), h-b(n) is more informed.informed.

ComparisonComparison

h-a is BFh-a is BF h-b is the # of tiles out of placeh-b is the # of tiles out of place Since h-a is 0, h-b is better informed Since h-a is 0, h-b is better informed

than h-a than h-a

P. 149P. 149

Comparison of two solutions that Comparison of two solutions that discover the optimal path to the goal discover the optimal path to the goal state:state:

1.1.BF: h(n) = 0BF: h(n) = 0

2.2.h(n) = number of tiles out of placeh(n) = number of tiles out of place

The better informed solution examines The better informed solution examines less extraneous information on its path less extraneous information on its path to the goalto the goal