CS 416Artificial Intelligence

Lecture 7Lecture 7

Informed SearchesInformed Searches

Lecture 7Lecture 7

Informed SearchesInformed Searches

Chess Match

Kasparov 1, Deep Junior 1, Draws 3Kasparov 1, Deep Junior 1, Draws 3Kasparov 1, Deep Junior 1, Draws 3Kasparov 1, Deep Junior 1, Draws 3

Chess Article• You can buy Deep Blue for $50 (maybe not tweaked to beat Kasparov)You can buy Deep Blue for $50 (maybe not tweaked to beat Kasparov)

• Style and strategy vs. knowledge (depth vs. breadth)Style and strategy vs. knowledge (depth vs. breadth)

– 1,000 top-quality games played each week and broadcast on internet1,000 top-quality games played each week and broadcast on internet

• ““PeoplPeople don’t experiment anymore”e don’t experiment anymore”

– KaKasparov pays a team of grandmasters to scour the web daily looking for sparov pays a team of grandmasters to scour the web daily looking for new opening strategiesnew opening strategies

• ““Computers are opening the game up”Computers are opening the game up”

– people are more likely to follow a range of published strategiespeople are more likely to follow a range of published strategies

http://www.nytimes.com/2003/02/06/nyregion/06CHES.htmlhttp://www.nytimes.com/2003/02/06/nyregion/06CHES.html

• You can buy Deep Blue for $50 (maybe not tweaked to beat Kasparov)You can buy Deep Blue for $50 (maybe not tweaked to beat Kasparov)

• Style and strategy vs. knowledge (depth vs. breadth)Style and strategy vs. knowledge (depth vs. breadth)

– 1,000 top-quality games played each week and broadcast on internet1,000 top-quality games played each week and broadcast on internet

• ““PeoplPeople don’t experiment anymore”e don’t experiment anymore”

– KaKasparov pays a team of grandmasters to scour the web daily looking for sparov pays a team of grandmasters to scour the web daily looking for new opening strategiesnew opening strategies

• ““Computers are opening the game up”Computers are opening the game up”

– people are more likely to follow a range of published strategiespeople are more likely to follow a range of published strategies

http://www.nytimes.com/2003/02/06/nyregion/06CHES.htmlhttp://www.nytimes.com/2003/02/06/nyregion/06CHES.html

Chess Article

Have we been down this road before?Have we been down this road before?

• The Road to Wigan Wier, 1900s (George Orwell)The Road to Wigan Wier, 1900s (George Orwell)

– ““machines are moving in and polluting the spiritual machines are moving in and polluting the spiritual landscape, not on purpose, but because they can't help it“landscape, not on purpose, but because they can't help it“

• David Gelernter, YaleDavid Gelernter, Yale

– ““This is the machine age, and no one uses a pump when This is the machine age, and no one uses a pump when you can turn on the tap. But don't think that won't cost us." you can turn on the tap. But don't think that won't cost us."

Have we been down this road before?Have we been down this road before?

• The Road to Wigan Wier, 1900s (George Orwell)The Road to Wigan Wier, 1900s (George Orwell)

– ““machines are moving in and polluting the spiritual machines are moving in and polluting the spiritual landscape, not on purpose, but because they can't help it“landscape, not on purpose, but because they can't help it“

• David Gelernter, YaleDavid Gelernter, Yale

– ““This is the machine age, and no one uses a pump when This is the machine age, and no one uses a pump when you can turn on the tap. But don't think that won't cost us." you can turn on the tap. But don't think that won't cost us."

New Horizons?

• Hans Berliner, CMUHans Berliner, CMU

– ““You don't have to be really good anymore to get good results. Chess You don't have to be really good anymore to get good results. Chess is winding down." is winding down."

• Bobby FisherBobby Fisher

– randomize pieces behind row of pawnsrandomize pieces behind row of pawns

• KasparovKasparov

– computer/human vs. computer/humancomputer/human vs. computer/human

• Play the Asian game, GoPlay the Asian game, Go

– no human player would dare depend on computer for adviceno human player would dare depend on computer for advice

• Hans Berliner, CMUHans Berliner, CMU

– ““You don't have to be really good anymore to get good results. Chess You don't have to be really good anymore to get good results. Chess is winding down." is winding down."

• Bobby FisherBobby Fisher

– randomize pieces behind row of pawnsrandomize pieces behind row of pawns

• KasparovKasparov

– computer/human vs. computer/humancomputer/human vs. computer/human

• Play the Asian game, GoPlay the Asian game, Go

– no human player would dare depend on computer for adviceno human player would dare depend on computer for advice

Another article

SongProSongProSongProSongPro

The inventor, Ronald Jones (left), with venture capitalist, Mark Bush

http://www.nytimes.com/2003/02/06/technology/circuits/06song.html

Tough Going

• Grew up the son of a maid and one of six black students in a Grew up the son of a maid and one of six black students in a high school of 1400high school of 1400

– love of math and engineeringlove of math and engineering

– dropped out of collegedropped out of college

– learned on the joblearned on the job

• Encountered resistance when finding funding oppsEncountered resistance when finding funding opps

– ““Who does this technology really belong to?”Who does this technology really belong to?”

– ““Is this yours?”Is this yours?”

• Grew up the son of a maid and one of six black students in a Grew up the son of a maid and one of six black students in a high school of 1400high school of 1400

– love of math and engineeringlove of math and engineering

– dropped out of collegedropped out of college

– learned on the joblearned on the job

• Encountered resistance when finding funding oppsEncountered resistance when finding funding opps

– ““Who does this technology really belong to?”Who does this technology really belong to?”

– ““Is this yours?”Is this yours?”

Tough Going

• Lucky to do contract for Rainbow Coalition and met Jesse Jackson who Lucky to do contract for Rainbow Coalition and met Jesse Jackson who could help outcould help out

• 3 years ago, Ron had $800 left and an idea3 years ago, Ron had $800 left and an idea

– one lucky break got him a contact and $11,000 for debtsone lucky break got him a contact and $11,000 for debts

• 2 years ago, Ron had lived with friends and family and lived off of $5 a 2 years ago, Ron had lived with friends and family and lived off of $5 a day (he knew how to get free food at bars with happy hours)day (he knew how to get free food at bars with happy hours)

• Last year, the device was releasedLast year, the device was released

– He still lives cheaply (Marc Hannah, a founder of Silicon Graphics He still lives cheaply (Marc Hannah, a founder of Silicon Graphics and one of the richest black scientists in Silicon Valley)and one of the richest black scientists in Silicon Valley)

• Lucky to do contract for Rainbow Coalition and met Jesse Jackson who Lucky to do contract for Rainbow Coalition and met Jesse Jackson who could help outcould help out

• 3 years ago, Ron had $800 left and an idea3 years ago, Ron had $800 left and an idea

– one lucky break got him a contact and $11,000 for debtsone lucky break got him a contact and $11,000 for debts

• 2 years ago, Ron had lived with friends and family and lived off of $5 a 2 years ago, Ron had lived with friends and family and lived off of $5 a day (he knew how to get free food at bars with happy hours)day (he knew how to get free food at bars with happy hours)

• Last year, the device was releasedLast year, the device was released

– He still lives cheaply (Marc Hannah, a founder of Silicon Graphics He still lives cheaply (Marc Hannah, a founder of Silicon Graphics and one of the richest black scientists in Silicon Valley)and one of the richest black scientists in Silicon Valley)

Subproblems

• Is 4-piece subproblem an admissible heuristic?Is 4-piece subproblem an admissible heuristic?

– it can never overestimate the true costit can never overestimate the true cost

• Is it consistent?Is it consistent?

– h(n) <= c(n, a, n’) + h(n’)h(n) <= c(n, a, n’) + h(n’)

• Is 4-piece subproblem an admissible heuristic?Is 4-piece subproblem an admissible heuristic?

– it can never overestimate the true costit can never overestimate the true cost

• Is it consistent?Is it consistent?

– h(n) <= c(n, a, n’) + h(n’)h(n) <= c(n, a, n’) + h(n’)

Genetic Algorithms (GAs)

Another randomized search algorithmAnother randomized search algorithm

Start with k initial guessesStart with k initial guesses• they form a they form a populationpopulation

• each each individualindividual from the population is a fixed-length string from the population is a fixed-length string (gene)(gene)

• each individual’s each individual’s fitness fitness is evaluatedis evaluated

• successors are generated from individuals according to successors are generated from individuals according to fitness functionfitness function results results

Another randomized search algorithmAnother randomized search algorithm

Start with k initial guessesStart with k initial guesses• they form a they form a populationpopulation

• each each individualindividual from the population is a fixed-length string from the population is a fixed-length string (gene)(gene)

• each individual’s each individual’s fitness fitness is evaluatedis evaluated

• successors are generated from individuals according to successors are generated from individuals according to fitness functionfitness function results results

What’s good about evolution?

Think about mother nature…Think about mother nature…Think about mother nature…Think about mother nature…

What’s bad about evolution?(think West Virginia)

Genetic Algorithms

• ReproductionReproduction

– ReuseReuse

– CrossoverCrossover

• MutationMutation

• ReproductionReproduction

– ReuseReuse

– CrossoverCrossover

• MutationMutation

Crossover

• Early states are diverseEarly states are diverse

– Crossover explores state broadlyCrossover explores state broadly

• Later stages are more similarLater stages are more similar

– Crossover fine tunes in small regionCrossover fine tunes in small region

• Early states are diverseEarly states are diverse

– Crossover explores state broadlyCrossover explores state broadly

• Later stages are more similarLater stages are more similar

– Crossover fine tunes in small regionCrossover fine tunes in small region} Like simulated

annealing

Mutation

Could screw up a good solutionCould screw up a good solution

• Like metropolis step in simulated annealingLike metropolis step in simulated annealing

Could explore untapped part of search spaceCould explore untapped part of search space

Could screw up a good solutionCould screw up a good solution

• Like metropolis step in simulated annealingLike metropolis step in simulated annealing

Could explore untapped part of search spaceCould explore untapped part of search space

GA Analysis

CombinesCombines

• uphill tendencyuphill tendency

• random explorationrandom exploration

• exchange information between multiple threadsexchange information between multiple threads

– like stochastic beam searchlike stochastic beam search

Crossover is not needed – theoreticallyCrossover is not needed – theoretically

• if starting states are sufficiently randomif starting states are sufficiently random

CombinesCombines

• uphill tendencyuphill tendency

• random explorationrandom exploration

• exchange information between multiple threadsexchange information between multiple threads

– like stochastic beam searchlike stochastic beam search

Crossover is not needed – theoreticallyCrossover is not needed – theoretically

• if starting states are sufficiently randomif starting states are sufficiently random

GA Analysis

It’s all in the representationIt’s all in the representation

• GA works best if representation stores related pieces of the GA works best if representation stores related pieces of the puzzle in neighboring cells of stringpuzzle in neighboring cells of string

• Not all problems are amenable to crossoverNot all problems are amenable to crossover

– TSPTSP

It’s all in the representationIt’s all in the representation

• GA works best if representation stores related pieces of the GA works best if representation stores related pieces of the puzzle in neighboring cells of stringpuzzle in neighboring cells of string

• Not all problems are amenable to crossoverNot all problems are amenable to crossover

– TSPTSP

Continuous Spaces

What does continuous mean to you?What does continuous mean to you?What does continuous mean to you?What does continuous mean to you?

A function is continuous if its graph can be drawn without lifting A function is continuous if its graph can be drawn without lifting the pencil from the paper Descartethe pencil from the paper Descarte

A function is continuous if its graph can be drawn without lifting A function is continuous if its graph can be drawn without lifting the pencil from the paper Descartethe pencil from the paper Descarte

In terms of searching?

Continuous search spaces have neighbors for all Continuous search spaces have neighbors for all statesstates

That means they have derivativesThat means they have derivatives

Can the derivative help out here?Can the derivative help out here?

Continuous search spaces have neighbors for all Continuous search spaces have neighbors for all statesstates

That means they have derivativesThat means they have derivatives

Can the derivative help out here?Can the derivative help out here?

Derivative directs future steps

One dimensional functionOne dimensional function

• Left or right?Left or right?

Two dimensional functionTwo dimensional function

• Direction in 3-spaceDirection in 3-space

N-dimensional functionN-dimensional function

• GradientGradient

One dimensional functionOne dimensional function

• Left or right?Left or right?

Two dimensional functionTwo dimensional function

• Direction in 3-spaceDirection in 3-space

N-dimensional functionN-dimensional function

• GradientGradient ),...,,(21 nx

f

x

f

x

ff

An Example

Place three airportsPlace three airports

• minimize sum of minimize sum of squared distances from squared distances from each city to closest each city to closest airport, f(airport, f(xx))

• find (xfind (x11, y, y11, x, x22, y, y22, x, x33, y, y33))

Place three airportsPlace three airports

• minimize sum of minimize sum of squared distances from squared distances from each city to closest each city to closest airport, f(airport, f(xx))

• find (xfind (x11, y, y11, x, x22, y, y22, x, x33, y, y33))

Airport Example

• Simulated AnnealingSimulated Annealing

– Trial and error experimentation with six valuesTrial and error experimentation with six values

• Genetic AlgorithmsGenetic Algorithms

– Create a gene with six values and crossover/mutateCreate a gene with six values and crossover/mutate

• Discretize some slight change in position, Discretize some slight change in position,

– each of six parameters has three values (+,-,same)each of six parameters has three values (+,-,same)

– branching factor of 18branching factor of 18

– A* searchA* search

• Simulated AnnealingSimulated Annealing

– Trial and error experimentation with six valuesTrial and error experimentation with six values

• Genetic AlgorithmsGenetic Algorithms

– Create a gene with six values and crossover/mutateCreate a gene with six values and crossover/mutate

• Discretize some slight change in position, Discretize some slight change in position,

– each of six parameters has three values (+,-,same)each of six parameters has three values (+,-,same)

– branching factor of 18branching factor of 18

– A* searchA* search

Airport Example

Derivatives!Derivatives!

• For given f (a, b, c, d, e, f), compute gradientFor given f (a, b, c, d, e, f), compute gradient

– change in f in response to small change in a, then b, …change in f in response to small change in a, then b, …

• Update the vector, Update the vector, xx

• Beware of jumping too far if Beware of jumping too far if too large too large

• Beware of local minBeware of local min

– each parameter may settle in its own local mineach parameter may settle in its own local min

Derivatives!Derivatives!

• For given f (a, b, c, d, e, f), compute gradientFor given f (a, b, c, d, e, f), compute gradient

– change in f in response to small change in a, then b, …change in f in response to small change in a, then b, …

• Update the vector, Update the vector, xx

• Beware of jumping too far if Beware of jumping too far if too large too large

• Beware of local minBeware of local min

– each parameter may settle in its own local mineach parameter may settle in its own local min

)( oldoldnew xfxx

Computing the Gradient

• Difficult to solve in Difficult to solve in closedclosed form form

– compute f’(compute f’(xx)=)=y y that works for all that works for all xx

• We can usually compute We can usually compute locallylocally

– compute f’(compute f’(xx)=)=y y that works for only that works for only x x near near zz

• We can also compute We can also compute empiricallyempirically

– pick some small offset to add to each element of x and pick some small offset to add to each element of x and compute difference beween f(compute difference beween f(xx) and f() and f(x+x+))

• Difficult to solve in Difficult to solve in closedclosed form form

– compute f’(compute f’(xx)=)=y y that works for all that works for all xx

• We can usually compute We can usually compute locallylocally

– compute f’(compute f’(xx)=)=y y that works for only that works for only x x near near zz

• We can also compute We can also compute empiricallyempirically

– pick some small offset to add to each element of x and pick some small offset to add to each element of x and compute difference beween f(compute difference beween f(xx) and f() and f(x+x+))

Derivative = 0 at Max/Min

Newton-RaphsonNewton-Raphson• Find the zero of an equation (where it crosses x-axis)Find the zero of an equation (where it crosses x-axis)

• set p = pset p = p00 – f(p – f(p00) / f’(p) / f’(p00))

– if p is close to pif p is close to p00 then return p then return p

• else set pelse set p00 = p and repeat = p and repeat

• Draw pictureDraw picture

Newton-RaphsonNewton-Raphson• Find the zero of an equation (where it crosses x-axis)Find the zero of an equation (where it crosses x-axis)

• set p = pset p = p00 – f(p – f(p00) / f’(p) / f’(p00))

– if p is close to pif p is close to p00 then return p then return p

• else set pelse set p00 = p and repeat = p and repeat

• Draw pictureDraw picture

Newton-Raphson

Why set p = pWhy set p = p00 – f(p – f(p00) / f’(p) / f’(p00))

• y-intercept equation of line: y = mx + by-intercept equation of line: y = mx + b

• let y-intercept, b = f (plet y-intercept, b = f (p00))

• let slope, m = f’(plet slope, m = f’(p00))

• we want to find x-value where y-value = 0we want to find x-value where y-value = 0

– let y = 0let y = 0

• 0 = f’(p0 = f’(p00) * x + f(p) * x + f(p00))

– solve for x: -f(psolve for x: -f(p00) / f’(p) / f’(p00))

Why set p = pWhy set p = p00 – f(p – f(p00) / f’(p) / f’(p00))

• y-intercept equation of line: y = mx + by-intercept equation of line: y = mx + b

• let y-intercept, b = f (plet y-intercept, b = f (p00))

• let slope, m = f’(plet slope, m = f’(p00))

• we want to find x-value where y-value = 0we want to find x-value where y-value = 0

– let y = 0let y = 0

• 0 = f’(p0 = f’(p00) * x + f(p) * x + f(p00))

– solve for x: -f(psolve for x: -f(p00) / f’(p) / f’(p00))

But we’re not finding Zero of f(x)

We’re finding zero of gradient(x)We’re finding zero of gradient(x)

• So, replace f(x) with gradient of xSo, replace f(x) with gradient of x

• replace f’(x) with second derivative of xreplace f’(x) with second derivative of x

– HessianHessian

We’re finding zero of gradient(x)We’re finding zero of gradient(x)

• So, replace f(x) with gradient of xSo, replace f(x) with gradient of x

• replace f’(x) with second derivative of xreplace f’(x) with second derivative of x

– HessianHessian

Hessian

Second derivative of a multivariable functionSecond derivative of a multivariable function

HHff(x) = Hessian = (x) = Hessian =

Second derivative of a multivariable functionSecond derivative of a multivariable function

HHff(x) = Hessian = (x) = Hessian =

23

2

23

2

13

232

2

22

2

12

231

2

21

2

21

2

x

f

xx

f

xx

fxx

f

x

f

xx

fxx

f

xx

f

x

f

Newton Raphson

Final EquationFinal EquationFinal EquationFinal Equation

)()(1 xfxHxx f

Online Searches

Online Searches

States and Actions are unknown apriori (before)States and Actions are unknown apriori (before)

• A real robot finding its way through a mazeA real robot finding its way through a maze

State is difficult to changeState is difficult to change

• A real robot cannot jump across state space at will to explore best A real robot cannot jump across state space at will to explore best potential pathspotential paths

State is difficult/impossible to reverseState is difficult/impossible to reverse

• Can you ride your bike backwards? Do you have space to turn around?Can you ride your bike backwards? Do you have space to turn around?

• When spelunking, do you send the smallest person through the tunnel When spelunking, do you send the smallest person through the tunnel first?first?

States and Actions are unknown apriori (before)States and Actions are unknown apriori (before)

• A real robot finding its way through a mazeA real robot finding its way through a maze

State is difficult to changeState is difficult to change

• A real robot cannot jump across state space at will to explore best A real robot cannot jump across state space at will to explore best potential pathspotential paths

State is difficult/impossible to reverseState is difficult/impossible to reverse

• Can you ride your bike backwards? Do you have space to turn around?Can you ride your bike backwards? Do you have space to turn around?

• When spelunking, do you send the smallest person through the tunnel When spelunking, do you send the smallest person through the tunnel first?first?

Online Searches

• Difficult to skip around when using A*Difficult to skip around when using A*

• Potential of irreversible dead end with depth-first searchPotential of irreversible dead end with depth-first search

• Local search is perfect for online searchesLocal search is perfect for online searches

– stop when you cannot improve any furtherstop when you cannot improve any further

– chances are high of stopping at local solutionchances are high of stopping at local solution

– add memory to permit continued exploration with ability to add memory to permit continued exploration with ability to return to best solution (the Brady Bunch again)return to best solution (the Brady Bunch again)

• Difficult to skip around when using A*Difficult to skip around when using A*

• Potential of irreversible dead end with depth-first searchPotential of irreversible dead end with depth-first search

• Local search is perfect for online searchesLocal search is perfect for online searches

– stop when you cannot improve any furtherstop when you cannot improve any further

– chances are high of stopping at local solutionchances are high of stopping at local solution

– add memory to permit continued exploration with ability to add memory to permit continued exploration with ability to return to best solution (the Brady Bunch again)return to best solution (the Brady Bunch again)

Learning in Online Search

Online agents must resolve ignoranceOnline agents must resolve ignorance

• exploreexplore the world the world

• build a build a mapmap

– mapping of (state, action) to resultsmapping of (state, action) to results

– also called a also called a modelmodel relating (state, action) to results relating (state, action) to results

Online agents must resolve ignoranceOnline agents must resolve ignorance

• exploreexplore the world the world

• build a build a mapmap

– mapping of (state, action) to resultsmapping of (state, action) to results

– also called a also called a modelmodel relating (state, action) to results relating (state, action) to results