CS420 lecture ten BACKTRACK. Solution vectors In optimization problems, or more general in search problems, a set of choices are to be made to arrive

CS420 lecture tenBACKTRACK

Solution vectors• In optimization problems, or more general in search

problems, a set of choices are to be made to arrive at an optimum or solution.

• A solution can be written in the form of a vector X = (x1,x2,..,xn) and the search is directed by criterion P, defining feasible choices of X.

• Often P can also decide whether a partial solution Xi=(x1,..,xi), in, is feasible, ie, there are extensions of Xi to a feasible solution.

• We can either search for one solution, or we can search for all solutions.

Constraints• Explicit constraints define the domain of the

choice variables, eg xi>0, or xi=0 or 1, or li<xi<ui.

• Implicit constraints specify that x1 .. xi satisfy P, ie relate the xi-s to each other.

• The explicit constraints define a possible solution space for the problem

• The implicit constraints narrow the solution space down to an answer space.

Search Spaces

• We concern us here with tree shaped spaces• A prefix x1 .. xi is a path from the root (the

empty vector) with edges labeled x1 to xi

• Children of xi form solution vectors of the form x1 .. xi+1.

n Queens

• Given an nxn chessboard and n queens, find all placements of the queens such that none of them attack each other– none of the queens occupy the same row, column, or

diagonal.

• As each queen occupies her own row, a (partial) choice vector X is a vector of integers from 1 to n, where xj is the column for the queen in row j.

• One solution to the 4-queens problem is (2,4,1,3)

Q

Q

Q

Q

4 queens 2 4 1 3 solution

Solution space for n-Queens

• root has n children, 1 for each column• each of these has n-1 children, excluding its

own column• next has n-2 children etc.• so the tree has n! nodes / states, many of

these are infeasible– diagonal constraint

• n! is a gross overestimate of the number of states we need to check

Subset sum (SSS)

• Given a set of integers m1 to mn and another integer M, find all subsets of mi-s such that their sum is M

• Eg for m ={1,2,3}, M=3 we have two solutions {1,2} and {3}.

Solution Vectors for SSS

• One option: the indices of the chosen elements

• then solution vectors have different length

Solution Vectors for SSS

• One option: the indices of the chosen elements

• then solution vectors have different length

X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

Alternative

• Another, more regular, solution space has nodes with up to n choice values ci = 0 if mi is not in the subset and ci = 1 if it is.

• In this case the leaves represent the subsets.

Alternative

• Another, more regular, solution space has nodes with up to n choice values ci = 0 if mi is not in the subset and ci = 1 if it is.

• In this case the leaves represent the subsets.C1= 0 C1= 1

C2= 1 C2= 1C2= 0C2= 0

C3= 0C3= 0 C3= 0 C3= 0 C3= 1C3= 1C3= 1C3= 1

Backtrack

• Backtrack walks depth first through the solution space

• and uses the criterion P (sometimes called bounding function B) to prune the solution space to create the answer space.

Iterative Backtrack

• We are searching for all possible answers.• X will hold the (partial) solution vector (x1,x2,..,xi). • B is the bounding function.• Next(X) is the set of valid next states xi+1 from state

xi , and therefore Bi+1(X[1..(i+1)) is true for all states in Next(X) and false for all other states.

• We also need a function that decides whether a state is an answer state.

IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and Bk(X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set} }


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=1x=[]cap = 3


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=2x=[1] feasiblecap=2


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=3x=[1,2] answer pathcap=0 print X=[1,2]


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=3x=[1,2,3] infeasiblebacktrack


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=2X=[1,3] infeasiblebacktrack


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=1X=[2] feasible


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=2X=[2,3] infeasiblebacktrack


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=1X=[3] feasible


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

k=2X=[3]cap=0 path to answer nodeprint X=[3]


X1=1X1=3

X2=2

X1=2

X2=3 X2=3

X3=3

paths to answer nodeX=[1,2]X=[3]

Recursive back track

• global variable X[1:n], initial call RBackTrack(1)

RBackTrack(k) { for each X[k] with Bk(X[1..k])==true

if (X[1..k] is a path to an answer state) print(X[1..k] RBackTrack(k+1) }

n-queens• X[1:k] is a partial solution with X[i] the column

number (1..n) of a queen in row i 1:k. • When we place queen k+1 we need to check

that she does not occupy any of the already occupied columns (that's easy) and any of the already occupied diagonals:– two queens in (i,j) (row i, column j) and (p,q)

occupy the same diagonal if (i-j) == (p-q) or (i+j)==(p+q), which is equivalent with |i-p|==|j-q|.

iterative n-queens with global partial solution X

Place(k) for (i = 1 to k-1) if (X[i]==X[k] or // same columns | X[i]-X[k] | == |i-k| ) // same diagonal return(false); // end for return(true)

nQueens(n) { X[1] = 0 //place first queen "in front of" row 1 k=1 while (k>0) { X[k]++ // move queen k to next column while (X[k]<=n and not(Place(k)) { X[k]++ } if (X[k] <= n) // found valid spot for queen k if (k==n) print(X) else {k++; X[k]=0} // start next row else k-- // backtrack}

Sum of Subsets

• Two ways to represent the search space for subset sum.

• Here we consider the regular case with choices xi=0 (do not take object i) and xi=1 (take object i) at level i.

• A choice for the bounding function Bk =

€

wixii=1

k

∑ + wii=k+1

n

∑ ≥ M

Simplifying / Strengthening assumption for SSS

• From hereon we assume that the objects are sorted in non-decreasing order, because this allows us to strengthen our observations.

Weight gathered so far

• In some state X[1..k] at level k the total weight gathered by objects 1 to k is either M, and we have found a solution and thus we don't need to go deeper in that state...

• or the weight gathered is less than M, and then X[1..k] cannot lead to an answer node if

€

wixii=1

k

∑ + wk+1 > M

Weight gathered so far

• In some state X[1..k] at level k the total weight gathered by objects 1 to k is either M, and we have found a solution and thus we don't need to go deeper in that state...

• or the weight gathered is less than M, and then X[1..k] cannot lead to an answer node if

• WHY?

€

wixii=1

k

∑ + wk+1 > M

A better bound

• Original bound plus previous observation leads to a better bound:

not too little and not too much

€

Bk = wix ii=1

k

∑ + wii=k+1

n

∑ ≥M and wii=1

k

∑ x i +wk+1 ≤ M

Recursive SSS

• s is the weight gathered by the objects chosen so far, and thus needs to be initialized at 0

• r is the total weight of the remaining objects, and thus needs to be initialized at the sum SumW of all the weights.

• k is the current 1-based level. If we take object k we can simplify the bound test because the sum of the weights of the chosen objects (now including k) plus remaining objects has not changed and is still >= M

• The initial call is SumOfSub(0,1,SumW).

No degenerate problems

We assume that W[1]<=M and SumW>=M, otherwise we have a degenerate problem.

SumOfSub(s,k,r) { // two possible next states X[k] = 1 //1: take object k, we can: Bk-1 = true if (s+W[k]==M) print X[1..k] // subset found else (if s+W[k]+W[k+1]<=M) // Bk = true SumOfSub(s+W[k],k+1,r-W[k]) // do not take object k, // now we have to check whether the rest // of the objects still can reach M and // the lightest remaining object is not too heavy if (s + r - W[k] >= M // still can reach M and s+W[k+1] <=M // rest objects not too heavy ) SumOfSub(s,k+1,r-W[k])}

SumOfSub(s,k,r) { // two possible next states X[k] = 1 //1: take object k, we can: Bk-1 = true if (s+W[k]==M) print X[1..k] // subset found else (if s+W[k]+W[k+1]<=M) // Bk = true SumOfSub(s+W[k],k+1,r-W[k]) // do not take object k, // now we have to check whether the rest // of the objects still can reach M and // the lightest remaining object is not too heavy if (s + r - W[k] >= M // still can reach M and s+W[k+1] <=M // rest objects not too heavy ) SumOfSub(s,k+1,r-W[k])}

Notice that the algorithm does not check k<=n.

This is implicit in the fact that when SumOfSub is called s<M and s+r>M, hence r>0 and thus k<=n.

Documents

CS420 lecture ten BACKTRACK. Solution vectors In optimization problems, or more general in search problems, a set of choices are to be made to arrive