Daa Unit 3

8/13/2019 Daa Unit 3

1/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN

Code & Name of the subject: 101401 / Design and Analysis of AlgorithmUnit No.: III Prepared by: Purushothaman.R__________________________________________________________________________________

31

DYNAMIC PROGRAMMING

Dynamic Programming is one of commonly used generalapproach to solving an optimization problem. The basic strategy of dynamic

programming is to solve the problem with small sizes first, and then use the results forsmall sizes to solve the problem with a larger sizes. Step by step, this method can solvethe problem with any large size. Thus,

Unlike the divide-and-conquer approach, the dynamic programming proposes thebottom-up approach. Dynamic programming is an optimization procedure that is

particularly applicable to problems requiring a sequence ofinterrelateddecisions.

GENERAL METHOD

The art of dynamic programming is how to discover the recursive relationship anddefine the sub-problem. There are two key points of dynamic programming:

(1) Optimal substructure:

An optimal solution to a problem (instance) contains optimal solutions to sub-problem. This step defines a set of values which are optimal solutions for smallestproblems, or define a set of values that can serve as boundary values upon whichsolutions for larger problems can be built. For example, if we want to build an optimal

binary search tree withn leaves, we start fromn trees, with each having a single node.Let us call this set S0.

(2) Overlapping sub problems:

A recursive solution contains a small number of distinct sub problems repeated

many times. This step repeatedly compute set Si+1 from the results in Si, Si-1, Si-2, ,S0, until an optimal solution for the original problem is obtained. A problem that can be

solved by dynamic programming must satisfy the principle of optimality. Everysolution in set Si, Si-1, , S0 must be optimal in order to produce solutions in Si+1which must be also optimal of course.

Over view:

http://csetube.co.nr/

8/13/2019 Daa Unit 3

2/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


32

Designed to solve optimization problem. Dynamic programming also divides a problem into several sub problems. However,

some sub problems may share sub problems. Dynamic programming saves time by computing each sub problem only once. To do so, it must be conducted in a bottom-up fashion.

MULTISTAGE GRAPHS

A multistage graph is a graph

o G=(V,E) with Vpartitioned into K >= 2 disjoint subsets such that if(a,b) is in E,then a is in Vi , and b is in Vi+1 for some subsets in the partition;

o and | V1 | = | VK | = 1.

The vertex s in V1 is called the source; the vertex t in VK is called the sink.

G is usually assumed to be a weighted graph.

The cost of a path from node v to node w is sum of the costs of edges in the path.

The "multistage graph problem" is to find the minimum cost path from s to t.

[Cf. the "network flow problem".]

Each set Vi is called a stage in the graph.

Consider the resource allocation problem:

Given n units of resources to be allocated to k projects.

For 1

8/13/2019 Daa Unit 3

3/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


33

{ V(i+1,j) } j=0 .. n

[we could denote the vertices in this set as: vi+1j[or could instead call them vj of set Vi]

The edges are weighted with C(i,j) = -P(i,j) [the negative of the profit] to make it aminimization problem.

Dynamic Programming solution:

Let path(i,j)be some specification of the minimal path from vertexj in set i to vertex t;C(i,j) is the cost of this path; c(j,t) is the weight of the edge fromj to t.

C(i,j) = min { c(j,l) + C(i+1,l) }

l in Vi+1

(j,l) in E

To write a simple algorithm, assign numbers to the vertices so those in stage Vi havelower number those in stage Vi+1.

int[] MStageForward(Graph G)

{

// returns vector of vertices to follow through the graph// let c[i][j] be the cost matrix of G

int n = G.n (number of nodes);

int k = G.k (number of stages);

float[] C = new float[n];

int[] D = new int[n];

int[] P = new int[k];

for (i = 1 to n) C[i] = 0.0;

for j = n-1 to 1 by -1 {

r = vertex such that (j,r) in G.E and c(j,r)+C(r) is minimum

C[j] = c(j,r)+C(r);

D[j] = r;

}

P[1] = 1; P[k] = n;

for j = 2 to k-1 {

P[j] = D[P[j-1]];

}


8/13/2019 Daa Unit 3

4/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


34

return P;

}

ALL PAIRS SHORTEST PATH PROBLEM

A weighted graph is a collection of points(vertices) connected by lines(edges), whereeach edge has a weight(some real number) associated with it. One of the most commonexamples of a graph in the real world is a road map. Each location is a vertex and eachroad connecting locations is an edge. We can think of the distance travelled on a roadfrom one location to another as the weight of that edge.

Given a weighted graph, it is often of interest to know the shortest path from one vertexin the graph to another. The Floyd-Warshall algorithm algorithm determines the shortest

path between all pairs of vertices in a graph.

Although I don't expect you all to understand the full derivation of this algorithm, I willgo through some of the intuition as to how it works and then the algorithm itself.

The the vertices in a graph be numbered from 1 to n. Consider the subset {1,2...,k} ofthese n vertices.

Imagine finding the shortest path from vertex i to vertex j that uses vertices in the set{1,2, ...,k} only. There are two situations:

1) k is an intermediate vertex on the shortest path.2) k is not an intermediate vertex on the shortest path.

In the first situation, we can break down our shortest path into two paths: i to k and thenk to j. Note that all the intermediate vertices from i to k are from the set {1,2,...,k-1} and

that all the intermediate vertices from k to j are from the set {1,2,...,k-1} also.

In the second situation, we simply have that all intermediate vertices are from the set{1,2,...,k-1}.


8/13/2019 Daa Unit 3

5/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


35

Now, define the function D for a weighted graph with the vetrtices {1,2,...n} as follows:

D(i,j,k) = the shortest distance from vertex i to vertex j using the intermediate vertices inthe set {1,2,...,k}

Now, using the ideas from above, we can actually recursively define the function D:

D(i,j,k) = w(i,j), if k=0min( D(i,j,k-1), D(i,k,k-1)+D(k,j,k-1) ) if k > 0

In English, the first line says that if we do not allow intermediate vertices, then theshortest path between two vertices is the weight of the edge that connects them. If nosuch weight exists, we usually define this shortest path to be of length infinity.

The second line pertains to allowing intermediate vertices. It says that the minimumpath from i to j through vertices {1,2,...,k} is either the minimum path from i to jthrough vertices {1,2,...,k-1} OR the sum of the minimum path from vertex i to kthrough {1,2,...,k-1} plus the minimum path from vertex k to j through {1,2,...k-1}.Since this is the case, we compute both and choose the smaller of these.

All of this points to storing a 2-dimensional table of shortest distances and usingdynamic programming for a solution.

Here is the basic idea:

1) Set up a 2D array that stores all the weights between one vertex and another.

Here is an example:

0 3 8 inf -4inf 0 inf 1 7inf 4 0 inf inf 2 inf -5 0 inf


8/13/2019 Daa Unit 3

6/22

8/13/2019 Daa Unit 3

7/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


37

Finally, after adding in the last vertex:0 1 -3 2 -43 0 -4 1 -17 4 0 5 32 -1 -5 0 -28 5 1 6 0

Looking at this example, we can come up with the following algorithm:

Let D1 store the matrix with the initial graph edge information. D2 will stored

calculated information look at D1.

For k=1 to n {For i=1 to n {

For j=1 to nD2[i,j] = min(D1[i,j], D1[i,k]+D1[k,j])

}Copy matrix D2 into D1

}Last D2 matrix will store all the shortest paths.

In order to code this up, we could do so in a static method. We need the adjacencymatrix of the graph passed into the method as a two dimensional double matrix. Thenwe need the auxiliary min and copy methods.

As it turns out, you do NOT need to use 2 separate matrices, even though we tracedthrough the algorithm in that manner. The reason for this is that when we look up valuesin the "current matrix", we know that those values will be at least as good as the valueswe would have looked up in the "previous matrix." Thus, in essence, we will not be

overlooking any possible shortest path.

Here is the code for these methods as well as a main that runs this example given above.

public class floyd {

// Runs Floyd Warshall algorithm. Returns the matrix of// shortest paths.


8/13/2019 Daa Unit 3

8/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


38

public static int[][] shortestpath(int[][] adj) {

int n = adj.length;int[][] m = new int[n][n];

// Initialize m to be graph adjacency matrixcopy(m, adj);

// Add in each vertex as intermediate point.for (int k=0; k

8/13/2019 Daa Unit 3

9/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


39

return a;else

return b;}

public static void main(String[] args) {

// Hard code in example adjacency matrix.int[][] m = new int[5][5];m[0][0] = 0; m[0][1] = 3; m[0][2] = 8; m[0][3] = 10000;

m[0][4] = -4;m[1][0] = 10000; m[1][1] = 0; m[1][2] = 10000; m[1][3] = 1;m[1][4]=7;m[2][0] = 10000; m[2][1] = 4; m[2][2] = 0; m[2][3] = 10000;m[2][4] = 10000;m[3][0] = 2; m[3][1] = 10000; m[3][2] = -5; m[3][3] = 0;m[3][4] = 10000;m[4][0] = 10000; m[4][1] = 10000; m[4][2] = 10000;m[4][3] = 6; m[4][4] =0;

int[][] shortpath;shortpath = shortestpath(m);

for (int i=0; i

8/13/2019 Daa Unit 3

10/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


40

Floyd's All Pairs Shortest Path| 1 2 3 4

----------------1| 0 U 4 12| 7 0 2 3 Distance Matrix3| 5 U 0 14| 8 U 3 0

| 1 2 3 4----------------

1|-1 -1 4 -12| 3 -1 -1 3 Path Matrix3|-1 -1 -1 -14| 3 -1 -1 -1

Shortest Paths==============

1

34

2

2

3

1

1

10

5


8/13/2019 Daa Unit 3

11/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


41

4 from 1 to 3 = 1->3 = 1->4->3

7 from 2 to 1 = 2->1 = 2->3->1

3 from 2 to 4 = 2->4 = 2->3->4

8 from 4 to 1 = 4->1 = 4->3->1

all other shortest paths are edges

OPTIMAL BINARY SEARCH TREE

Binary search trees

Abinary search tree is a binary tree of items that come from an ordered set, such that

1. Each node contains one key.2. The keys in the left sub-tree of a given node are less than or equal to the key in

that node.3. The keys in the right sub-tree of a given node are greater than or equal to the key

in that node.

Atree is a graph with a specialized structure. Thedepth or thelevelof a node in a treeis the number of edges in the unique path from the root to the node. Thedepth of a treeis the maximum depth of all nodes in the tree. A tree is said to be balanced if the depthof the two sub-trees of every node never differ by more than 1.

A binary search tree that is organized in a way such that the average time it takes to

locate (search for) a key is minimized is calledoptimal. Constructing the optimal binarysearch tree is an optimization problem and theprinciple of optimality must apply; moreabout this later.

Searching the Tree


8/13/2019 Daa Unit 3

12/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


42

We will use a proceduresearch to locate a key in a binary search tree. The number ofcomparisons done by the procedure search to locate a key is called the search time. Ourgoal is to determine a tree for which the average search time is minimal.

void search (node-pointer tree, keytype keyin, node-pointer & p){

bool found;p = false;while (! found)

if (p->key == keyin)

found = true;else if ( keyin < p->key);p = p->left

elsep = p->right;

}

Assume a branch statement is used to implement the nested if-else statement and onlyone comparison is performed per each iteration of the while loop. The search time (i.e.,the number of comparisons) for a given key is:

depth(key) + 1

Let Key1, Key2, . . . , Keynbe the n keys in order, and let pi be the probability thatKeyi is the search key. If ci is the number of comparisons needed to find Keyi in a giventree, the average search time for that tree is: c

ip

i for i = 1 to i = n where ci =depth(keyi) + 1

Example:

Consider these trees with n = 3. (Values of keys are not important; the only requirementis that they be ordered). We have p1 = 0.7, p2 = 0.2 and p3 = 0.1


8/13/2019 Daa Unit 3

13/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


43

The average search time for the above trees is:1. 3 (0.7) + 2 (0.2) + 1 (0.1) = 2.6

2. 2 (0.7) + 1 (0.2) + 2 (0.1) = 1.83. 1 (0.7) + 2 (0.2) + 3 (0.1) = 1.4

The last tree is optimal.

Constructing the optimal binary search tree

Let A[i][j] represent the average number of comparisons for searching an optimal treecontaining keys i through j.

A[i][j] = pkck= pk(depthk+1) for k=i to j

This is the value that we seek to minimize in an optimal search tree.

Because it takes one comparison to locate a key in a tree containing one node, A[i][i]=1.

Criterion for an optimal tree (principle of optimality):

Each optimal binary search tree is composed of a root and (at most) two optimalsubtrees, the left and the right. Or, stated in another way: in an optimal tree, all thesubtrees must also be optimal with respect to the keys that they contain.

Let mij = pk for k=i to j and A[i][j] be the average number of comparisonscarried out in an optimal subtree containing the keys keyi, keyi+1, keyi+2, , keyj (asdefined before). One of the keys, say k, must occupy the root of the subtree, as shown

below.


8/13/2019 Daa Unit 3

14/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


44

When we look for a key in the main tree, the probability that it is in the sequencekeyi,

keyi+1, keyi+2, , keyj is mij . In this case, one comparison is made with cost ck and theothers may then be made in L or R. The average number of comparisons carried out istherefore:

A[i][j] = mij + A[i][k-1] + A[k+1][j]

To obtain a dynamic programming scheme, the root, k, must be chosen to minimizeA[i][j]:

A[i][j] = mij + min i

8/13/2019 Daa Unit 3

15/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


45

the column space is indexed from 0 to n. R[i][j] is the index of the key in the root of thetree containing keys i through j.

void optsearchtree (int n, const float p[], float & minavg, index R[][]){

index i, j, k, diagonal;float A[1, , n+1][0 n];for (i = 1; i

8/13/2019 Daa Unit 3

16/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


46

Inputs: n, the number of keys, and array, Key containing the n keys in order, and thearray R produced previously.

Outputs: a pointer to an optimal binary search containing the n keys.

node_pointer tree(index i, j){

index k;node_pointer p;

k = R[i][j];if (k==0)return NULL;

else {p = new nodetype;p->key = Key[k];p->left = tree(i, k-1);p->right = tree(k+1, j);return p;

}}

0/1 KNAPSACK

The 0-1 Knapsack problem is posed as follows: A thief robbing a store finds n items;the i-th item is worth ci dollars and weights wipounds (or kilograms), where ci and wiare integers. He wants to take as valuable a load as possible, but he can carry at most W

pounds in his knapsack for some integer W. Which item he should take?

The problem is re-stated formally: given some set N of n items, where the i-th item isworth ci dollars and weights wi pounds. Find the subset S of n items to maximize

i

i S

c such that i

i S

w W

.


8/13/2019 Daa Unit 3

17/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


47

We have to make decision that pick an item or drop it. Suppose that W is a smallinteger. The 0-1 Knapsack problem could be decomposed into O(NW) number of sub

problems.

({wi, ci}i=1n, W)

= cn + MAX({wi, ci}i=1n-1, W-wn) if pick (wn, cn),

= ({wi, ci}i=1n-1, W) otherwise

Example of 0-1 Knapsack problem: W = 4ci 6 4 4

wi 3 2 2

Compute the table bottom-up

4 0 6 6 83 0 6 6 62 0 4 4 41 0 0 0 00 0 0 0 0

Weight/{N items} 0 1 2 3

Each cell in the above table represent the value V(i, j) that is the maximum total value ifwe can picki objects from the set and put them in the knapsack ofweight j.

V(i, j) = MAX {ci + V(i -1, j wi) if wi j; V(i-1, j)}V(i, j) = 0 if i = 0 or j = 0.

Example of 0-1 Knapsack problem with W = 10ci 3 4 2 10 3wi 2 3 5 1 4

The best value is 20, computed in the following figure:


8/13/2019 Daa Unit 3

18/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


48

0 0000 0

0 10101010 10

0 10

0 54321

0

6

5

4

3

2

1

7

0 13131310 13

0 14141410 14

0 14141410 14

0 17171410 17

0 17171410 17

9

8

10

0 17171410 17

0 17171410 17

0 20171410 20

Total

weight

Bag of

size i

101010 10

Figure 3 0-1 Knapsack computationThe algorithm takes as input the maximum weightW, the number of items n, and thetwo sequencesv = andw = . It stores thec[i,j] valuesin the table, that is, a two dimensional array,c[0 . .n, 0 . .w] whose entries arecomputed in a row-major order. That is, the first row ofc is filled in from left to right,then the second row, and so on. At the end of the computation,c[n,w] contains themaximum value that can be picked into the knapsack.

Dynamic-0-1-knapsack (v, w, n, W)

forw = 0 toWdoc[0,w] = 0

fori = 1 tondoc[i, 0] = 0

forw = 1 toWdo ifwi w

then ifvi +c[i-1,w-wi]


8/13/2019 Daa Unit 3

19/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


49

thenc[i,w] =vi +c[i-1,w-wi]elsec[i,w] =c[i-1,w]

elsec[i,w] =c[i-1,w]

The set of items to take can be deduced from the table, starting atc[n. w] and tracingbackwards where the optimal values came from. Ifc[i, w] =c[i-1,w] itemi is not part ofthe solution, and we are continue tracing withc[i-1,w]. Otherwise item i is part of thesolution, and we continue tracing withc[i-1,w-W].

Analysis

This dynamic-0-1-kanpsack algorithm takes (nw) times, broken up as follows:

(nw) times to fill thec-table, which has (n+1).(w+1) entries, each requiring (1) time tocompute.O(n) time to trace the solution, because the tracing process starts in rown ofthe table and moves up 1 row at each step.

THE TRAVELING SALESPERSON PROBLEM

The Traveling Sales Person problem is an optimization problem: find the shortestroute through a set of cities visiting each city only once. The set of cities is representedas a graph and the roads between them as edges. The objective of the TSP problem is todetermine the shortest path from one vertex in a graph, going through every other vertexin the graph exactly once, and ending up back at the starting vertex

e.g. a directed graph :


8/13/2019 Daa Unit 3

20/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


50

1 2

3

2

44

2

56

7

104

8

39

cost matrix :

1 2 3 4

1 2 10 52 2 9 3 4 3 44 6 8 7

multi-stage graph:


8/13/2019 Daa Unit 3

21/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


51

(1) (1,3)

(1,2)

(1,4)

2

5

10

(1,2,3)

(1,2,4)

(1,3,2)

(1,3,4)

(1,4,2)

(1,4,3)

9

3

4

8

7

(1,2,3,4)

(1,2,4,3)

(1,3,2,4)

(1,3,4,2)

(1,4,2,3)

(1,4,3,2)

4

7

8

9

3

1

4

6

6

2

4

2

Fig. A Multi-Stage Graph Describing All Possible Tours of a Directed graph.

find the shortest path:1, 4, 3, 2, 1 5+7+3+2=17

Suppose that we have 6 vertices in the graph.We can combine {1, 2, 3, 4} and {1, 3, 2, 4} into one node.

combine

(2), (4,5,6)

(b)

(3), (4,5,6)

(4), (5,6)

(1,3,2)

(1,2,3) (1,2,3,4)

(1,3,2,4)

(a)

the last vertex visited is 3the remaining vertices to be visited are 4, 5, 6


8/13/2019 Daa Unit 3

22/22

http://

csetub

e.co

.nr/

GKMCET

LECTURE PLAN


Let g(i, S) be the length of a shortest path starting at vertex i, going through allvertices in S and terminating at vertex 1.

The length of an optimal tour :g(1, V-{1}) = min {c1k+ g(k, V-{1, k})}

2knThe general form:

g(i, S) = min {cij + g(j, S-{j})}jS

time complexity:

n+ ( )( )( )n n kn knk

n

1 22

= O(n22n)

( ),( ) (n-k)

(n-1) ( )n kn

2

Documents

Daa Unit 3