0 1 Knapsack

5/27/2018 0 1 Knapsack

1/54

1

0-1 Knapsack problem

Dr. Ying [email protected]

RAIK 283

Data Structures & Algorithms

5/27/2018 0 1 Knapsack

2/54

2

Giving credit where credit is due:

Most of slides for this lecture are based on slides

created by Dr. David Luebke, University of Virginia.

Some slides are based on lecture notes created by Dr.Chuck Cusack, Hope College.

I have modified them and added new slides.

RAIK 283

Data Structures & Algorithms

5/27/2018 0 1 Knapsack

3/54

3

Given some items, pack the knapsack to get

the maximum total value. Each item has some

weight and some value. Total weight that we can

carry is no more than some fixed number W.So we must consider weights of items as well as

their values.

Item # Weight Value

1 1 8

2 3 6

3 5 5

Knapsack problem

5/27/2018 0 1 Knapsack

4/54

4

Knapsack problem

There are two versions of the problem:1. 0-1 knapsack problem Items are indivisible; you either take an item or not. Some

special instances can be solved with dynamic programming

2. Fractional knapsack problem Items are divisible: you can take any fraction of an item

5/27/2018 0 1 Knapsack

5/54

5

Given a knapsack with maximum capacity W, and

a set Sconsisting of nitems

Each item ihas some weight wi

and benefit value

bi (all wi and Ware integer values)

Problem: How to pack the knapsack to achieve

maximum total value of packed items?


5/27/2018 0 1 Knapsack

6/546

Problem, in other words, is to find

Ti

i

Ti

i Wwb subject tomax


The problem is called a 0-1 problem,

because each item must be entirely

accepted or rejected.

5/27/2018 0 1 Knapsack

7/547

Lets first solve this problem with a

straightforward algorithm

Since there are nitems, there are 2npossiblecombinations of items.

We go through all combinations and find the one

with maximum value and with total weight less or

equal to W

Running time will be O(2n)

0-1 Knapsack problem:

brute-force approach

5/27/2018 0 1 Knapsack

8/548

We can do better with an algorithm based on

dynamic programming

We need to carefully identify the subproblems

0-1 Knapsack problem:

dynamic programming approach

5/27/2018 0 1 Knapsack

9/549




and benefit value




Defining a Subproblem

5/27/2018 0 1 Knapsack

10/5410

We can do better with an algorithm based on

dynamic programming

We need to carefully identify the subproblems

Lets try this:

If items are labeled 1..n, then a subproblemwould be to find an optimal solution for

Sk= {items labeled 1, 2, .. k}


5/27/2018 0 1 Knapsack

11/5411

If items are labeled 1..n, then a subproblem would be

to find an optimal solution for Sk= {items labeled

1, 2, .. k}

This is a reasonable subproblem definition.

The question is: can we describe the final solution

(Sn) in terms of subproblems (Sk)?

Unfortunately, we cantdo that.


5/27/2018 0 1 Knapsack

12/5412

Max weight: W = 20

For S4:Total weight: 14

Maximum benefit: 20

w1 =2b1 =3

w2 =4b2 =5

w3 =5b3 =8

w4 =3b4 =4 wi bi

10

85

54

43

32

Weight Benefit

9

Item

#

4

3

2

1

5

S4

S5

w1 =2b1 =3

w2 =4b2 =5

w3 =5b3 =8

w5 =9b5 =10

For S5:

Total weight: 20

Maximum benefit: 26

Solution for S4is

not part of the

solution for S5!!!

?


5/27/2018 0 1 Knapsack

13/5413

As we have seen, the solution for S4is not part of the

solution for S5

So our definition of a subproblem is flawed and weneed another one!


5/27/2018 0 1 Knapsack

14/5414




and benefit value





5/27/2018 0 1 Knapsack

15/5415

Lets add another parameter: w, which will represent

the maximum weight for each subset of items

The subproblem then will be to compute V[k,w], i.e.,to find an optimal solution for Sk= {items labeled 1,

2, .. k} in a knapsack of size w


5/27/2018 0 1 Knapsack

16/5416

The subproblem will then be to compute V[k,w], i.e.,

to find an optimal solution for Sk= {items labeled 1,

2, .. k} in a knapsack of size w

Assuming knowing V[i, j], where i=0,1, 2, k-1,

j=0,1,2, w, how to derive V[k,w]?

Recursive Formula for

subproblems

5/27/2018 0 1 Knapsack

17/5417

It means, that the best subset of Skthat has total

weight wis:

1) the best subset of Sk-1that has total weight w, or2) the best subset of Sk-1that has total weight w-wkplus

the item k

else}],1[],,1[max{

if],1[],[

kk

k

bwwkVwkV

wwwkVwkV

Recursive formula for subproblems:

Recursive Formula for

subproblems (continued)

5/27/2018 0 1 Knapsack

18/5418

Recursive Formula

The best subset of Skthat has the total weight w,either contains item kor not.

First case: wk>w. Item kcant be part of the solution,

since if it was, the total weight would be > w, which

is unacceptable. Second case: wkw. Then the item kcan be in the

solution, and we choose the case with greater value.

else}],1[],,1[max{

if],1[],[

kk

k

bwwkVwkV

wwwkVwkV

5/27/2018 0 1 Knapsack

19/5419

for w = 0 to WV[0,w] = 0

for i = 1 to n

V[i,0] = 0

for i = 1 to n

for w = 0 to W

if wi V[i-1,w]

V[i,w] = bi+ V[i-1,w- wi]

else

V[i,w] = V[i-1,w]

else V[i,w] = V[i-1,w] // wi> w

0-1 Knapsack Algorithm

5/27/2018 0 1 Knapsack

20/5420

for w = 0 to W

V[0,w] = 0

for i = 1 to n

V[i,0] = 0

for i = 1 to n

for w = 0 to W

< the rest of the code >

What is the running time of this algorithm?

O(W)

O(W)

Repeat ntimes

O(n*W)

Remember that the brute-force algorithm

takes O(2n)

Running time

5/27/2018 0 1 Knapsack

21/5421

Lets run our algorithm on the

following data:

n = 4 (# of elements)

W = 5 (max weight)

Elements (weight, benefit):

(2,3), (3,4), (4,5), (5,6)

Example

5/27/2018 0 1 Knapsack

22/5422

for w = 0 to W

V[0,w] = 0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W

Example (2)

5/27/2018 0 1 Knapsack

23/5423

for i = 1 to n

V[i,0] = 0

0

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W

Example (3)

5/27/2018 0 1 Knapsack

24/54

24

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]

else V[i,w] = V[i-1,w] // wi

> w

0

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

0

i=1

bi=3

wi=2

w=1

w-wi=-1

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W

0

0

0

Example (4)

5/27/2018 0 1 Knapsack

25/54

25

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

300

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=1

bi=3

wi=2

w=2

w-wi=0

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]

else V[i,w] = V[i-1,w] // wi

> w

Example (5)

5/27/2018 0 1 Knapsack

26/54

26

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

300

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=1

bi=3

wi=2

w=3

w-wi=1

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


3

Example (6)

5/27/2018 0 1 Knapsack

27/54

27

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

300

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=1

bi=3

wi=2

w=4

w-wi=2

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


3 3

Example (7)

5/27/2018 0 1 Knapsack

28/54

28

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

300

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=1

bi=3

wi=2

w=5

w-wi=3

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


3 3 3

Example (8)

5/27/2018 0 1 Knapsack

29/54

29

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=2

bi=4

wi=3

w=1

w-wi=-2

3 3 3 3

0

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


Example (9)

I

5/27/2018 0 1 Knapsack

30/54

30

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=2

bi=4

wi=3

w=2

w-wi=-1

3 3 3 3

3

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


0

Example (10)

I

5/27/2018 0 1 Knapsack

31/54

31

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=2

bi=4

wi=3

w=3

w-wi=0

3 3 3 3

0

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


43

Example (11)

I

5/27/2018 0 1 Knapsack

32/54

32

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=2

bi=4

wi=3

w=4

w-wi=1

3 3 3 3

0

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


43 4

Example (12)

It

5/27/2018 0 1 Knapsack

33/54

33

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=2

bi=4

wi=3

w=5

w-wi=2

3 3 3 3

0

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


73 4 4

Example (13)

It

5/27/2018 0 1 Knapsack

34/54

34

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=3

bi=5

wi=4

w= 1..3

3 3 3 3

0 3 4 4

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


7

3 40

Example (14)

It

5/27/2018 0 1 Knapsack

35/54

35

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=3

bi=5

wi=4

w= 4

w- wi=0

3 3 3 3

0 3 4 4 7

0 3 4 5

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


Example (15)

It

5/27/2018 0 1 Knapsack

36/54

36

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=3

bi=5

wi=4

w= 5

w- wi=1

3 3 3 3

0 3 4 4 7

0 3 4

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


5 7

Example (16)

It

5/27/2018 0 1 Knapsack

37/54

37

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=4

bi=6

wi=5

w= 1..4

3 3 3 3

0 3 4 4

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


7

3 40

70 3 4 5

5

Example (17)

It

5/27/2018 0 1 Knapsack

38/54

38

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

0

0

0

0 0 0 0 000

1

2

3

4 50 1 2 3

4

i\W i=4

bi=6

wi=5

w= 5

w- wi=0

3 3 3 3

0 3 4 4 7

0 3 4

if wi V[i-1,w]


else

V[i,w] = V[i-1,w]


5

7

7

0 3 4 5

Example (18)

5/27/2018 0 1 Knapsack

39/54

39

Exercise

P303 8.2.1 (a).

How to find out which items are in the optimal subset?

5/27/2018 0 1 Knapsack

40/54

40

Comments

This algorithm only finds the max possible value

that can be carried in the knapsack

i.e., the value in V[n,W]

To know the items that make this maximum value,an addition to this algorithm is necessary

5/27/2018 0 1 Knapsack

41/54

41

All of the information we need is in the table.

V[n,W] is the maximal value of items that can beplaced in the Knapsack.

Let i=n and k=Wif V[i,k] V[i1,k] then

mark the ithitem as in the knapsack

i = i1, k= k-wi

else

i = i1 // Assume the ithitem is not in the knapsack

//Could it be in the optimally packed knapsack?

How to find actual Knapsack

Items

Items:

5/27/2018 0 1 Knapsack

42/54

42

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

00

0

0 0 0 0 000

1

23

4 50 1 2 3

4

i\W i=4

k= 5

bi=6

wi=5V[i,k] = 7

V[i1,k] =7

3 3 3 3

0 3 4 4 70 3 4

i=n, k=W

while i,k > 0

if V[i,k] V[i1,k] then


i = i1, k= k-wi

else

i = i

1

5 7

0 3 4 5 7

Finding the Items

Items:

5/27/2018 0 1 Knapsack

43/54

43

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

00

0

0 0 0 0 000

1

23

4 50 1 2 3

4

i\W i=4

k= 5

bi=6

wi=5V[i,k] = 7

V[i1,k] =7

3 3 3 3

0 3 4 4 70 3 4

i=n, k=W

while i,k > 0



i = i1, k= k-wi

else

i = i1

5 7

0 3 4 5 7

Finding the Items (2)

Items:

5/27/2018 0 1 Knapsack

44/54

44

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

00

0

0 0 0 0 000

1

23

4 50 1 2 3

4

i\W i=3

k= 5

bi=5

wi=4V[i,k] = 7

V[i1,k] =7

3 3 3 3

0 3 4 4 70 3 4

i=n, k=W

while i,k > 0



i = i1, k= k-wi

else

i = i1

5 7

0 3 4 5 7


Items:

5/27/2018 0 1 Knapsack

45/54

45

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

00

0

0 0 0 0 000

1

23

4 50 1 2 3

4

i\W i=2

k= 5

bi=4

wi=3V[i,k] = 7

V[i1,k] =3

k wi=2

3 3 3 3

0 3 4 4 70 3 4

i=n, k=W

while i,k > 0



i = i1, k= k-wi

else

i = i1

5 7

0 3 4 5 7

7


Items:

5/27/2018 0 1 Knapsack

46/54

46

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

00

0

0 0 0 0 000

1

23

4 50 1 2 3

4

i\W i=1

k= 2

bi=3

wi=2V[i,k] = 3

V[i1,k] =0

k wi=0

3 3 3 3

0 3 4 4 70 3 4

i=n, k=W

while i,k > 0



i = i1, k= k-wi

else

i = i1

5 7

0 3 4 5 7

3


Items:

5/27/2018 0 1 Knapsack

47/54

47

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

00

0

0 0 0 0 000

1

23

4 50 1 2 3

4

i\W

3 3 3 3

0 3 4 4 70 3 4

i=n, k=W

while i,k > 0


mark the nthitem as in the knapsack

i = i1, k= k-wi

else

i = i

1

5 7

0 3 4 5 7

i=0

k= 0

The optimal

knapsack

should contain

{1, 2}


Items:

5/27/2018 0 1 Knapsack

48/54

48

Items:

1: (2,3)

2: (3,4)

3: (4,5)4: (5,6)

00

00

0

0 0 0 0 000

1

23

4 50 1 2 3

4

i\W

3 3 3 3

0 3 4 4 70 3 4

i=n, k=W

while i,k > 0


mark the nthitem as in the knapsack

i = i1, k= k-wi

else

i = i

1

5 7

0 3 4 5 7

The optimal

knapsack

should contain

{1, 2}

7

3


5/27/2018 0 1 Knapsack

49/54

49

Memorization (Memory Function Method)

Goal:

Solve only subproblems that are necessary and solve it only once

Memorizationis another way to deal with overlapping subproblems

in dynamic programming

With memorization, we implement the algorithm recursively:

If we encounter a new subproblem, we compute and store the solution.

If we encounter a subproblem we have seen, we look up the answer

Most useful when the algorithm is easiest to implement recursively

Especially if we do not need solutions to all subproblems.

5/27/2018 0 1 Knapsack

50/54

50

fori = 1 ton

forw = 1 toW

V[i,w] = -1

forw = 0 toW

V[0,w] = 0

fori = 1 ton

V[i,0] = 0

MFKnapsack(i, w)

if V[i,w] < 0

ifw < wi

value = MFKnapsack(i-1, w)else

value = max(MFKnapsack(i-1, w),

bi+ MFKnapsack(i-1, w-wi))

V[i,w] = value

returnV[i,w]

0-1 Knapsack Memory Function Algorithm

5/27/2018 0 1 Knapsack

51/54

51

Dynamic programming is a useful technique of

solving certain kind of problems

When the solution can be recursivelydescribed in

terms of partial solutions, we can store thesepartial solutions and re-use them as necessary

(memorization)

Running time of dynamic programming algorithmvs. nave algorithm:

0-1 Knapsack problem: O(W*n)vs. O(2n)

Conclusion

5/27/2018 0 1 Knapsack

52/54

52

In-Class Exercise

5/27/2018 0 1 Knapsack

53/54

53

Brute-Force Approach

Design and Analysis of Algorithms - Chapter 8 53

5/27/2018 0 1 Knapsack

54/54

Dynamic-Programming Approach

(1) SMaxV(0) = 0

(2) MaxV(0) = 0

(3) for i=1 to n

(4) SMaxV(i) = max(SmaxV(i-1)+xi, 0) (5) MaxV(i) = max(MaxV(i-1), SMaxV(i))

(6) return MaxV(n)

Run the algorithm on the following example instance:

30, 40, -100, 10, 20, 50, -60, 90, -180, 100

54

Documents

0 1 Knapsack