The Unbounded Knapsack Problem and the Generalized Cordel ... · The Unbounded Knapsack Problem and the Generalized Cordel Property Lisa Schreiber Friedrich-Schiller-Universitat Jena,¨

Knapsack Problems Penalty Method GeCoP Experimental Results Open Questions

The Unbounded Knapsack Problemand the

Generalized Cordel Property

Lisa Schreiber

Friedrich-Schiller-Universitat Jena,Institut fur Angewandte Mathematik

November the 26th, 2010


Outline

1 Knapsack Problems

2 A Penalty Method for the Unbounded Knapsack Problem

3 The Generalized Cordel Property (GeCoP)

4 Experimental Results

5 Open Questions


Outline

1 Knapsack Problems




5 Open Questions


Outline

1 Knapsack Problems




5 Open Questions


Outline

1 Knapsack Problems




5 Open Questions


Outline

1 Knapsack Problems




5 Open Questions


Outline

1 Knapsack Problems




5 Open Questions


The Knapsack Problem

Given n items and a knapsack with capacity C.

Every item has a value v(i) and a weight w(i).Question: Which items shall be packed into the knapsacksuch that the capacity C is not injured and the total value isas large as possible?

Definition: Knapsack Problem

maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C

G = {0,1}: 0-1 Knapsack ProblemG = N: Unbounded Knapsack Problem



Given n items and a knapsack with capacity C.Every item has a value v(i) and a weight w(i).

Question: Which items shall be packed into the knapsacksuch that the capacity C is not injured and the total value isas large as possible?


maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C




Given n items and a knapsack with capacity C.Every item has a value v(i) and a weight w(i).Question: Which items shall be packed into the knapsacksuch that the capacity C is not injured and the total value isas large as possible?


maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C






maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C






maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C

G = {0,1}: 0-1 Knapsack Problem

G = N: Unbounded Knapsack Problem





maxx∈Gn

f (x) :=n

∑i=1

v(i)xi subject ton

∑i=1

w(i)xi ≤ C



Outline

1 Knapsack Problems




5 Open Questions


The Penalty Method

Aim: Compute not only an optimal solution but also one (ormore) alternative solutions.

two Criteria:1 An alternative should be good with respect to the objective

function.2 An alternative should not be too similar to the optimal

solution.

One good method: the Penalty Methodexamined by Schwarz (2003), Sameith (2005) andDornfelder (2009)


The Penalty Method

Aim: Compute not only an optimal solution but also one (ormore) alternative solutions.two Criteria:

1 An alternative should be good with respect to the objectivefunction.

2 An alternative should not be too similar to the optimalsolution.



The Penalty Method






The Penalty Method




One good method: the Penalty Method

examined by Schwarz (2003), Sameith (2005) andDornfelder (2009)


The Penalty Method






The Penalty Method for the Knapsack Problem

The goal is to compute a second solution, which shalldiffer from the optimal solution B0.have a good function value.

Main idea: Punish the items used in the optimal solutionby reducing their values.

Important properties:

ε increases→ punishment gets higherB0(i) > B0(j)→ punishment of item i is higher











vε (i) =

{v (i) , if B0(i) = 0v (i) · [1− ε · B0(i)] , if B0(i) > 0







vε (i) =








vε (i) =


Important properties:ε increases→ punishment gets higher

B0(i) > B0(j)→ punishment of item i is higher





vε (i) =


Important properties:ε increases→ punishment gets higherB0(i) > B0(j)→ punishment of item i is higher





vε (i) = v (i) · [1− ε · B0(i)]



A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]

optimal solution: B0 = (2,0,1,0) with f (B0) = 15

e.g. ε = 0.7. This leads to the following punished values:

vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9

C, vε = [−2.4, 8, 0.9, 1] and w provide Bε = (0,1,2,0) asbest solution and penalty alternative


A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]



vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9



A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]



vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9



A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]



vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9



A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]



vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9



A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]



vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9



A first example

C = 13values v =[6,8,3,1]

weights w=[5,7,3,6]



vε(2) = v(2) = 8 andvε(4) = v(4) = 1

vε(1) = v (1) · [1− ε · B0(1)] = 6 · [1− 0.7 · 2] = −2.4

vε(3) = v (3) · [1− ε · B0(3)] = 3 · [1− 0.7 · 1] = 0.9



Properties of Penalty Alternatives

Properties of Penalty Alternatives (Schwarz 2003)B penalty-alternative⇒ B is optimal for all parameters ε inan optimality interval IB = [ε1, ε2] and nowhere else

P1,P2 penalty alternatives with optimality intervals I1 and I2⇒ three possible cases

1 I1 = I22 I1 ∩ I2 = {ε}, intersection contains only a single epsilon3 I1 ∩ I2 = ∅, empty intersection

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .

This provides an algorithm, how to compute all penaltyalternatives P0,P1, . . . ,Pk !



Properties of Penalty Alternatives (Schwarz 2003)B penalty-alternative⇒ B is optimal for all parameters ε inan optimality interval IB = [ε1, ε2] and nowhere elseP1,P2 penalty alternatives with optimality intervals I1 and I2⇒ three possible cases


∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .





1 I1 = I2

2 I1 ∩ I2 = {ε}, intersection contains only a single epsilon3 I1 ∩ I2 = ∅, empty intersection

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .





1 I1 = I22 I1 ∩ I2 = {ε}, intersection contains only a single epsilon

3 I1 ∩ I2 = ∅, empty intersection

∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .






∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .






∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .






∞0ε0 ε1 ε2

P0 = B0 P1 P2 . . .




vε (i) = v (i) · [1− ε · B0(i)]

⇒ fε(B) =n

∑i=1

vε(i) · B(i) =n

∑i=1

v(i) · [1− ε · B0(i)] · B(i)

=n

∑i=1

v(i) · B(i)︸︷︷︸=f (B)

−εn

∑i=1

v(i) · B0(i) · B(i)︸︷︷︸=:p(B)



vε (i) = v (i) · [1− ε · B0(i)]

⇒ fε(B) =n

∑i=1

vε(i) · B(i) =n

∑i=1

v(i) · [1− ε · B0(i)] · B(i)

=n

∑i=1

v(i) · B(i)︸︷︷︸=f (B)

−εn

∑i=1

v(i) · B0(i) · B(i)︸︷︷︸=:p(B)

· · · ∞0ε0 ε1 ε2 εk

threshold parameters

f (P0) ≥ f (P1) > . . . > f (Pk)p (P0) ≥ p (P1) > . . . > p (Pk)



vε (i) = v (i) · [1− ε · B0(i)]

⇒ fε(B) =n

∑i=1

vε(i) · B(i) =n

∑i=1

v(i) · [1− ε · B0(i)] · B(i)

=n

∑i=1

v(i) · B(i)︸︷︷︸=f (B)

−εn

∑i=1

v(i) · B0(i) · B(i)︸︷︷︸=:p(B)

· · · ∞0ε0 ε1 ε2 εk

threshold parameters

f (P0) ≥ f (P1) > . . . > f (Pk)p (P0) ≥ p (P1) > . . . > p (Pk)


Threshold Parameters

Let ε be the threshold parameter between the optimalityintervals of the two penalty alternatives Bl and Br .e.g.: Bl = Pi and Br = Pi+1Then we can compute ε the following way.

fε (Bl) = fε (Br )

⇔ f (Bl)− ε · p (Bl) = f (Br )− ε · p (Br )

⇔ ε =f (Bl)− f (Br )

p (Bl)− p (Br )


An Algorithm for Computing all Penalty Alternatives(Schwarz)

1 Initialization: Compute B0 (optimal solution) and B∞.Go to step 2 with [Bl = B0,Br = B∞].

2 Compute the possible threshold parameter ε between Bland Br and the penalty alternative Bε. Then we considerthe following two cases:fε (Bε) = fε (Bl) = fε (Br ):

bagagagsadgsagsadgdsaglabla

No further branching. ε is the real thresholdparameter between Bl and Br .

fε (Bε) 6= fε (Bl) = fε (Br ):

bagagagsadgsagsadgdsaglabla

With Bε we found a new penalty alternative,so we have to branch. Go to step 2 with[Bl = Bl ,Br = Bε] and [Bl = Bε,Br = Br ]


Outline

1 Knapsack Problems




5 Open Questions


From Chess history

Siegbert Tarrasch(1862−1934)In every chess positionthere exists exactly onebest move!

Emanuel Lasker(1868−1941)In every chess positionthere exist as manyappropriate moves asthere are different players!

Oskar Cordel’s (1843−1913) “Three moves law”In every chess position there exists either

exactly one best moveor at least three equally best moves.

This assumption does not hold in every chess position!


From Chess history







From Chess history







From Chess history







Generalized Cordel Property (GeCoP)

Generalizing Cordel: examine differences betweencandidates

moves a1 and a2 are equally good, if |f (a1)− f (a2)| isvery small

Definition: Generalized Cordel Property (GeCoP)

a1,a2, and a3 solutions of a given maximization problemf (a1) ≥ f (a2) ≥ f (a3)

define differences di := f (ai)− f (ai+1)

We say that a1,a2, and a3 fulfill the Generalized CordelProperty (GeCoP), iff the following holds:

d1 ≥ d2 (GeCoP)



Generalizing Cordel: examine differences betweencandidatesmoves a1 and a2 are equally good, if |f (a1)− f (a2)| isvery small





d1 ≥ d2 (GeCoP)








d1 ≥ d2 (GeCoP)




Definition: Generalized Cordel Property (GeCoP)a1,a2, and a3 solutions of a given maximization problem

f (a1) ≥ f (a2) ≥ f (a3)



d1 ≥ d2 (GeCoP)




Definition: Generalized Cordel Property (GeCoP)a1,a2, and a3 solutions of a given maximization problemf (a1) ≥ f (a2) ≥ f (a3)



d1 ≥ d2 (GeCoP)







d1 ≥ d2 (GeCoP)







d1 ≥ d2 (GeCoP)


Connection between Cordel and GeCoP

as a reminder: Cordel said there are never exactly twobest moves!

best moves in chess position: a1,a2 and a3

assume a1 and a2 are equally good (two best moves)

⇒ d1 = f (a1)− f (a2) will be very small⇒ d2 = f (a2)− f (a3) has to be very small too,

because of GeCoP (d1 ≥ d2)⇒ at least three best moves!

My thesis: analyzing the validity of GeCoP for severalkinds of optimization problems andkinds of solutions (e.g. three best solutions,penalty-alternatives)





















assume a1 and a2 are equally good (two best moves)⇒ d1 = f (a1)− f (a2) will be very small

⇒ d2 = f (a2)− f (a3) has to be very small too,because of GeCoP (d1 ≥ d2)

⇒ at least three best moves!






assume a1 and a2 are equally good (two best moves)⇒ d1 = f (a1)− f (a2) will be very small⇒ d2 = f (a2)− f (a3) has to be very small too,

because of GeCoP (d1 ≥ d2)

⇒ at least three best moves!

















GeCoP for Penalty Alternatives

0ε0 ε1 ε2P0 P1 P2 . . .



0ε0 ε1 ε2P0 P1 P2 . . .

d1 := f(P0)− f(P1) d2 := f(P1)− f(P2)



0ε0 ε1 ε2P0 P1 P2 . . .

d1 := f(P0)− f(P1) d2 := f(P1)− f(P2)≥?


Outline

1 Knapsack Problems




5 Open Questions


Random instances

according to Martello, Toth: “Knapsack Problems” (1990)two parameters v , r , e.g. v = 1000, r = 100w(i) uniformly randomly generated in the int range [1, v ]

C = 0.5 ·∑ni=1 w(i)

Three different types of instances:

Uncorrelated: v(i) uniformly randomly generatedin the range [1, v ].

Weakly correlated: v(i) uniformly randomly generated inthe range [w(i)− r ,w(i) + r ]with respect to v(i) > 0.

Strongly correlated: v(i) = w(i) + r .


Random instances

according to Martello, Toth: “Knapsack Problems” (1990)two parameters v , r , e.g. v = 1000, r = 100w(i) uniformly randomly generated in the int range [1, v ]C = 0.5 ·∑n

i=1 w(i)

Three different types of instances:





Random instances


i=1 w(i)Three different types of instances:





Random instances







Random instances







Random instances







Comparison of P (d1 ≥ d2) and P (d2 ≥ d3)

w(i), v(i) ∈ [1,100], v(i) ∈ [w(i)− 10,w(i) + 10], v(i) = w(i) + 10

●● ●

●●

●

●●

● ●

●

●

5 10 20 50 100

0.0

0.2

0.4

0.6

0.8

1.0

●● ● ●

●● ●

●

● ●

●●

● ●

●● ●

●

●

●

5n =

Prob(d1 ≥ d2)

●

●

●

uncorrelatedweakly correlatedstrongly correlated

●

●● ●

●●

● ● ● ●●

●

5 10 20 50 100

0.0

0.2

0.4

0.6

0.8

1.0

●

●

●

●● ● ●

● ●● ● ●

● ● ● ● ●

● ● ●

5n =

Prob(d2 ≥ d3)

●

●

●




w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●●

●

●●

●

●● ●

● ●

●

●

●●

●

● ●

5 10 20 50 100 200 500

0.0

0.2

0.4

0.6

0.8

1.0

● ●●●

●●

●●

● ●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

●

●

5 1000n =

Prob(d1 ≥ d2)

●

●

●


● ●●●●●

●

●●

●●

●

●

●● ●

●●

5 10 20 50 100 200 500

0.0

0.2

0.4

0.6

0.8

1.0

● ●●●

●●● ●

●● ●

●

●●

● ●

●●

●●

●● ●●

●●

● ●

5 1000n =

Prob(d2 ≥ d3)

●

●

●




w(i), v(i) ∈ [1,100 000], v(i) ∈ [w(i)− 10 000,w(i) + 10 000], v(i) = w(i) + 10 000

●●

●

●●

●

●

● ●

●● ● ● ● ●

● ●●

5 10 20 50 100 200 500

0.0

0.2

0.4

0.6

0.8

1.0

●

●

●●●

●

●

●● ●

● ● ●● ● ●

●●

●●

●●

●●

●

●●

●● ●

● ● ●●

●●

5 1000n =

Prob(d1 ≥ d2)●

●

●


●●●

●●●

●

●

●

●

● ● ●

● ●●

●

●

5 10 20 50 100 200 500

0.0

0.2

0.4

0.6

0.8

1.0

●

●

●●●

●

●

●

●

●

●●

●

●●

●

●●

● ●

●

●

●●

●

●

●

●

●●

●●

●●

●●

5 1000n =

Prob(d2 ≥ d3)●

●

●



Results

1 Prob (d1 ≥ d2) �Prob (d2 ≥ d3)

2 The behavior changes heavily with the rangesize v .

The first phenomenon is unique according to our wider studieson other optimization problems!


Results

1 Prob (d1 ≥ d2) �Prob (d2 ≥ d3)




Results

1 Prob (d1 ≥ d2) �Prob (d2 ≥ d3)




Average number of penalty alternatives

w(i), v(i) ∈ [1,100], v(i) ∈ [w(i)− 10,w(i) + 10], v(i) = w(i) + 10

●●●●●

● ● ●

● ●

●●

20 40 60 80 100

24

68

10

●●●●●●

●●

●●

●●

●

●●●●● ●

●

35

79

●

●

●




w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●●●●●●

●●●

●●●

●

●●

●

● ●

0 200 400 600 800 1000

24

68

10

●

●●●●●

●●●●

●

●

●

●●

●●

●●●

●●

●●

●●

●

●

35

79

●

●

●




w(i), v(i) ∈ [1,100 000], v(i) ∈ [w(i)− 10 000,w(i) + 10 000], v(i) = w(i) + 10 000

●

●●●●●

●

●

●

●

●●

●

● ● ●

●●

0 200 400 600 800 1000

24

68

10

●●●●●●

●

●

●

●

●●

●

● ●●

● ●

●

●●●

●●

●

●

●

●

●

● ●

●

●●

●

●

35

79

●

●

●



Outline

1 Knapsack Problems




5 Open Questions


Open Questions

1 Do these phenomenona also occur at other optimizationproblems?

2 Is there a better method to generate alternatives for theunbounded knapsack problem?Is there a good method, that generates more than only 6alternatives in average?

3 How can the results presented be used in practice?Is Cordel more than a Gedanken-Experiment?


Open Questions





Open Questions




additional slides




vε (i) = v (i) · [1− ε · B0(i)]


additional slides


The goal is to compute a second solution, which shalldiffer from a given reference solution Br .have a good function value.


vε (i) = v (i) · [1− ε · Br (i)]


additional slides

Reference Solutions

Possible reference solutions are:1 An optimal solution.

2 A Greedy solution

sort items in decreasing order of value per unit of weightput every item in sequence with maximal frequency in theknapsack such that the capacity is not exceeded

Example: v = [6,8,3,1], w = [5,7,3,6], C = 13

optimal solution: B0 = (2,0,1,0)

additional slides

Reference Solutions

Possible reference solutions are:1 An optimal solution.2 A Greedy solution


Example: v = [6,8,3,1], w = [5,7,3,6], C = 13vw ≈ [1.2, 1.14, 1, 0.17]→ order: 1,2,3,4

(2,0,0,0)→ (2,0,0,0)→ (2,0,1,0)

additional slides

Reference Solutions



Example: v = [6,8,3,1], w = [5,7,3,6], C = 13vw ≈ [1.2, 1.14, 1, 0.17]→ order: 1,2,3,4(2,0,0,0),C ′ = 3

additional slides

Reference Solutions



Example: v = [6,8,3,1], w = [5,7,3,6], C = 13vw ≈ [1.2, 1.14, 1, 0.17]→ order: 1,2,3,4(2,0,0,0),C ′ = 3→ (2,0,0,0),C ′ = 3

additional slides

Reference Solutions



Example: v = [6,8,3,1], w = [5,7,3,6], C = 13vw ≈ [1.2, 1.14, 1, 0.17]→ order: 1,2,3,4(2,0,0,0),C ′ = 3→ (2,0,0,0),C ′ = 3→ (2,0,1,0),C ′ = 0

additional slides

How often is the Greedy Solution Optimal?

w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●

●●●●●●● ● ● ● ● ● ●

0 20 40 60 80 100

0.5

0.6

0.7

0.8

0.9

1.0

●

●●●

●●●●

●

●●

●

● ●

●

●

●●●

●●●● ● ● ● ●

●

n =

P(greedy optimal)

●

●

●


additional slides

Optimal Solution as Reference Solution

w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●●

●

●●

●

●● ●

● ●

●

●

●●

●

● ●

5 10 20 50 100 200 500

0.0

0.2

0.4

0.6

0.8

1.0

● ●●●

●●

●●

● ●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

●

●

5 1000n =

Prob(d1 ≥ d2)

●

●

●


● ●●●●●

●

●●

●●

●

●

●● ●

●●

5 10 20 50 100 200 500

0.0

0.2

0.4

0.6

0.8

1.0

● ●●●

●●● ●

●● ●

●

●●

● ●

●●

●●

●● ●●

●●

● ●

5 1000n =

Prob(d2 ≥ d3)

●

●

●


additional slides

Greedy Solution as Reference Solution

w(i), v(i) ∈ [1,1 000], v(i) ∈ [w(i)− 100,w(i) + 100], v(i) = w(i) + 100

●

●

●●

●●

● ●● ●

●

●

●

● ●

●

●

●

5 10 20 50 100 200 500

0.0

0.2

0.4

0.6

0.8

1.0

●

●●●

●●

●● ●

●

●●

●

●●

●

● ●●

●●●

● ●

● ●

●

●

5 1000n =

Prob(d1 ≥ d2)

●

●

●


●●

●●

●●

● ●

● ●

● ●

● ●● ● ●

●

5 10 20 50 100 200 500

0.0

0.2

0.4

0.6

0.8

1.0

●

●●●●●

●● ● ●

●●

●

●

●●

●●

●●●●

● ●●

●●

●

5 1000n =

Prob(d2 ≥ d3)

●

●

●


additional slides

Other Studied Optimization-Problems

Shortest Path ProblemBinary Knapsack ProblemMinimal Spanning TreesAssignment Problemsome theoretical problem types

Documents

The Unbounded Knapsack Problem and the Generalized Cordel ... · The Unbounded Knapsack Problem and the Generalized Cordel Property Lisa Schreiber Friedrich-Schiller-Universitat Jena,¨