Lecture 5 COMPSCI.220.FS.T - 2004 1

Worst-Case Performance

• Upper bounds: simple to obtain• Lower bounds: a difficult matter...• Worst case data may be unlikely to be met in practice• Unknown “Big-Oh” constants c and n0 may not be small• Inputs in practice lead to much lower running times• Example: the most popular fast sorting algorithm,

QuickSort, has O(n2) running time in the worst case but in practice the time is O(n log n)

Lecture 5 COMPSCI.220.FS.T - 2004 2

Average-Case Performance

• Estimate average time for each operation• Estimate frequencies of operations • May be a difficult challenge... (take COMPSCI.320

for details)• May be no natural “average” input at all• May be hard to estimate (average time for each

operation depends on data)

Lecture 5 COMPSCI.220.FS.T - 2004 3

Recurrent Algorithms

Divide-and-conquer principle:• to divide a large problem into smaller ones and

recursively solve each subproblem, then• to combine solutions of the subproblems to solve

the original problemRunning time: by a recurrence relation combining the size

and number of the subproblems and the cost of dividing the problem into the subproblems

Lecture 5 COMPSCI.220.FS.T - 2004 4

“Telescoping” a Recurrence• Recurrence relation and its base condition (i.e.,

the difference equation and initial condition): T(n) = 2T(n1) + 1; T(1) = 1

• Closed (explicit) form for T(n) by “telescoping”:

1)1T(2)2T(

...

1)2T(2)1T(

1)1T(2)T(

nn

nn

Lecture 5 COMPSCI.220.FS.T - 2004 5

“Telescoping” ≡ Substitution

11

232

2

2)1T(2)2T(2

...

2)3T(2)2T(2

2)2T(2)1T(2

1)1T(2)T(

nnn

nn

nn

nn

1222...221)T( 112 nnnn

Lecture 5 COMPSCI.220.FS.T - 2004 6

Basic Recurrence: 1

• Closed (explicit) form by “telescoping”:2

)1()T( )1T()T(

nnnnnn

1)1T(

2)1T()2T(

...

1)2T()1T(

)1T()T(

nnn

nnn

Lecture 5 COMPSCI.220.FS.T - 2004 7

1: Explicit Expression for T(n)

2

)1()1()2(...21

)1()2(...2)1T(

)1()2(...3)2T(...

)1()2()3T(

)1()2T(

)1T()T(

nnnnn

nnn

nnn

nnnn

nnn

nnn

Lecture 5 COMPSCI.220.FS.T - 2004 8

Guessing to Solve a Recurrence

• Infer (guess) a hypothetic solution T(n); n ≥ 0 from a sequence of numbers T(0), T(1), T(2), …, obtained from the recurrence relation

• Prove T(n) by math induction:Base condition: T holds for n nbase ,e.g. T(0) or T(1)

Induction hypothesis to verify: for every n nbase, if T holds for n 1, then T holds for n

Strong induction: if T holds for nbase, …, n - 1, then…

Lecture 5 COMPSCI.220.FS.T - 2004 9

Explicit Expression for T(n)

• T(1) = 1; T(2) = 1 + 2 = 3; T(3) = 3 + 3 = 6;

T(4) = 6 + 4 = 10 Hypothesis:• Base condition holds: T(1) = 1∙2 2 = 1• If the hypothetic closed-form relationship T(n) holds for

n 1 then it holds also for n:

• Thus, the expression for T(n) holds for all n 1

2

)1()T(

nnn

2

)1(

2

)1()1T()T(

nnn

nnnnn

Lecture 5 COMPSCI.220.FS.T - 2004 10

Basic Recurrence: 2

• Repeated halving principle: halve the input in one step:

• “Telescoping” (for n = 2m):

nnnn 2log)T( 1)2T()T(

0)2T(

1)2T()2T(

1)2T()2T(

...

1)2T()2T(

1)2T()2T(

0

01

12

21

1

mm

mm

Lecture 5 COMPSCI.220.FS.T - 2004 11

2: Explicit Expression for T(n)

nnmm

m

mm

2

0

1

2

1

log)T( )2T(

11...11)2T(

1...11)2T(...

11)2T(

1)2T()2T(

Lecture 5 COMPSCI.220.FS.T - 2004 12

Basic Recurrence: 3

• Scan and halve the input:

• “Telescoping” (for n = 2m):nnnnn 2)T( )2T()T(

1)2T(

2)2T()2T(

2)2T()2T(

...

2)2T()2T(

2)2T()2T(

0

101

212

121

1

mmm

mmm

Lecture 5 COMPSCI.220.FS.T - 2004 13

3: Explicit Expression for T(n)

nnmm

mm

mm

mmm

mmm

2)T( 12)2T(

22...22)2T(

22...2)2T(...

22)2T(

2)2T()2T(

1

1210

121

12

1

Lecture 5 COMPSCI.220.FS.T - 2004 14

Basic Recurrence: 4

• “Divide-and-conquer” prototype:

• “Telescoping”:

• For

nnnnnn 2log)T( )2T(2)T(

0)1T( ;1

2

2T)T(

n

n

n

n

12

)2T(

2

)2T(2

1

1

m

m

m

mmn

Lecture 5 COMPSCI.220.FS.T - 2004 15

4: Explicit Expression for T(n)

m

mm

mmmm

1...10

11...112)2T(

11...12)2T(...

112)2T(

12)2T(2)2T(

00

11

22

11

nnnm mm2log)T( 2)2T(

Lecture 5 COMPSCI.220.FS.T - 2004 16

General “Divide-and-Conquer”

Theorem: The recurrence with integer constants a≥1 and b≥2 and positive constants

c and k has the solution:

Proof by telescoping:

ccnbnan k )1T()T()T( ;

mkmmm cbbabbn )T()T( 1

abnO

abnnO

abnO

nkk

kk

kab

if)(

if)log(

if)(

)T(

log

Lecture 5 COMPSCI.220.FS.T - 2004 17

General “Divide-and-Conquer”

• Telescoping:

kmmm

kmmm

kmmm

mkmm

cbaTabTa

acbbTabTa

acbbTabaT

cbbaTbT

11

)2(3322

)1(221

1

)1()(

... ...

)()(

)()(

)()(

m

t

m

t

ta

bmtktm

mkkmkmmm

kcabac

cbacbcbacab

0 0

)1(1 ...)T(

Lecture 5 COMPSCI.220.FS.T - 2004 18

Capabilities and Limitations

• Rough complexity analysis cannot result immediately in an efficient practical program but it helps in predicting of empirical running time of the program

• “Big-Oh” analysis is unsuitable for small input and hides the constants c and n0 crucial for a practical task

• “Big-Oh” analysis is unsuitable if costs of access to input data items vary and if there is lack of sufficient memory

• But complexity analysis provides ideas how to develop new efficient methods