ALGORITHMS

THIRD YEAR

BANHA UNIVERSITYFACULTY OF COMPUTERS AND INFORMATIC

Lecture five

Dr. Hamdy M. Mousa

Recurrences

• A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs.

• For Examples,• The worst-case running time T (n) of the MERGE-

SORT procedure could be described by the recurrence

T (n) = (1) if n = 1 ,T(n) = 2T (n/2) + (n) if n > 1 ,

• solution T (n) = (n lg n).

Recurrence

Recurrence Solution Methods• Three methods for solving recurrences - that is, for

obtaining asymptotic “” or “O” bounds on the solution.

• The substitution method, we guess a bound and then use mathematical induction to prove our guess correct.

• The recursion-tree method converts the recurrence into a tree whose nodes represent the costs incurred at various levels of the recursion; we use techniques for bounding summations to solve the recurrence.

• The master method provides bounds for recurrences of the formT (n) = aT (n/b) + f (n),where a ≥ 1, b > 1, and f (n) is a given function.

Running time

• The running time T (n) of an algorithm is only defined when n is an integer, since for most algorithms, the size of the input is always an integer.

• For example, the recurrence describing the worst-case running time of MERGE-SORT

1)(])2/([])2/([

,1)1()(

nifnnTnT

nifnT

Running time• Example: T (n) = 2T (n/2) + (n),

with solution T (n) = (n lg n).• Since the running time of an algorithm on a

constant-sized input is a constant,• The boundary conditions are usually expressed

as “T (n) = O(1) for sufficiently small n.”• When we desire an exact, rather than an

asymptotic, solution, we need to deal with boundary conditions.

• In practice, we just use asymptotics most of the time, and we ignore boundary conditions.

Substitution method

• The substitution method for solving recurrences entails two steps:

1. Guess the solution.

2. Use induction to find the constants and show that the solution works.

Example:

1])2/([2

,11)(

nifnnT

nifnT

1. Guess:

T (n) = n lg n + n. [Here, we have a recurrence with an exact function,

rather than asymptotic notation, and the solution is also exact rather than asymptotic. We’ll have to check boundary conditions and the base case.]

2. Induction:

Basis: n = 1 n lg n + n = 1 = T (n)

Substitution method

Substitution method

Inductive step: • Inductive hypothesis is that T (k) = k lg k + k for all k < n.• We’ll use this inductive hypothesis for T (n/2).

For the substitution method:• Name the constant in the additive term.• Show the upper (O) and lower () bounds

separately. Might need to use different constants for each.

Example:T (n) = 2T (n/2)+(n). If we want to show an upper bound of

T (n) = 2T (n/2) + O(n), we write T (n) ≤ 2T (n/2) + cn for some positive constant c.

Substitution method

1. Upper bound:

Guess: T (n) ≤ dn lg n for some positive constant d. We are given c in the recurrence, and we get to choose d as any positive constant. It’s OK for d to depend on c.

Therefore, T (n) = O(n lg n).

Substitution method

2. Lower bound: Write

T (n) ≥ 2T (n/2) + cn for some positive constant c.

Guess: T (n) ≥ dn lg n for some positive constant d.

Substitution:

Substitution method

Avoiding pitfalls

It is easy to err in the use of asymptotic notation. • For example, in the recurrence

T (n) = 2T (n/2) + n we can falsely “prove” T (n) = O(n) by guessing

T (n) ≤ cn and then arguingT (n) ≤ 2(c n/2) + n

≤ cn + n= O(n) , ⇐ wrong!!

• since c is a constant. The error is that we haven’t proved the exact form of the inductive hypothesis, that is, that T (n) ≤ cn.

Changing variables• Sometimes, a little algebraic manipulation can make an

unknown recurrence similar to one you have seen before.

As an example, consider the recurrenceT (n) = 2T (√n) + lg n ,

• We can simplify this recurrence, though, with a change of variables. Renaming m = lg n yields

T (2m) = 2T (2m/2) + m .• rename S(m) = T (2m) to produce the new recurrence

S(m) = 2S(m/2) + m ,• which is very much like recurrence (4.4). Indeed, this

new recurrence has the same solution:S(m) = O(m lgm).

Changing back from S(m) to T (n), we obtain

T (n) = T (2m) = S(m) = O(m lg m) = O(lg n lg lg n).

The recursion-tree method

• In a recursion tree, each node represents the cost of a single subproblem somewhere in the set of recursive function invocations. We sum the costs within each level of the tree to obtain a set of per-level costs, and then we sum all the per-level costs to determine the total cost of all levels of the recursion.

• Recursion tree use to generate a guess. Then verify by substitution method.

Example: T (n) = T (n/3)+T (2n/3)+(n).

For upper bound, rewrite as T (n) ≤ T (n/3) + T (2n/3) + cn;

For lower bound, as T (n) ≥ T (n/3) + T (2n/3) + cn.

• By summing across each level, the recursion tree shows the cost at each level of recursion (minus the costs of recursive calls, which appear in subtrees):

The recursion-tree method

There are log3 n full levels, and after log3/2 n levels, the problem size is down to 1.

•Each level contributes ≤ cn. •Lower bound guess: ≥ dn log3 n = (n lg n) for some positive constant d. •Upper bound guess: ≤ dn log3/2 n = O(n lg n) for some positive constant d.

•Then prove by substitution.

The recursion-tree method

Therefore, T (n) = O(n lg n).

The recursion-tree method

2. Lower bound:Guess: T (n) ≥ dn lg n.Substitution:

Same as for the upper bound, but replacing ≤ by ≥. End up needing

Therefore, T (n) = (n lg n).

Since T (n) = O(n lg n) and T (n) = (n lg n),

we conclude thatT (n) = (n lg n).

The recursion-tree method