1 Recursion Algorithm Analysis Standard Algorithms Chapter 7

1

RecursionAlgorithm Analysis

Standard Algorithms

Chapter 7

2

Recursion

• Consider things that reference themselves– Cat in the hat in the hat in the hat …– A picture of a picture– Having a dream in your dream!!

• Recursion has its base in mathematical induction

• Recursion always has– an anchor (or base or trivial) case– an inductive case

3

Recursion

• A recursive function will call or reference itself.

• Considerint R(int x){ return 1 + R(x); }

• What is wrong with this picture?– Nothing will stop repeated recursion– Like an endless loop, but will eventually cause

your program to run out of memory

The problem is that this function has no anchor.

The problem is that this function has no anchor.

4

Recursion

A proper recursive function will have• An anchor or base case

– the function’s value is defined for one or more values of the parameters

• An inductive or recursive step – the function’s value (or action) for the current

parameter values is defined in terms of …– previously defined function values (or

actions) and/or parameter values.

5

Recursive Example

int Factorial(int n){ if (n == 0) return 1; else return n * Factorial(n - 1);}

• Which is the anchor?• Which is the inductive or recursive part?• How does the anchor keep it from going forever?

6

A Bad Use of Recursion

• Fibonacci numbers1, 1, 2, 3, 5, 8, 13, 21, 34f1 = 1, f2 = 1 … fn = fn -2 + fn -1

– A recursive functiondouble Fib (unsigned n){ if (n <= 2) return 1; else return Fib (n – 1) + Fib (n – 2); }

• Why is this inefficient?– Note the recursion tree on pg 327

7

Uses of Recursion

• Easily understood recursive functions are not always the most efficient algorithms

• "Tail recursive" functions– When the last statement in the recursive function is a

recursive invocation.– These are much more efficiently written with a loop

• Elegant recursive algorithms– Binary search (see pg 328)– Palindrome checker (pg 330)– Towers of Hanoi solution (pg 336)– Parsing expressions (pg 338)

8

Comments on Recursion

• Many iterative tasks can be written recursively– but end up inefficient

However• There are many problems with good

recursive solutions• And their iterative solutions are

– not obvious– difficult to develop

9

Algorithm Efficiency

• How do we measure efficiency– Space utilization – amount of memory

required– Time required to accomplish the task

• Time efficiency depends on :– size of input– speed of machine – quality of source code– quality of compiler

These vary from one platform to another

These vary from one platform to another

10

Algorithm Efficiency

• We can count the number of times instructions are executed– This gives us a measure of efficiency of an

algorithm

• So we measure computing time as:

T(n) = computing time of an algorithm for input of size n = number of times the instructions are executed

11

Example: Calculating the Mean

Task # times executed

1. Initialize the sum to 0 12. Initialize index i to 0 13. While i < n do following n+14. a) Add x[i] to sum n5. b) Increment i by 1 n6. Return mean = sum/n 1 Total 3n + 4

12

Computing Time Order of Magnitude

• As number of inputs increases T(n) = 3n + 4 grows at a rate proportional to n

• Thus T(n) has the "order of magnitude" n

• The computing time of an algorithm on input of size n, T(n) said to have order of magnitude f(n), written T(n) is O(f(n))

if … there is some constant C such that T(n) < Cf(n) for all sufficiently large values of n

13

Big Oh Notation

Another way of saying this:• The complexity of the algorithm is

O(f(n)).

• Example: For the Mean-Calculation Algorithm:

T(n) is O(n)

• Note that constants and multiplicative factors are ignored.

14

Big Oh Notation

• f(n) is usually simple:

n, n2, n3, ...2n

1, log2nn log2nlog2log2n

15

Big-O Notation

• Cost function– A numeric function that gives performance of

an algorithm in terms of one or more variables– Typically the variable(s) capture number of

data items

• Actual cost functions are hard to develop

• Generally we use approximating functions

16

Function Dominance

• Asymptotic dominance– g dominates f if there is a positive constant c

such that

• Example: suppose the actual cost function is

• Both of these will dominate T(n)

( ) ( )c g x f x

2( ) 1.01T n n

2

3

2( ) 1.5

3( )

T n n

T n n

for sufficiently large values of n

17

Estimating Functions

Characteristics for good estimating functions

• It asymptotically dominates the actual time function

• It is simple to express and understand

• It is as close an estimate as possible

2

2

( ) 5 100

( )

actual

estimate

T n n n

T n n

Because any constant c > 1 will make n2 larger

18

Estimating Functions

• Note how the c*n2 dominates

Function Dominance

020000400006000080000

0 100 200 300

Number of data items

Tim

e

Actual

n^2

1.5n^2

Thus we use n2 as an estimate of the

time required

19

Order of a Function

• To express time estimates concisely we use the concept “order of a function”

• Definition:Given two nonnegative functions f and g, the order of f is g, iff g asymptotically dominates f

• Stated – “f is of order g”– “f = O(g)” big-O notation

O stands for “Order”

20

Order of a Function

• Note the possible confusion– The notation does NOT say “the order of g is f”

nor does it say “f equals the order of g”

– It does say “f is of order g” ( )f O g

21

Big-O Arithmetic

• Given f and g functions, k a constant

1. ( ) ( )

2. ( ) ( ) ( )

( / ) ( ) / ( )

3. ( ) ( ) dominates

4. ( ) [ ( ), ( )]

O k f O f

O f g O f O g and

O f g O f O g

O f O g f g

O f g Max O f O g

22

Example: Calculating the Mean

Task # times executed

1. Initialize the sum to 0 12. Initialize index i to 0 13. While i < n do following n+14. a) Add x[i] to sum n5. b) Increment i by 1 n6. Return mean = sum/n 1 Total 3n + 4

Based on Big-O arithmetic this

algorithm has O(n)

Based on Big-O arithmetic this

algorithm has O(n)

23

Worst-Case Analysis

• The arrangement of the input items may affect the computing time.

• How then to measure performance?– best case not very informative – average too difficult to calculate– worst case usual measure

• Consider Linear search of the list a[0], . . . , a[n – 1].

24

Worst-Case Analysis

Algorithm:1. found = false.2. loc = 0.3. While (loc < n && !found )4. If item = a[loc] found = true // item found 5. Else Increment loc by 1 // keep searching

• Worst case: Item not in the list: TL(n) is O(n)

• Average case (assume equal distribution of values) is O(n)

Linear search of a[0] … a[n-1]

Linear search of a[0] … a[n-1]

25

Binary Search

1. found = false.2. first = 0.3. last = n – 1.4. While (first < last && !found )

5. Calculate loc = (first + last) / 2.6. If item < a[loc] then7. last = loc – 1.// search first half8. Else if item > a[loc] then9. first = loc + 1. // search last half

10. Else found = true. // item found

• Each pass cuts the list in half• Worst case : item not in list TB(n) = O(log2n)

Binary search of a[0] … a[n-1]

Binary search of a[0] … a[n-1]

26

Common Computing Time Functions

log2log2n log2n n n log2n n2 n3 2n

--- 0 1 0 1 1 2

0.00 1 2 2 4 8 4

1.00 2 4 8 16 64 16

1.58 3 8 24 64 512 256

2.00 4 16 64 256 4096 65536

2.32 5 32 160 1024 32768 4294967296

2.58 6 64 384 4096 262144 1.84467E+19

3.00 8 256 2048 65536 16777216 1.15792E+77

3.32 10 1024 10240 1048576 1.07E+09 1.8E+308

4.32 20 1048576 20971520 1.1E+12 1.15E+18 6.7E+315652

For our binary searchFor our binary search

27

Computing in Real Time• Suppose each instruction can be done in 1

microsecond

• For n = 256 inputs how long for various f(n)Function Time

log2log2n 3 microseconds

Log2n 8 microseconds

n .25 milliseconds

n log2n 2 milliseconds

n2 65 milliseconds

n3 17 seconds

2n 3.7+E64 centuries!!

28

Conclusion

• Algorithms with exponential complexity– practical only for situations where number of

inputs is small

• Bubble sort has O(n2)– OK for n < 100– Totally impractical for large n

29

Computing Times Of Recursive Functions

// Towers of Hanoivoid Move(int n, char source, char destination, char

spare){if (n <= 1) // anchor (base) case cout << "Move the top disk from " << source << " to " << destination << endl; else { // inductive case Move(n-1, source, spare, destination); Move(1, source, destination, spare); Move(n-1, spare, destination, source); }}

T(n) = O(2n)T(n) = O(2n)