1

CS 106Computing Fundamentals II

Chapter 79“Recursion”

Herbert G. Mayer, PSU CSHerbert G. Mayer, PSU CSstatus 6/24/2013status 6/24/2013

Initial content copied verbatim fromInitial content copied verbatim fromCS 106 material developed byCS 106 material developed by

CS professors: Cynthia Brown & Robert MartinCS professors: Cynthia Brown & Robert Martin

2

Syllabus Recursion Recursion Mimic a While LoopMimic a While Loop RecursionRecursion Step by StepStep by Step Recursion can be ClearerRecursion can be Clearer Recursion IssuesRecursion Issues Functional ProgramsFunctional Programs Sample: FibonacciSample: Fibonacci MergingMerging Merge Sort OutlineMerge Sort Outline AnalysisAnalysis More ideasMore ideas

3

Recursion• Recursion is a powerful technique for thinking about Recursion is a powerful technique for thinking about

certain types of algorithmscertain types of algorithms

• It can be used to simulate a loop, or other kinds of It can be used to simulate a loop, or other kinds of applicationsapplications

• Simplistically speaking, a recursive function is a function Simplistically speaking, a recursive function is a function that calls itselfthat calls itself

• More correctly and completely: A recursive function is a More correctly and completely: A recursive function is a function that is partly defined in simpler versions of itselffunction that is partly defined in simpler versions of itself

• As with a while loop, there is a danger of an infinite As with a while loop, there is a danger of an infinite recursion –referred to as infinite regress-- so recursion –referred to as infinite regress-- so there has to there has to be a test for stopping it, and something (usually a be a test for stopping it, and something (usually a parameter) must change between callsparameter) must change between calls

3

4

Mimic a While Loop

Here’s the code for the while loop to build a Here’s the code for the while loop to build a multiplication table in our Loop Multiplication demo:multiplication table in our Loop Multiplication demo:

j = 1j = 1

Do While Do While j <= MAXj <= MAX

lstAnswer.AddItem(strM & " X " & lstAnswer.AddItem(strM & " X " & __CStr(j) & " = " & _CStr(j) & " = " & _ CStr(numM CStr(numM

* j))* j))

j = j + 1j = j + 1

LoopLoop

4

5

A Recursive Procedure Procedure call to get things started:Procedure call to get things started:

RecurMult(numM, strM, 1)RecurMult(numM, strM, 1)

Procedure code:Procedure code:

Sub Sub RecurMult(RecurMult(ByValByVal numM numM As IntegerAs Integer, , ByValByVal strM strM As As StringString, , ByValByVal j j As IntegerAs Integer))

IfIf j <= MAX j <= MAX ThenThen

lstAnswer.Items.Add(strM & " X " & _lstAnswer.Items.Add(strM & " X " & _CStr(j) & " = " & _CStr(j) & " = " & _ CStr(numM * j))CStr(numM * j))

RecurMult(numM, strM, j + 1)RecurMult(numM, strM, j + 1)

End IfEnd If

End SubEnd Sub

5

6

How does it work?

• Note MAX is global. In the initial call, parameter j is 1. Note MAX is global. In the initial call, parameter j is 1. We print the line of the table with j = 1 and do the next We print the line of the table with j = 1 and do the next call, with j = 2.call, with j = 2.

• In the second call, we print the line of the table with j In the second call, we print the line of the table with j = 2, and do the call with j = 3.= 2, and do the call with j = 3.

• This continues till we do a call with j > MAX. In that This continues till we do a call with j > MAX. In that case we just return without initiating another call.case we just return without initiating another call.

• This triggers all the other calls, in reverse sequence, This triggers all the other calls, in reverse sequence, to return, and we’re doneto return, and we’re done

6

7

It’s not an easy concept…

So now we’ll look at a simple exampleSo now we’ll look at a simple example

7

8

Recursive Definition

Sometimes it makes sense to define a quantity Sometimes it makes sense to define a quantity recursively: for example, the sum of the first recursively: for example, the sum of the first nn numbers:numbers:

S(1) = 1S(1) = 1

S(n) = n + S(n-1) for n>1S(n) = n + S(n-1) for n>1

(As opposed to S(n) = 1 + 2 + … + n)(As opposed to S(n) = 1 + 2 + … + n)

8

9

Questions for Recursive Definitions

• If I know how to compute the answer for n-1, how do If I know how to compute the answer for n-1, how do I compute it for n? (Potentially you can use any I compute it for n? (Potentially you can use any values for arguments smaller than n in the definition)values for arguments smaller than n in the definition)

• In our example, S(n) = S(n-1) + nIn our example, S(n) = S(n-1) + n

• How does the recursion stop? What is the “bottom” How does the recursion stop? What is the “bottom” argument and what is the value for that argument?argument and what is the value for that argument?

• In our example, S(1) = 1In our example, S(1) = 1

9

10

Recursive Computation

Private Function Private Function SumOfN(SumOfN(ByVal ByVal n n As IntegerAs Integer) ) As IntegerAs Integer

IfIf n <= 1 n <= 1 Then Then ‘expecting n = 1 but catch error‘expecting n = 1 but catch error

SumOfN = 1SumOfN = 1

ElseElse

SumOfNSumOfN = = n + SumOfN(n – 1))n + SumOfN(n – 1))

End IfEnd If

End FunctionEnd Function

X = SumOfN(5) returns 5 + 4 + 3 + 2 + 1 = 15X = SumOfN(5) returns 5 + 4 + 3 + 2 + 1 = 15

10

11

Step by Step

SumOfN(5) returns 5 + SumOfN(4)SumOfN(5) returns 5 + SumOfN(4)

SumOfN(4) returns 4 + SumOfN(3)SumOfN(4) returns 4 + SumOfN(3)

SumOfN(3) returns 3 + SumOfN(2)SumOfN(3) returns 3 + SumOfN(2)

SumOfN(2) returns 2 + SumOfN(1)SumOfN(2) returns 2 + SumOfN(1)

SumOfN(1) returns 1SumOfN(1) returns 1

Once we “hit bottom,” Sum0fN(1)returns its value; then Once we “hit bottom,” Sum0fN(1)returns its value; then SumOfN(2) can compute and return its value, and so SumOfN(2) can compute and return its value, and so on.on.

11

12

Compare to While Loop:

j = nj = n

sum = 0sum = 0

Do While Do While j >= 1j >= 1

sum = sum + jsum = sum + j

j = j – 1j = j – 1

LoopLoop

12

13

Recursion can be Clearer

Recursion directly implements the definitionRecursion directly implements the definition

It is plain to see that it computes the correct valueIt is plain to see that it computes the correct value

Coming up with the loop is not that easy for more Coming up with the loop is not that easy for more complex recursive definitions, and its structure is complex recursive definitions, and its structure is quite differentquite different

Note there is also a closed form for this recursion: S(n) Note there is also a closed form for this recursion: S(n) = n(n + 1) / 2= n(n + 1) / 2

Closed forms can be hard to findClosed forms can be hard to find

13

14

Recursion Issues

• As with a while loop, a recursion can be infinite if we As with a while loop, a recursion can be infinite if we do not include a way to stop itdo not include a way to stop it

• The test to see if it is done should usually be the first The test to see if it is done should usually be the first thing a recursive function doesthing a recursive function does

• Recursion uses more resources than a loop and it Recursion uses more resources than a loop and it may not be possible to do a very large recursion may not be possible to do a very large recursion (depends on language implementation)(depends on language implementation)

14

15

Functional Programming

• This is a style of programming that replaces loops This is a style of programming that replaces loops with recursions and assignment statements with with recursions and assignment statements with parameter/argument associationsparameter/argument associations

• After you get the knack of doing it, it can result in After you get the knack of doing it, it can result in very clear, concise programsvery clear, concise programs

• There are languages especially designed to support There are languages especially designed to support this stylethis style

15

16

Example: Fibonacci Numbers

A program that implements Fibonacci numbers A program that implements Fibonacci numbers several ways. Here’s the recursive definition:several ways. Here’s the recursive definition:

Fib(0) = 0Fib(0) = 0

Fib(1) = 1Fib(1) = 1

Fib(n) = Fib(n – 1) + Fib(n – 2)Fib(n) = Fib(n – 1) + Fib(n – 2)

16

17

Demo: Fibonacci Numbers

17

18

Recursive Fib(5)

18

Fib(5)

Fib(4)

Fib(3)

Fib(3)

Fib(2)

Fib(2)

Fib(1)

Fib(2)

Fib(1)

Fib(1)

Fib(0)

Fib(1)

Fib (0)

Fib(1)

Fib(0)

19

Recursive Fib(5): Order of Calls

19

Fib(5)

Fib(4)

Fib(3)

Fib(3)

Fib(2)

Fib(2)

Fib(1)

Fib(2)

Fib(1)

Fib(1)

Fib(0)

Fib(1)

Fib (0)

Fib(1)

Fib(0)

1

23

4

56

8

9

7

1011

12

13

14

20

Array Fib(5)

20

0

1

0

0

0

0

0

1

1

0

0

0

0

1

1

2

0

0

0

1

1

2

3

0

0

1

1

2

3

5

In the first picture, we add elements 0 and 1 to get element 2 (Fib(2)), giving the second picture. Next add elements 1 and 2 to get Fib(3). Etc. (We start indexing with zero.)

21

Loop Fib(5)

fn = fnm1 + fnm2fn = fnm1 + fnm2

fnm2 = fnm1fnm2 = fnm1

fnm1 = fnfnm1 = fn

21

fn 0

fnm1 1

fnm2 0

fn 1

fnm1 1

fnm2 1

fn 2

fnm1 2

fnm2 1

fn 3

fnm1 3

fnm2 2

fn 5

fnm1 5

fnm2 3

start

step 1

step 2

step 3

step 4

22

Recursive Fib(5): Order of Calls

22

Fib(5)

Fib(4)

Fib(3)

Fib(3)

Fib(2)

Fib(2)

Fib(1)

Fib(2)

Fib(1)

Fib(1)

Fib(0)

Fib(1)

Fib (0)

Fib(1)

Fib(0)

1

23

4

56

8

9

7

1011

12

13

14

23

Memoized Fib(5)

23

Fib(5)

Fib(4)

Fib(3)

Fib(3)

Fib(2)

Fib(2)

Fib(1)

Fib(1)

Fib(0)

1

23

4

56

7

8

24

Using Recursion in Sorting

• Motivation: to develop a fast sorting algorithmMotivation: to develop a fast sorting algorithm

• Recall Selection Sort: it takes time proportional to nRecall Selection Sort: it takes time proportional to n22, , where we use the number of comparisons as a way to where we use the number of comparisons as a way to estimate the time, and n is the number of elements to estimate the time, and n is the number of elements to be sortedbe sorted

• This is too slow for sorting large data sets, even on a This is too slow for sorting large data sets, even on a very fast computervery fast computer

25

Why Selection Sort is Slow

• Selection sort and similar algorithms require us to do Selection sort and similar algorithms require us to do something similar to comparing every element to something similar to comparing every element to every other elementevery other element

• We can be clever and avoid some of the comparisons We can be clever and avoid some of the comparisons but the basic nature of the algorithm-- that it takes but the basic nature of the algorithm-- that it takes time proportional to ntime proportional to n22-- remains the same unless we -- remains the same unless we use a radically different approachuse a radically different approach

• So in selection sort, instead of n + n +…+n, n times, = So in selection sort, instead of n + n +…+n, n times, = n*n, we have n + (n-1) + (n-2) + … + 2 + 1, which n*n, we have n + (n-1) + (n-2) + … + 2 + 1, which equals n(n-1)/2. Less than half as big, but still roughly equals n(n-1)/2. Less than half as big, but still roughly proportional to nproportional to n22, especially for large n, especially for large n

26

The Fundamental Idea

• The fundamental idea is to divide the problem The fundamental idea is to divide the problem roughly in half each time, solve the subproblems, and roughly in half each time, solve the subproblems, and then put them back togetherthen put them back together

• If done cleverly, this can give us a time proportional If done cleverly, this can give us a time proportional to n log n. to n log n.

• One way to do this is based on merging sorted lists. One way to do this is based on merging sorted lists. Let’s first look at how that works.Let’s first look at how that works.

27

Merging

Say we have two sorted lists that we want to combine to Say we have two sorted lists that we want to combine to make one sorted list. For example:make one sorted list. For example:

List A: 1, 5, 8, 15, 19List A: 1, 5, 8, 15, 19

List B: 2, 5, 7, 20, 22List B: 2, 5, 7, 20, 22

Method: look at the first element in each list. Put the Method: look at the first element in each list. Put the smaller one in the answer. If one list is empty, put all smaller one in the answer. If one list is empty, put all the elements from the other list in the answerthe elements from the other list in the answer

28

Step 1

List A: 1, 5, 8, 15, 19List A: 1, 5, 8, 15, 19

List B: 2, 5, 7, 20, 22List B: 2, 5, 7, 20, 22

Answer: [empty]Answer: [empty]

List A: 5, 8, 15, 19List A: 5, 8, 15, 19

List B: 2, 5, 7, 20, 22List B: 2, 5, 7, 20, 22

Answer: 1Answer: 1

29

Step 2

List A: 5, 8, 15, 19List A: 5, 8, 15, 19

List B: 2, 5, 7, 20, 22List B: 2, 5, 7, 20, 22

Answer: 1Answer: 1

List A: 5, 8, 15, 19List A: 5, 8, 15, 19

List B: 5, 7, 20, 22List B: 5, 7, 20, 22

Answer: 1, 2Answer: 1, 2

30

Step 3

List A: 5, 8, 15, 19List A: 5, 8, 15, 19

List B: 5, 7, 20, 22List B: 5, 7, 20, 22

Answer: 1, 2Answer: 1, 2

List A: 8, 15, 19List A: 8, 15, 19

List B: 5, 7, 20, 22List B: 5, 7, 20, 22

Answer: 1, 2, 5Answer: 1, 2, 5

31

Step 4

List A: 8, 15, 19List A: 8, 15, 19

List B: 5, 7, 20, 22List B: 5, 7, 20, 22

Answer: 1, 2, 5Answer: 1, 2, 5

List A: 8, 15, 19List A: 8, 15, 19

List B: 7, 20, 22List B: 7, 20, 22

Answer: 1, 2, 5, 5Answer: 1, 2, 5, 5

32

Step 5

List A: 8, 15, 19List A: 8, 15, 19

List B: 7, 20, 22List B: 7, 20, 22

Answer: 1, 2, 5, 5Answer: 1, 2, 5, 5

List A: 8, 15, 19List A: 8, 15, 19

List B: 20, 22List B: 20, 22

Answer: 1, 2, 5, 5, 7Answer: 1, 2, 5, 5, 7

33

Step 6

List A: 8, 15, 19List A: 8, 15, 19

List B: 20, 22List B: 20, 22

Answer: 1, 2, 5, 5, 7Answer: 1, 2, 5, 5, 7

List A: 15, 19List A: 15, 19

List B: 20, 22List B: 20, 22

Answer: 1, 2, 5, 5, 7, 8Answer: 1, 2, 5, 5, 7, 8

34

Step 7

List A: 15, 19List A: 15, 19

List B: 20, 22List B: 20, 22

Answer: 1, 2, 5, 5, 7, 8Answer: 1, 2, 5, 5, 7, 8

List A: 19List A: 19

List B: 20, 22List B: 20, 22

Answer: 1, 2, 5, 5, 7, 8, 15Answer: 1, 2, 5, 5, 7, 8, 15

35

Step 8

List A: 19List A: 19

List B: 20, 22List B: 20, 22

Answer: 1, 2, 5, 5, 7, 8, 15Answer: 1, 2, 5, 5, 7, 8, 15

List A: List A:

List B: 20, 22List B: 20, 22

Answer: 1, 2, 5, 5, 7, 8, 15, 19Answer: 1, 2, 5, 5, 7, 8, 15, 19

36

Step 9

List A: List A:

List B: 20, 22List B: 20, 22

Answer: 1, 2, 5, 5, 7, 8, 15, 19Answer: 1, 2, 5, 5, 7, 8, 15, 19

List A: List A:

List B:List B:

Answer: 1, 2, 5, 5, 7, 8, 15, 19, 20, 22Answer: 1, 2, 5, 5, 7, 8, 15, 19, 20, 22

37

Comparisons for Merging

• Until we empty one of the lists, it takes one Until we empty one of the lists, it takes one comparison to get one element into the answercomparison to get one element into the answer

• So, roughly, the number of comparisons to merge So, roughly, the number of comparisons to merge two sorted lists is proportional to the total number of two sorted lists is proportional to the total number of elements in the two lists.elements in the two lists.

38

Merge Sort Outline

• Divide the list in half and sort each halfDivide the list in half and sort each half

• Merge the two sorted halves into a sorted listMerge the two sorted halves into a sorted list

• How do we sort each half? Using MergeSort!How do we sort each half? Using MergeSort!

• Huh? Isn’t this a circular definition or something? Huh? Isn’t this a circular definition or something?

39

Start at the Bottom

• We’ll think about it this way: suppose we have an We’ll think about it this way: suppose we have an unsorted list of 8 elements. We are going to divide it unsorted list of 8 elements. We are going to divide it into 8 tiny lists of one element each, and merge them into 8 tiny lists of one element each, and merge them in pairsin pairs

• Here’s our example list. We’ll show it as an array so it Here’s our example list. We’ll show it as an array so it is easy to talk about each element.is easy to talk about each element.

15 2 3 8 5 12 20 1

0 1 2 3 4 5 6 7

40

Merging One-Element Pairs

15 2 3 8 5 12 20 10 1 2 3 4 5 6 7

2 15 3 8 5 12 1 200 1 2 3 4 5 6 7

41

Merging Two-Element Pairs

2 15 3 8 5 12 1 200 1 2 3 4 5 6 7

2 3 8 15 1 5 12 200 1 2 3 4 5 6 7

42

Merging Four-Element Pairs

2 3 8 15 1 5 12 200 1 2 3 4 5 6 7

1 2 3 5 8 12 15 200 1 2 3 4 5 6 7

43

Analysis

• We double the size of the pairs each time. The We double the size of the pairs each time. The number of times we can double to reach size n, number of times we can double to reach size n, starting with 1, is log n. So there are log n stages.starting with 1, is log n. So there are log n stages.

• The time for each stage is proportional to n, since the The time for each stage is proportional to n, since the total elements being merged each time is n.total elements being merged each time is n.

• So the overall time is n log nSo the overall time is n log n

44

More Ideas

• There are lots more recursive sorting algorithmsThere are lots more recursive sorting algorithms

• For example, in Quicksort, we divide the problem in For example, in Quicksort, we divide the problem in half (in time n) by putting the elements bigger than half (in time n) by putting the elements bigger than some given element in the back and the smaller ones some given element in the back and the smaller ones in the front.in the front.

• Do the same to each piece till you get to size one: Do the same to each piece till you get to size one: there are log n stagesthere are log n stages

45

Sorting Summary (1)

• Selection Sort and Bubble Sort are not recursive. Selection Sort and Bubble Sort are not recursive. They both take time proportional to nThey both take time proportional to n22, though Bubble , though Bubble sort can be somewhat faster than Selection sortsort can be somewhat faster than Selection sort

• Mergesort and Quicksort are both recursive in Mergesort and Quicksort are both recursive in concept, though there are ways to avoid explicit concept, though there are ways to avoid explicit recursions in the implementation. Mergesort is recursions in the implementation. Mergesort is excellent especially for sorting data too big to all fit in excellent especially for sorting data too big to all fit in memory. It always takes time proportional to n log nmemory. It always takes time proportional to n log n

46

Sorting Summary (2)

• Quicksort is recursive and usually very fast, Quicksort is recursive and usually very fast, proportional to n log n. In the worst case (sorted proportional to n log n. In the worst case (sorted data!) it can take time proportional to ndata!) it can take time proportional to n22, though. , though. Clever variations try to avoid this problem.Clever variations try to avoid this problem.

• The best possible time to sort just based on The best possible time to sort just based on comparisons is proportional to n log n. You can do a comparisons is proportional to n log n. You can do a bit better if, for example, you know that the data will bit better if, for example, you know that the data will be numbersbe numbers

47

Sorting Summary (3)

• For small amounts of data just use a simple For small amounts of data just use a simple algorithm, or rely on the ones built into Excel or VBAalgorithm, or rely on the ones built into Excel or VBA

• If your program handles huge amounts of data then If your program handles huge amounts of data then writing your own fast sort is one thing to try for writing your own fast sort is one thing to try for speeding it upspeeding it up

• Our main purpose here was to give you a feeling for Our main purpose here was to give you a feeling for the vast variety of clever algorithms that can be the vast variety of clever algorithms that can be developed to perform a taskdeveloped to perform a task

48

The Sorting Sampler

The sorting sampler lets you play around with several The sorting sampler lets you play around with several algorithms and look at how they behave when sorting algorithms and look at how they behave when sorting the same data.the same data.