19.102 - Computing II Recursion: How to solve a problem by doing some work and sending the rest out to be handled the same way. Many mathematical functions

19.102 - Computing II

Recursion:

How to solve a problem by doing some work and sending the rest out to be handled the same way.

Many mathematical functions are defined recursively, and redefining them in a non-recursive manner is NOT obvious.

Many problems can have their solution described recursively in an easy to understand manner, while a non-recursive solution can be described only with difficulty.


Recursion: some misconceptions.

It’s hard to understand.

NO, it’s not - but it may require some practice. Like many unfamiliar ways of doing something, it will require a bit of work to become “easy”.

It’s inefficient.

NO, it’s not. First of all, any description of a process that makes it EASIER for the human to understand and code correctly is to be preferred to a harder one. Second, efficiency depends (usually) more on choice of algorithm and compiler technology than on “twiddling bits”.


Summing Integers:

∑n = 0

m n = ??

0 + 1 + 2 + … + (m - 1) + m = ??

The standard formula for this sum is

(∑n = 0

m n) = m (m + 1) / 2,

So nobody in his right mind would actually SUM the things one at a time, since a quick formula exists. Similar formulae are known for the sums of many low powers.


Thinking Recursively:

split the solution into three parts:

a) do some work towards solving the problem;

b) apply the method to one (or more) smaller problem(s) on one (or more) proper subset(s) of the set of data;

c) glue the work done in a) and b) together.


Example:

go(bed, school):

from bed to bedroom_door

+ go(bedroom_door, school);

go(bedroom_door, school):

from bedroom_door to house_door

+ go(house_door, school);

etc…

go(school, school):

just return; (i.e., you’re done - do nothing else, or indicate success)


Apply this idea to the problem of summing the integers:

int SumInts(int low, int high)

{ /* first check the BASE CASE */

/* safety… don’t let low exceed high EVER */

if (low >= high) // done - return the last value

return(high);

else // this is where we do some work

return(low + SumInts(low + 1, high));

}


The “termination condition”, although safe, does NOT quite return the correct value if we input n < m. A better solution (although a little less obvious) would be:

int SumInts(int m, int n)

{

/* first check the BASE CASE */

if (m > n) /* safety… m has exceeded n */

return(0); /* overshot: no ints to add */

else /* do some work */

return(m + SumInts(m + 1, n));

}

We need to check this is correct: verify it!!


Let’s make a full program out of it:#include <stdio.h>long low, high;long SumInts(long m, long n){ /* first check the BASE CASE */ if (m > n) return(0);// overshot: no ints to add else return(m + SumInts(m + 1, n)); // do the work}int main(void){

printf("Enter a non-negative integer : ");scanf("%d", &low);printf("Enter another non-negative integer: ");scanf("%d", &high);printf("The sum is: %d.\n", SumInts(low, high));

}


Aside: The idea of a “stack frame”.

Each time a function is called, the system needs to save some information, in particular:

The values of the parameters passed;

The local variable declared in the body;

What the function is returning (return value);

Where the function is returning (return address);

The memory allocated to hold this information is called a “stack frame”. These frames are allocated by the system when a function is called and are deallocated when the function terminates.


MainProc A Proc D

B CProc D

Proc D Proc D

Time

Space

E


Recursive function calls could result in a large number of such memory chunks being in use at one time. This may cause problems, especially in situations when memory is at a premium.

Rudimentary compilers are unable to optimize code in such a way that such frames will be allocated only when necessary; more sophisticated compilers do perform such optimizations.

Check your compiler…

End of aside.


There are many ways of “solving a problem”: we saw how to add integers “going up” - essentially adding one integer at a time.

We can solve the same problem by adding “going down”, i.e. starting from the “upper” end of the set {m, m + 1, …, n - 1, n}



if (n < m) // safety… m has exceeded n

return(0); // overshot

else // do some work

return(n + SumInts(m, n - 1));

}


There is a method called “divide and conquer”: it splits the work into two roughly equal parts and glues the results:int SumInts(int m, int n)

{ int mid;


if (n < m) return(0); // overshot

else // do some work

{ mid = (m + n)/2; // compute midpoint

return(SumInts(m, mid - 1)

+ mid

+ SumInts(mid + 1, n));

}

}


What is wrong with this?


{ int mid;


if (n < m) // overshot

return(0);

else { // do some work

mid = (m + n)/2; // compute midpoint

return(SumInts(m, mid) // return result


}

}


Solution:


{ int mid;



else if (n == m) return(n); // endpoint

else // do the work

{ mid = (m + n)/2; // compute midpoint

return(SumInts(m, mid)


}

}


How can we visualize the behavior of a recursive algorithm?

int SumInts(int low, int high)


if (low >= high)

return(high);

else

return(low + SumInts(low + 1, high));

}

Try a SPECIFIC case: SumInts(0, 5).


SumInts(0, 5)

= 0 + SumInts(1, 5)

= 0 + (1 + SumInts(2, 5))

= 0 + (1 + (2 + SumInts(3, 5)))

= 0 + (1 + (2 + (3 + SumInts(4, 5))))

= 0 + (1 + (2 + (3 + (4 + SumInts(5, 5)))))

= 0 + (1 + (2 + (3 + (4 + 5))))

= 0 + (1 + (2 + (3 + 9)))

= 0 + (1 + (2 + 12))

= 0 + (1 + 14)

= 0 + 15 = 15 = (5*6)/2 // from the formula


0 5 0+

1 5

3+

4+54

5

1+

3

2 5 2+

55

4+53+92+121+140+15

15


Code for “visualization”. How can we change at least some of our recursive procedures so that the shape of the recursion becomes visible? We could begin by adding a couple of printf statements:

int SumInts(int low, int high){ /* first check the BASE CASE */

if (low >= high) {printf(“Base Case: %5d.\n”, high);return(high);

} else {printf(“low = %3d + SumInts(%d,%d).\n”,

low, low+1, high);return(low + SumInts(low + 1, high));

}}

Execution will produce the following output:

low = 5 + SumInts(6,10).low = 6 + SumInts(7,10).low = 7 + SumInts(8,10).low = 8 + SumInts(9,10).low = 9 + SumInts(10,10).Base Case: 10.

Not particularly informative. We could print both on entry and on exit from each function call - that might give a better picture.



int SumInts(int low, int high){ int recResult;

/* first check the BASE CASE */ if (low >= high) {

printf(“Base Case: %5d.\n”, high);return(high);

} else {printf(“= %3d + SumInts(%d,%d).\n”,

low, low+1, high);recResult = SumInts(low + 1, high);printf(“= %3d + %3d.\n”, low, recResult); return(low + recResult);

}}

Printout:

= 5 + SumInts(6,10).= 6 + SumInts(7,10).= 7 + SumInts(8,10).= 8 + SumInts(9,10).= 9 + SumInts(10,10).Base Case: 10.= 9 + 10.= 8 + 19.= 7 + 27.= 6 + 34.= 5 + 40.

SumInts gives 45



Better, but not particularly informative, since it gives us no sense of the depth of the recursion for each call: the whole thing is too flat. We need to introduce some other trick: how about spaces?

void EmitNSpaces(int spaces)

{ int index;

for (index = 0; index < spaces; index++)

printf(“ “);

}

Will print out any spaces number of spaces…

How do we keep track of how many spaces to print?

Add a parameter to the recursive function: level.


int SumInts(int low, int high, int level){ int recResult;


EmitNSpaces(level);printf(“Base Case: %5d.\n”, high);return(high);

} else {EmitNSpaces(level)printf(“= %3d + SumInts(%d,%d).\n”,

low, low+1, high);recResult = SumInts(low + 1, high, level+1);EmitNSpaces(level);printf(“= %3d + %3d.\n”,

low, recResult); return(low + recResult);

}}


The call SumInts(5, 10, 1);produces the output:

= 5 + SumInts(6,10). = 6 + SumInts(7,10). = 7 + SumInts(8,10). = 8 + SumInts(9,10). = 9 + SumInts(10,10). Base Case: 10. = 9 + 10. = 8 + 19. = 7 + 27. = 6 + 34. = 5 + 40.

SumInts gives 45


Add printed information about the level:int SumInts(int low, int high, int level){ int recResult;


printf(“Level = %3d; “,level); EmitNSpaces(level);printf(“ Base Case: %5d.\n”, high);return(high);

} else {printf(“Level = %3d; “,level);EmitNSpaces(level)printf(“%3d + SumInts(%d,%d).\n”,

low, low+1, high);recResult = SumInts(low + 1, high, level+1);printf(“Level = %3d; “,level);EmitNSpaces(level);printf(“%3d + %3d.\n”, low, recResult); return(low + recResult);

} }

The call SumInts(5, 10, 1);produces the output:

Level = 1; 5 + SumInts(6,10).Level = 2; 6 + SumInts(7,10).Level = 3; 7 + SumInts(8,10).Level = 4; 8 + SumInts(9,10).Level = 5; 9 + SumInts(10,10).Level = 6; Base Case: 10.Level = 5; 9 + 10.Level = 4; 8 + 19.Level = 3; 7 + 27.Level = 2; 6 + 34.Level = 1; 5 + 40.

SumInts gives 45press enter:



Solution Methods and Depth of the Recursion Tree.

In the previous solution, the depth of the recursion (the maximum number of ACTIVE stack frames) is equal to the number of integers being summed: summing 1000 integers requires 1000 stack frames. This is not serious if we sum a few integers this way, but many thousands could be a problem. Using the formula instead would be the preferred way - no recursion and no loops, just one addition, one multiplication and one division…

There may be situations where we can’t find a convenient formula - or where the mere existence of such a formula makes no sense (we’ll see some later in the course).


One of the solutions already shown lets us do MUCH better:int SumInts(int m, int n)

{ int mid;

// check BASE CASE (or cases…)


else if (n == m) return(n); // endpoint

else

{

mid = (m + n)/2; // compute midpoint

return(SumInts(m, mid)


}

}

The Formatted Output Function:int SumInts(int m, int n, int level){ int mid, leftH, rightH;

if (n < m) { // overshotprintf("Level = %3d; ",level); EmitNSpaces(level);printf("Error - Overshot: %5d.\n", 0);return(0);

} else if (n == m) {// endpoint - base caseprintf("Level = %3d; ",level); EmitNSpaces(level);printf("%d. Base Case.\n", n);return(n);

} else {mid = (m + n)/2; // compute midpointprintf("Level = %3d; ",level); EmitNSpaces(level);printf("SumInts(%d,%d) + SumInts(%d, %d). \n",m,mid,

mid+1,n);leftH = SumInts(m, mid,level + 1) ;rightH = SumInts(mid+1, n,level + 1);printf("Level = %3d; ",level); EmitNSpaces(level);printf("%d + %d. Returning Sum.\n",leftH, rightH);return(leftH + rightH);

}}


The Formatted Output:Level = 1; SumInts(5,7) + SumInts(8, 10). Level = 2; SumInts(5,6) + SumInts(7, 7). Level = 3; SumInts(5,5) + SumInts(6, 6). Level = 4; 5. Base Case.Level = 4; 6. Base Case.Level = 3; 5 + 6. Returning Sum.Level = 3; 7. Base Case.Level = 2; 11 + 7. Returning Sum.Level = 2; SumInts(8,9) + SumInts(10, 10). Level = 3; SumInts(8,8) + SumInts(9, 9). Level = 4; 8. Base Case.Level = 4; 9. Base Case.Level = 3; 8 + 9. Returning Sum.Level = 3; 10. Base Case.Level = 2; 17 + 10. Returning Sum.Level = 1; 18 + 27. Returning Sum.

SumInts gives 45press enter:



To compute SumInt(0, 10), the solution discussed earlier would require 11 stack frames active at the same time. How many active stack frames does this one require?

We’ll see that the actual number of stack frames CONSTRUCTED is still large (in fact, even larger), but NOT ALL OF THEM NEED TO BE ACTIVE AT THE SAME TIME: the amount of space required by the algorithm TO RUN is thus much less, although the total amount of space that needs to be MANAGED is more. This is an example of the rule: “there is no such thing as a free lunch”. Much of Computer Science is devoted to finding ways of making lunch cheaper, or at least well varied for roughly the same price. Remember that PRICE is not JUST execution time or space.


0 10

0 5 6 10

0 2 3 5 86 9 10

0 1 2 2 3 4 5 5 6 7 8 8 9 9 1010

0 0 1 1 3 3 4 4 6 6 7 7


As it turns out, this last algorithm requires a number of active stack frames roughly equal to the LOGARITHM in base 2 of the number of integers we want to sum…

1024 integers ≈ 10 stack frames;

1,000,000 integers ≈ 20 stack frames;

1,000,000,000 integers ≈ 30 stack frames;

Etc.

Not much space for an “inefficient recursive algorithm”…


The number of stack frames actually constructed in the example is about 20, and would, in general, be no more than twice as many as by the previous method: we have traded more total stack frames for a MUCH smaller depth of recursion at any one time. We have gained what might be a large amount of space, by giving up some time - the time to allocate and deallocate twice as many stack frames.


We have gained something else, though: by “dividing” we have made it possible to solve the subproblems independently. We could thus send one subproblem to one processor, and the other one to a second. The splitting of the subproblems into sub-subproblems would allow four processors to contribute to the solution, etc., up to the largest set of processors for which the overhead of keeping parallelism straight is more than what we gain by parallelizing (again, no free lunch).

What we have gained is OBVIOUS PARALLELISM - which may be a lot more important than the “trivial” doubling of a few stack frames...


Another example: fast raising to a power.

This works much better in programming environments where integers can be of arbitrary size, and not limited to a 32 (16?) bit ( 2 billion) representation. Here is the naïve way of doing it

long Power(long base, long exponent)

{ /* base != 0, exponent >= 0 */

if (exponent == 0) return(1);

else return(base*Power(base, exponent - 1));

}

This is fine, as long as the exponent is small (< a few dozen?) and the base is also small.


Unfortunately, secure encryption requires we deal with integers in the 150-200 decimal digit range… and beyond. Other fancy number theoretic studies require even bigger numbers.

A recursion which is 200 decimal digits deep in size? No matter how small the stack frame you allocate, there won’t be enough memory available in the universe to carry out the computation…

There won’t be enough time in the universe to complete it: at 1 nanosecond per frame, this means fewer than 1015 frames per day, which would then require 10185 days, or 3*10182 years to complete...


Here is a useful auxiliary function:

long Square(long x)

{

return(x*x);

}

Notice that it takes just ONE copy of the parameter, so that only one parameter evaluation is going to be needed. Once the value is obtained, it can be used TWICE. This is crucial.


long FastPower(long base, long exponent)

{

if (0 == (exponent % 2)) /* is exponent even? */

return(Square(FastPower(base, exponent/2)));

else

return(base*FastPower(base, exponent - 1));

}

This will require at most 2*log2(exponent) multiplications, rather than (exponent - 1) multiplications. Fortunately, after every operation, we only need deal with the remainder of the result after division by a fixed large number, so the ACTUAL numbers we need to multiply will not fill up the universe: just 400 decimal digits for a 200 decimal digit encryption scheme.


One should notice that log2(10200) ≈ 700, so that the maximum stack depth we could encounter is, roughly, 1400. Which is somewhat better than 10200.

All this could have been done non-recursively. Recursion is simply another technique for problem-solving: it gives us insights that other techniques either deny us, hide from us, or require the programming to be more complex.

The functions in Maple V: (NOT C or C++ !!!!)

> FastPower := proc(base, exponent, modulo)> if (exponent = 0) then 1> elif (0 = (exponent mod 2)) then > (Square(FastPower(base,exponent/2,modulo),modulo) > mod modulo) > else ((base*FastPower(base, exponent - 1, modulo)) > mod modulo);> fi;> end:

> Square := proc(x, modulo) > ((x*x) mod modulo); > end:


Example runs in Maple V (neither C nor C++ supports indefinite precision integer arithmetic):

> FastPower(123, 456, 78910); 34321

> FastPower(123456789101112131415, 9876543213579864211121314525758975321, 99887766554433221112131415161718192021222324252627282931987654321);36986746982596106906265409037450751115866469611492482412857379680

> FastPower(2, 16, 100000); 65536



Some List Manipulation Code.

It will be a little different from the code of the text, and will try to cover slightly different problems.

We start with a function that returns the contents of the first node of a list: this assumes we can return a structure - ANSI C does allow the return of any types OTHER than : a) array of T(where T is any type); b) function returning T. Also, structures CAN be assigned.

NodeInfo Head(NodeType *L)

{

return(L->Info);

}


A function that returns the REST of a list:NodeType *Tail(NodeType *L)

{

return(L->Link);

}

A function that CONSTRUCTS a list:NodeType *Cons(NodeInfo info, NodeType *L)

{

NodeType *N;

N = (NodeType *)malloc(sizeof(NodeType));

N->Info = info;

N->Link = L;

return(N);

}


A function that computes the LENGTH of a list:int Length(NodeType *L)

{ if (L == NULL) return(0);

else return(1 + Length(Tail(L)));

}

Its ITERATIVE form:int Length(NodeType *L)

{ int len = 0;

while (L != NULL) {

len++;

L = L->link;

}

return(len);

}


A function that returns the pointer to the Nth element of the list:

NodeType *Nth(NodeType *L, int n)

{ if (L == NULL) return(NULL);

else if (n == 0) return(L);

else return(Nth(Tail(L), n - 1));

}

Note the 0 corresponds to the first (or zeroth?) element of the list.

A reasonable design question would be: do I return a pointer or do I return the object (i.e. the Info structure)? A way of answering it can be guided by the question: what do I do if I don’t find it?


A DIFFERENT WAY OF IMPLEMENTING LINKED LISTS.

The current implementation of linked lists is not very general: the LINK and the INFORMATION fields appear together, in the SAME structure. All the list manipulation languages that have been developed over the decades opted for a different approach: separate the information from the links.

typedef NodeInfo…; /* this depends on the use */

typedef struct node {

NodeInfo *info;

node *Link;

} Node;

A node is just TWO pointers. Since pointers are all of the same form (their type must be changed, but they all take up one word), it is possible to just RE-CAST a pointer to one of a different type, so that, at least in principle, the same pointer pairs can be used to represent list of objects of many types. Each list is normally assumed to contain objects of just ONE type.

Let’s see how this would work.

The EMPTY list:

Node *L;

L = NULL;


L


The Head Function: for potential generality, C requires we return a pointer…

NodeInfo *Head(Node *L)

{ if (L == NULL) return(error);

else return(L->Info); // this is now a pointer // to an Info structure

}

L

info1

Info Link ….

Head(L)


The Tail function:

Node *Tail(Node *L)


else return(L->Link);

}

L

info1

Info Link ….

Tail(L)

Info Link

info2


The Constructor function: how do we handle the incoming information? We will share structures, so DON’T make a copy.

Node *Cons(Info *info, Node *L)

{

Node *N;

N = (Node *)malloc(sizeof(Node));

N->Link = L; // Connect to existing front

N->Info = info; // Connect to the information

return(N);

}


A picture, to firm up our ideas. Start with the empty list.

L info1

L

info1

Info Link

This is the original

L = Cons(&info1, L);

L

info1

Info Link


Add a second element: L = Cons(&info2, L);

info2

L

info1

Info Link

info2

Info Link

Etc….


The call L1 = Cons(&info2, L); would have resulted in the picture:

L1

info1

Info Link

info2

Info Link

L

Both the OLD and the NEW list are accessible.


The textbook List Reversal function destroyed the original list. This approach allows us to reverse a list while leaving the original unchanged, and WITHOUT making copies of the information stored in it:

Node *Reverse(Node* L)

{

return(AuxReverse(L, NULL));

}

Node *AuxReverse(Node *L, Node *N)

{

if (L == NULL) return(N);

else return(AuxReverse(Tail(L),Cons(Head(L), N)));

}


L1 = Reverse(L);

L

L1


Although AuxReverse is Tail-Recursive (and can thus run in constant space), here is a fully iterative version of Reverse:

Node *Reverse(Node* L)

{ Node *rev;

rev = NULL;

while (L != NULL) {

rev = Cons(Head(L), rev);

L = Tail(L);

}

return(rev);

}


Appending two lists, while leaving the originals unchanged:

Node *Append(Node *L1, Node *L2)

{ if (L1 == NULL) return(L2);

else return(Cons(Head(L1), Append(Tail(L1), L2)));

}

L1 L2

L3

L3 = Append(L1, L2);


Using Append, we can implement a different reversal:

Node *Reverse(Node *L)


else return(Append(Reverse(Tail(L)),

Cons(Head(L), NULL)));

}

Call: L1 = Reverse(L);

UNDESIRABLE: WHY? Too many LOST nodes… Too many intermediate lists that are constructed and then irretrievably forgotten: C does NOT support automatic garbage collection, but this is NOT the worst problem. Even if it did, the algorithm would take too long to run with long lists...


Some Comments on Infinite Regress.

One of the problems with all repetition constructs (loops and recursion) is the statement of the termination condition and the guarantee that the condition will occur after a finite number of repetitions.

A for construct is normally used as a counting loop: some index starts at some value; the index is checked against some condition (usually an inequality); the index is incremented according to some rule.

A while construct checks the value of some boolean expression before executing the body, within which the value of the expression is modified.


A do … while construct performs the check after execution of the body.

It should be clear that in all of these repetition constructs there is a possibility of error that will lead to the repetition never terminating.

Recursion has the same problems:

A) at entry a check is performed to detect termination;

B) if the check fails, the rest of the body is executed, which will result in a recursive call to the same function with somewhat different parameter values.

There are at least two distinct ways in which this scheme will lead to “infinite regress” - equivalent to “infinite looping”.


The first one is: the termination condition is never satisfied. Ex.:

long factorial(long n){

if (n == 0) return 1;

else return(n*factorial(n-1));

}

Call: factorial(-1);

Symptoms: program runs out of memory or continues running forever… (in this case, it will run out of memory).

Solution: make sure all inputs to the function are accounted for and lead to termination.


The second is: the parameters are not changed from one recursive call to the next.

long SumInts(long low, long high) {

long mid=(low + high)/2;

if (low > high) return(0);

else

return(SumInts(low, mid) +

SumInts(mid+1,high));

}

Call: SumInts(2,2);

Solution: make sure the parameters change from one call to the next.

Documents

19.102 - Computing II Recursion: How to solve a problem by doing some work and sending the rest out to be handled the same way. Many mathematical functions