Lecture 5 Principles of Functional Programming Summer 202015150/resources/lectures/5/lecture5.pdf · 2020-05-28 · Lecture 5 Principles of Functional Programming Summer 2020 Asymptotic

Lecture 5Principles of Functional Programming

Summer 2020

Asymptotic Analysis

The benefits of tail recursion,recurrences and big-O, list sorting

Section 1

Tail Recursion II: Reverse!

Jacob Neumann Asymptotic Analysis 26 May 2020 2 / 48

Tail Recursion

Recall fact and tfact :

fact : int -> int

REQUIRES: n ≥ 0

ENSURES: fact n ∼= n!

5.0

1 fun fact 0 = 1

2 | fact (n:int):int = n * fact(n-1)


Tail Recursion

Recall fact and tfact :

tfact : int * int -> int

REQUIRES: n ≥ 0

ENSURES: tfact(n,acc) ∼= acc * fact(n)

5.1

1 fun tfact (0:int ,acc:int):int = acc

2 | tfact(n,acc) = tfact(n-1,n*acc)


Lots of pending computations

fact 8 =⇒ 8 * fact (7)

=⇒ 8 * (7 * fact (6))

=⇒ 8 * (7 * (6 * fact (5)))

=⇒ 8 * (7 * (6 * (5 * fact (4))))

=⇒ 8 * (7 * (6 * (5 * (4 * fact (3)))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * fact (2))))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * (2 * fact (1)))))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * (2 * (1 * fact(0

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * (2 * (1 * 1)))))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * (2 * 1))))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * 2)))))

=⇒ 8 * (7 * (6 * (5 * (4 * 6))))

=⇒ 8 * (7 * (6 * (5 * 24)))

=⇒ 8 * (7 * (6 * 120))

=⇒ 8 * (7 * 720)

=⇒ 8 * 5040

=⇒ 40320


Much nicer

fact ’ 8 =⇒ tfact (8,1)

=⇒ tfact (7,8)

=⇒ tfact (6,56)

=⇒ tfact (5 ,336)

=⇒ tfact (4 ,1680)

=⇒ tfact (3 ,6720)

=⇒ tfact (2 ,20160)

=⇒ tfact (1 ,40320)

=⇒ tfact (0 ,40320)

=⇒ 40320


Equivalence

Theorem

For all values n:int with n ≥ 0 and all values acc:int,

acc * (fact n) ∼= tfact(n,acc)


General notes about this example

Anything (in this case, acc) thatchanges in the recursive call (besidesthe variable we’re inducting on) needsto be quantified in the IH.

Making the problem more generalmade it easier

We can generalize the problem byadding extra “accumulator” inputs,and later instantiating them to startervalues

Tail recursion: After “calling itself”,tfact doesn’t “do anything with theresult”, but just returns it. So all thecomputation is happening in thearguments. Here, tail recursion savedus space. Next we’ll see it can save ustime.


Naıve reverse

rev : int list -> int list

REQUIRES: trueENSURES: rev L evaluates to a list containing exactly theelements of L, in the opposite order they appeared in L

5.2

1 fun rev ([]: int list):int list = []

2 | rev (x::xs) = (rev xs)@[x]

3

4 val [3,2,1] = rev [1,2,3]

5 val [] = rev []


.

.

.

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::(9::(8:

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::(9::(8:

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::(9::(8:

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::(9::(8:

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::(9::(8:

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::(9::([8

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::([9 ,8 ,7

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::([10 ,9 ,8 ,7 ,6

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::([11 ,10 ,9 ,8 ,7 ,6 ,5

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::([12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::([13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::([14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::([15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::([16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::([17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))))

=⇒ (25::(24::(23::(22::(21::(20::(19::([18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))))))

=⇒ (25::(24::(23::(22::(21::(20::([19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))))))@[

=⇒ (25::(24::(23::(22::(21::([20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2]))))))@[1]

=⇒ (25::(24::(23::(22::([21 ,20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))))@[1]

=⇒ (25::(24::(23::([22 ,21 ,20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2]))))@[1]

=⇒ (25::(24::([23 ,22 ,21 ,20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))@[1]

=⇒ (25::([24 ,23,22 ,21,20,19 ,18,17,16 ,15,14,13 ,12,11,10,9,8,7,6,5,4,3,2]))@[1]

=⇒ ([25,24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2])@[1]

=⇒ 25::([24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2]@[1])

=⇒ 25::(24::([23 ,22 ,21 ,20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2]@[1]))

.

.

.

Tail reverse

trev : int list * int list -> int list

REQUIRES: trueENSURES: trev(L,acc) ∼= (rev L)@acc

5.3

1 fun trev ([]: int list ,acc:int list) = acc

2 | trev (x::xs ,acc):int list = trev(xs,x::acc

)

3

4 val [3,2,1,∼7] = trev ([1,2,3],[∼7])


trev ([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[∼7])

=⇒ trev ([2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[1,∼7])

=⇒ trev ([3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[2,1,∼7])

=⇒ trev ([4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[3,2,1,∼7])

=⇒ trev ([5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[4,3,2,1,∼7])

=⇒ trev ([6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[5,4,3,2,1,∼7])

=⇒ trev ([7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[6,5,4,3,2,1,∼7])

=⇒ trev ([8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[7,6,5,4,3,2,1,∼7])

=⇒ trev ([9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([12,13,14,15,16,17,18,19,20,21,22,23,24,25],[11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([13,14,15,16,17,18,19,20,21,22,23,24,25],[12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([14,15,16,17,18,19,20,21,22,23,24,25],[13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([15,16,17,18,19,20,21,22,23,24,25],[14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([16,17,18,19,20,21,22,23,24,25],[15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([17,18,19,20,21,22,23,24,25],[16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([18,19,20,21,22,23,24,25],[17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([19,20,21,22,23,24,25],[18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([20,21,22,23,24,25],[19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([21,22,23,24,25],[20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([22,23,24,25],[21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([23,24,25],[22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([24,25],[23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([25],[24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([],[25,24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ [25,24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7]

Exercise

Prove that for all values L and acc of type int list ,

trev(L,acc) ∼= (rev L) @ acc


Section 2

Work Analysis


What we want

For a given function f, we want to know how long (f v) takes toevaluate to a value, for each v such that (f v) is valuable.

v1 f v1 =⇒13 v1 ’

v2 f v1 =⇒1 v2 ’

v3 f v1 =⇒96000 v3’

v4 f v1 =⇒115 v4 ’

In general, for each v such that (f v) is valuable, we want to knowthe least n such that

f v =⇒n v’

for some value v’.


A few problems with this. . .

Internal representations?

6+6 =⇒? 12

Hardware dependence: want algorithm analysis to be the same forboth our computers, even if yours is twice as fast as mine.

For small inputs, the costs of initiating the process and storingvariables is much higher relative to the costs of actual computation


Solution: Asymptotic Analysis

We instead generally assume large inputs, and instead seek to classifywhat inpact doubling, tripling, etc. the size of the input has on thecomputation time.

Wf : N→ N: n 7→ (the number of steps it takes to evaluate

(f v) if v is some input of size n)

This assumes we have a well-defined notion of size defined on the inputtype of f, such that (f v) and (f v’) take the same number ofsteps to evaluate whenever v and v’ are of the same size.


We can classify such functions with big-O

The big-O complexity of a (mathematical) function W : N→ N tells ushow fast W grows, proportionally to its input:Rough idea: Let W, g : N→ N. We say W is O(g) if there’s someconstant c > 0 such that

W (n) ≤ cg(n) for sufficiently large n

A bound is tight if there’s no tighter bound which suffices. For theexamples encountered in this class, it should be clear whether a bound istight or not.Example: W (n) = 3n2 + 4n+ 2. This function is O(n4) and O(n3),but a tight bound is O(n2).


Some common big-O classes

If W is O(log n), then quadrupling the size of the input adds 2(units of time) to the runtime.

W (4n) ≈W (n) + 2

If W is O(n), then quadrupling the size of the input approximatelyquadruples the runtime:

W (4n) ≈ 4W (n)


Some common big-O classes

If W is O(n log n), then quadrupling the size of the input scales andincrements the runtime:

W (4n) ≈ 4(W (n) + 2)

If W is O(n2), then quadrupling the input size multiplies theruntime by 16.

W (3n) ≈ 9W (n) W (4n) ≈ 16W (n) W (5n) ≈ 25W (n)

If W is O(2n), then doubling the input size squares the output size,and tripling cubes the runtime.

W (2n) ≈ (W (n))2 W (3n) ≈ (W (n))3


Section 3

Work Analysis of Recursive Functions


The Tree Method

0 How you’re quantifying input size

1 Recurrence

2 Description of work tree

3 Measurements of work tree (height, and width at each level)

4 Summation

5 Big-O


Example: append

5.4

1 fun ([]: int list) @ (L:int list) = L

2 | (x::xs) @ L = x::( xs@L)

0 Measure of input size: length of the first list

1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)


1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

2 Work Tree


1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

2 Work Tree


1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

2 Work Tree


1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

2 Work Tree


2 Work Tree

3 MeasurementsHeight: nWork on the i-th level: k1


3 MeasurementsHeight: nWork on the i-th level: k1

4 Sum:

W (n) ≈ k0 +

n∑i=0

k1

5 Big O:W (n) is O(n)


Example: rev

5.2

1 fun rev ([]: int list):int list = []

2 | rev (x::xs) = (rev xs)@[x]

3

4 val [3,2,1] = rev [1,2,3]

5 val [] = rev []

0 Measure of input size: length of the input list

1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)


1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

2 Work Tree


1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

2 Work Tree


1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

2 Work Tree


1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

2 Work Tree


2 Work Tree

3 MeasurementsHeight: nWork on the i-th level: k1 + k2(n− i)


3 MeasurementsHeight: nWork on the i-th level: k1 + k2(n− i)

4 Sum:

W (n) ≈n∑

i=0

(k1 + k2(n− i))

= k1n+ k2

n∑i=0

(n− i)

= k1n+ k2

n∑i=0

i

= k1k + k2n(n− 1)

2

5 Big O:W (n) is O(n2)


Exercises

Show that fact is O(n)

Show that tfact is O(n)

Show that trev is O(n) for any acc

5.4

1 fun ([]: int list) @ (L:int list) = L

2 | (x::xs) @ L = x::( xs@L)


Section 4

Asymptotic Analysis of Multi-Step Algorithms


Sorting

Sorting is a classic algorithmic problem in computer science: finding thefastest way to put all the elements of a list in order.

A value [x_1 ,...,x_n] : int list is sorted if for eachi = 1, . . . , n− 1, Int.compare(x_i ,x_(i+1)) 6∼= GREATER .

Or, recursively: a value v:int list is sorted if either v=[] orv=[x] for some x, or v=x::x’::xs whereInt.compare(x,x’) 6∼= GREATER and x’::xs is sorted.


Spec

5.5

1 fun isSorted ([]: int list):bool = true

2 | isSorted [x] = true

3 | isSorted (x::x’::xs) = (x<=x’) andalso

isSorted(x’::xs)

sort : int list -> int list

REQUIRES: trueENSURES: sort(L) evaluates to a sorted permutation of L

A “permutation” of L is just a list that contains the same elements thesame number of times as L, just in a possibly different order. So[1,1,2,3] is a permutation of [3,1,2,1] but not of [3,2,1].


Sorting Algorithms

There are many sorting algorithms: insertion sort, quick sort, merge sort,bubble sort, . . .

We’ll be focusing on merge sort, which consists of the following threesteps:

1 Split the input list in half

2 Sort each half

3 merge the sorted halves together to obtain a sorted whole


Specs

split : int list -> int list * int list

REQUIRES: trueENSURES: split L evaluates to (A,B) where A and B differ inlength by at most one, and A@B is a permutation of L

merge : int list * int list -> int list

REQUIRES: A and B are sortedENSURES: merge(A,B) evaluates to a sorted permutation of A@B

msort : int list -> int list

REQUIRES: trueENSURES: msort(L) evaluates to a sorted permutation of L


split

split : int list -> int list * int list

REQUIRES: trueENSURES: split L evaluates to (A,B) where A and B differ inlength by at most one, and A@B is a permutation of L

5.6

1 fun split ([]: int list):int list * int list =

([] ,[])

2 | split [x] = ([x],[])

3 | split (x::x’::xs) =

4 let

5 val (A,B) = split xs

6 in

7 (x::A,x’::B)

8 end


merge

merge : int list * int list -> int list

REQUIRES: A and B are sortedENSURES: merge(A,B) evaluates to a sorted permutation of A@B

5.7

1 fun merge (L1:int list ,[]: int list):int list =

L1

2 | merge ([],L2) = L2

3 | merge (x::xs ,y::ys) =

4 (case Int.compare(x,y) of

5 GREATER => y:: merge(x::xs ,ys)

6 | _ => x:: merge(xs ,y::ys))


msort

msort : int list -> int list

REQUIRES: trueENSURES: msort(L) evaluates to a sorted permutation of L

5.8

1 fun msort ([]: int list):int list = []

2 | msort [x] = [x]

3 | msort L =

4 let

5 val (A,B) = split L

6 in

7 merge(msort A,msort B)

8 end


Analysis

5.5

1 fun isSorted ([]: int list):bool = true

2 | isSorted [x] = true

3 | isSorted (x::x’::xs) = (x<=x’) andalso

isSorted(x’::xs)

0 Measure of size: length of input list

1

Wsplit(0) = k0

Wsplit(1) = k1

Wsplit(n) = k2 +Wsplit(n− 2)

2-4 . . .

1 Wsplit(n) is O(n)


Analysis

5.6

1 fun split ([]: int list):int list * int list =

([] ,[])

2 | split [x] = ([x],[])

3 | split (x::x’::xs) =

4 let

5 val (A,B) = split xs

6 in

7 (x::A,x’::B)

8 end

0 Measure of size: sum of lengths of input lists1

Wmerge(0) = k0

Wmerge(n) ≤ k1 +Wmerge(n− 1)

2-4 . . .1 Wmerge(n) is O(n)


Analysis

5.7

1 fun merge (L1:int list ,[]: int list):int list =

L1

2 | merge ([],L2) = L2

3 | merge (x::xs ,y::ys) =

4 (case Int.compare(x,y) of

5 GREATER => y:: merge(x::xs ,ys)

6 | _ => x:: merge(xs ,y::ys))

0 Measure of size: length of input list

1


Documents

Lecture 5 Principles of Functional Programming Summer 202015150/resources/lectures/5/lecture5.pdf · 2020-05-28 · Lecture 5 Principles of Functional Programming Summer 2020 Asymptotic