Lecture 5 Principles of Functional Programming Summer...

Lecture 5Principles of Functional Programming

Summer 2020

Asymptotic Analysis

The benefits of tail recursion,recurrences and big-O, list sorting

Section 1

Tail Recursion II: Reverse!

Jacob Neumann Asymptotic Analysis 26 May 2020 2 / 48

Tail Recursion

Recall fact and tfact :

fact : int -> int

REQUIRES: n ≥ 0

ENSURES: fact n ∼= n!

1 fun fact 0 = 1

2 | fact (n:int):int = n * fact(n-1)

Tail Recursion

Recall fact and tfact :

tfact : int * int -> int

REQUIRES: n ≥ 0

ENSURES: tfact(n,acc) ∼= acc * fact(n)

1 fun tfact (0:int ,acc:int):int = acc

2 | tfact(n,acc) = tfact(n-1,n*acc)

Lots of pending computations

fact 8 =⇒ 8 * fact (7)

=⇒ 8 * (7 * fact (6))

=⇒ 8 * (7 * (6 * fact (5)))

=⇒ 8 * (7 * (6 * (5 * fact (4))))

=⇒ 8 * (7 * (6 * (5 * (4 * fact (3)))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * fact (2))))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * (2 * fact (1)))))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * (2 * (1 * fact(0

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * (2 * (1 * 1)))))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * (2 * 1))))))

=⇒ 8 * (7 * (6 * (5 * (4 * (3 * 2)))))

=⇒ 8 * (7 * (6 * (5 * (4 * 6))))

=⇒ 8 * (7 * (6 * (5 * 24)))

=⇒ 8 * (7 * (6 * 120))

=⇒ 8 * (7 * 720)

=⇒ 8 * 5040

=⇒ 40320

Much nicer

fact ’ 8 =⇒ tfact (8,1)

=⇒ tfact (7,8)

=⇒ tfact (6,56)

=⇒ tfact (5 ,336)

=⇒ tfact (4 ,1680)

=⇒ tfact (3 ,6720)

=⇒ tfact (2 ,20160)

=⇒ tfact (1 ,40320)

=⇒ tfact (0 ,40320)

=⇒ 40320

Equivalence

Theorem

For all values n:int with n ≥ 0 and all values acc:int,

acc * (fact n) ∼= tfact(n,acc)

General notes about this example

Anything (in this case, acc) thatchanges in the recursive call (besidesthe variable we’re inducting on) needsto be quantified in the IH.

Making the problem more generalmade it easier

We can generalize the problem byadding extra “accumulator” inputs,and later instantiating them to startervalues

Tail recursion: After “calling itself”,tfact doesn’t “do anything with theresult”, but just returns it. So all thecomputation is happening in thearguments. Here, tail recursion savedus space. Next we’ll see it can save ustime.

Naıve reverse

rev : int list -> int list

REQUIRES: trueENSURES: rev L evaluates to a list containing exactly theelements of L, in the opposite order they appeared in L

1 fun rev ([]: int list):int list = []

2 | rev (x::xs) = (rev xs)@[x]

4 val [3,2,1] = rev [1,2,3]

5 val [] = rev []

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::(9::(8:

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::(9::([8

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::(10::([9 ,8 ,7

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::(11::([10 ,9 ,8 ,7 ,6

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::(12::([11 ,10 ,9 ,8 ,7 ,6 ,5

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::(13::([12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::(14::([13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::(15::([14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::(16::([15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::(17::([16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))

=⇒ (25::(24::(23::(22::(21::(20::(19::(18::([17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))))

=⇒ (25::(24::(23::(22::(21::(20::(19::([18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))))))

=⇒ (25::(24::(23::(22::(21::(20::([19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))))))@[

=⇒ (25::(24::(23::(22::(21::([20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2]))))))@[1]

=⇒ (25::(24::(23::(22::([21 ,20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))))@[1]

=⇒ (25::(24::(23::([22 ,21 ,20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2]))))@[1]

=⇒ (25::(24::([23 ,22 ,21 ,20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2])))@[1]

=⇒ (25::([24 ,23,22 ,21,20,19 ,18,17,16 ,15,14,13 ,12,11,10,9,8,7,6,5,4,3,2]))@[1]

=⇒ ([25,24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2])@[1]

=⇒ 25::([24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2]@[1])

=⇒ 25::(24::([23 ,22 ,21 ,20 ,19 ,18 ,17 ,16 ,15 ,14 ,13 ,12 ,11 ,10 ,9 ,8 ,7 ,6 ,5 ,4 ,3 ,2]@[1]))

Tail reverse

trev : int list * int list -> int list

REQUIRES: trueENSURES: trev(L,acc) ∼= (rev L)@acc

1 fun trev ([]: int list ,acc:int list) = acc

2 | trev (x::xs ,acc):int list = trev(xs,x::acc

4 val [3,2,1,∼7] = trev ([1,2,3],[∼7])

trev ([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[∼7])

=⇒ trev ([2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[1,∼7])

=⇒ trev ([3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[2,1,∼7])

=⇒ trev ([4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[3,2,1,∼7])

=⇒ trev ([5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[4,3,2,1,∼7])

=⇒ trev ([6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[5,4,3,2,1,∼7])

=⇒ trev ([7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[6,5,4,3,2,1,∼7])

=⇒ trev ([8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[7,6,5,4,3,2,1,∼7])

=⇒ trev ([9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([11,12,13,14,15,16,17,18,19,20,21,22,23,24,25],[10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([12,13,14,15,16,17,18,19,20,21,22,23,24,25],[11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([13,14,15,16,17,18,19,20,21,22,23,24,25],[12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([14,15,16,17,18,19,20,21,22,23,24,25],[13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([15,16,17,18,19,20,21,22,23,24,25],[14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([16,17,18,19,20,21,22,23,24,25],[15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([17,18,19,20,21,22,23,24,25],[16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([18,19,20,21,22,23,24,25],[17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([19,20,21,22,23,24,25],[18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([20,21,22,23,24,25],[19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([21,22,23,24,25],[20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([22,23,24,25],[21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([23,24,25],[22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([24,25],[23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([25],[24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ trev ([],[25,24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7])

=⇒ [25,24,23,22,21,20,19,18,17,16,15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,∼7]

Exercise

Prove that for all values L and acc of type int list ,

trev(L,acc) ∼= (rev L) @ acc

Section 2

Work Analysis

What we want

For a given function f, we want to know how long (f v) takes toevaluate to a value, for each v such that (f v) is valuable.

v1 f v1 =⇒13 v1 ’

v2 f v1 =⇒1 v2 ’

v3 f v1 =⇒96000 v3’

v4 f v1 =⇒115 v4 ’

In general, for each v such that (f v) is valuable, we want to knowthe least n such that

f v =⇒n v’

for some value v’.

A few problems with this. . .

Internal representations?

6+6 =⇒? 12

Hardware dependence: want algorithm analysis to be the same forboth our computers, even if yours is twice as fast as mine.

For small inputs, the costs of initiating the process and storingvariables is much higher relative to the costs of actual computation

Solution: Asymptotic Analysis

We instead generally assume large inputs, and instead seek to classifywhat inpact doubling, tripling, etc. the size of the input has on thecomputation time.

Wf : N→ N: n 7→ (the number of steps it takes to evaluate

(f v) if v is some input of size n)

This assumes we have a well-defined notion of size defined on the inputtype of f, such that (f v) and (f v’) take the same number ofsteps to evaluate whenever v and v’ are of the same size.

We can classify such functions with big-O

The big-O complexity of a (mathematical) function W : N→ N tells ushow fast W grows, proportionally to its input:Rough idea: Let W, g : N→ N. We say W is O(g) if there’s someconstant c > 0 such that

W (n) ≤ cg(n) for sufficiently large n

A bound is tight if there’s no tighter bound which suffices. For theexamples encountered in this class, it should be clear whether a bound istight or not.Example: W (n) = 3n2 + 4n+ 2. This function is O(n4) and O(n3),but a tight bound is O(n2).

Some common big-O classes

If W is O(log n), then quadrupling the size of the input adds 2(units of time) to the runtime.

W (4n) ≈W (n) + 2

If W is O(n), then quadrupling the size of the input approximatelyquadruples the runtime:

W (4n) ≈ 4W (n)

Some common big-O classes

If W is O(n log n), then quadrupling the size of the input scales andincrements the runtime:

W (4n) ≈ 4(W (n) + 2)

If W is O(n2), then quadrupling the input size multiplies theruntime by 16.

W (3n) ≈ 9W (n) W (4n) ≈ 16W (n) W (5n) ≈ 25W (n)

If W is O(2n), then doubling the input size squares the output size,and tripling cubes the runtime.

W (2n) ≈ (W (n))2 W (3n) ≈ (W (n))3

Section 3

Work Analysis of Recursive Functions

The Tree Method

0 How you’re quantifying input size

1 Recurrence

2 Description of work tree

3 Measurements of work tree (height, and width at each level)

4 Summation

5 Big-O

Example: append

1 fun ([]: int list) @ (L:int list) = L

2 | (x::xs) @ L = x::( xs@L)

0 Measure of input size: length of the first list

1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

2 Work Tree

1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

2 Work Tree

1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

2 Work Tree

1 Recurrence:

W (0) = k0

W (n) = k1 +W (n− 1)

2 Work Tree

3 MeasurementsHeight: nWork on the i-th level: k1

4 Sum:

W (n) ≈ k0 +

n∑i=0

5 Big O:W (n) is O(n)

Example: rev

1 fun rev ([]: int list):int list = []

2 | rev (x::xs) = (rev xs)@[x]

4 val [3,2,1] = rev [1,2,3]

5 val [] = rev []

0 Measure of input size: length of the input list

1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

2 Work Tree

1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

2 Work Tree

1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

2 Work Tree

1 Recurrence:

W (0) = k0

W (n) = k1 + k2n+W (n− 1)

2 Work Tree

3 MeasurementsHeight: nWork on the i-th level: k1 + k2(n− i)

4 Sum:

W (n) ≈n∑

(k1 + k2(n− i))

= k1n+ k2

n∑i=0

(n− i)

= k1n+ k2

n∑i=0

= k1k + k2n(n− 1)

5 Big O:W (n) is O(n2)

Exercises

Show that fact is O(n)

Show that tfact is O(n)

Show that trev is O(n) for any acc

1 fun ([]: int list) @ (L:int list) = L

2 | (x::xs) @ L = x::( xs@L)

Section 4

Asymptotic Analysis of Multi-Step Algorithms

Sorting

Sorting is a classic algorithmic problem in computer science: finding thefastest way to put all the elements of a list in order.

A value [x_1 ,...,x_n] : int list is sorted if for eachi = 1, . . . , n− 1, Int.compare(x_i ,x_(i+1)) 6∼= GREATER .

Or, recursively: a value v:int list is sorted if either v=[] orv=[x] for some x, or v=x::x’::xs whereInt.compare(x,x’) 6∼= GREATER and x’::xs is sorted.

1 fun isSorted ([]: int list):bool = true

2 | isSorted [x] = true

3 | isSorted (x::x’::xs) = (x<=x’) andalso

isSorted(x’::xs)

sort : int list -> int list

REQUIRES: trueENSURES: sort(L) evaluates to a sorted permutation of L

A “permutation” of L is just a list that contains the same elements thesame number of times as L, just in a possibly different order. So[1,1,2,3] is a permutation of [3,1,2,1] but not of [3,2,1].

Sorting Algorithms

There are many sorting algorithms: insertion sort, quick sort, merge sort,bubble sort, . . .

We’ll be focusing on merge sort, which consists of the following threesteps:

1 Split the input list in half

2 Sort each half

3 merge the sorted halves together to obtain a sorted whole

split : int list -> int list * int list

REQUIRES: trueENSURES: split L evaluates to (A,B) where A and B differ inlength by at most one, and A@B is a permutation of L

merge : int list * int list -> int list

REQUIRES: A and B are sortedENSURES: merge(A,B) evaluates to a sorted permutation of A@B

msort : int list -> int list

REQUIRES: trueENSURES: msort(L) evaluates to a sorted permutation of L

split : int list -> int list * int list

REQUIRES: trueENSURES: split L evaluates to (A,B) where A and B differ inlength by at most one, and A@B is a permutation of L

1 fun split ([]: int list):int list * int list =

([] ,[])

2 | split [x] = ([x],[])

3 | split (x::x’::xs) =

5 val (A,B) = split xs

7 (x::A,x’::B)

merge : int list * int list -> int list

REQUIRES: A and B are sortedENSURES: merge(A,B) evaluates to a sorted permutation of A@B

1 fun merge (L1:int list ,[]: int list):int list =

2 | merge ([],L2) = L2

3 | merge (x::xs ,y::ys) =

4 (case Int.compare(x,y) of

5 GREATER => y:: merge(x::xs ,ys)

6 | _ => x:: merge(xs ,y::ys))

msort : int list -> int list

REQUIRES: trueENSURES: msort(L) evaluates to a sorted permutation of L

1 fun msort ([]: int list):int list = []

2 | msort [x] = [x]

3 | msort L =

5 val (A,B) = split L

7 merge(msort A,msort B)

Analysis

1 fun isSorted ([]: int list):bool = true

2 | isSorted [x] = true

3 | isSorted (x::x’::xs) = (x<=x’) andalso

isSorted(x’::xs)

0 Measure of size: length of input list

Wsplit(0) = k0

Wsplit(1) = k1

Wsplit(n) = k2 +Wsplit(n− 2)

2-4 . . .

1 Wsplit(n) is O(n)

Analysis

1 fun split ([]: int list):int list * int list =

([] ,[])

2 | split [x] = ([x],[])

3 | split (x::x’::xs) =

5 val (A,B) = split xs

7 (x::A,x’::B)

0 Measure of size: sum of lengths of input lists1

Wmerge(0) = k0

Wmerge(n) ≤ k1 +Wmerge(n− 1)

2-4 . . .1 Wmerge(n) is O(n)

Analysis

1 fun merge (L1:int list ,[]: int list):int list =

2 | merge ([],L2) = L2

3 | merge (x::xs ,y::ys) =

4 (case Int.compare(x,y) of

5 GREATER => y:: merge(x::xs ,ys)

6 | _ => x:: merge(xs ,y::ys))

0 Measure of size: length of input list

Lecture 5 Principles of Functional Programming Summer...

Documents

Principles and Practice in Programming …csci3155.cs.colorado.edu/s18/csci3155-notes.pdfprogramming paradigms that persist (e.g., imperative programming and functional programming)

Introduction to Functional Programming and ML CS 331 Principles of Programming Languages

Functional Programming I * Functional Programming and Interactive Theorem …csulrich/ftp/fp1/fp1-print.pdf · 2015-05-05 · Functional Programming I * Functional Programming

Build Functional Reactive - developermarch.com · Functional Reactive Programming (FRP) principles. Functional Reactive Programming is Abstract. defining FRP ... Javascript libraries

Introduction to Functional Programming · Introduction to Functional Programming 5 Functional programming In the functional programming style, the computational task to be programmed

Functional programming principles and Java 8

Functional Programming Principles in Scalaansuman/sdv/ScalaTut.pdf · 2013-11-14 · Functional Programming Languages In a restricted sense, a functional programming language is one

*3mm Principles of Programming Languages COMP3031: Functional Programming …dekai/3031/lectures/sml/h.sml.pdf · 2012-11-28 · Principles of Programming Languages COMP3031: Functional

Functional programming principles

Principles ofProgramming Languages · 2016. 9. 9. · principles of programming languages 13 Course Overview We will begin our study of functional programming with Racket, a di-alect

Functional programming from first principles

Functional Programming - O'Reilly Mediaassets.en.oreilly.com/1/event/80/Learn Functional Programming by... · Functional Programming ... Let’s turn back to our functional bag of

Functional Programming Principles in Scala · Functional Programming Languages In a restricted sense, a functional programming language is one which does not have mutable variables,

Functional Programming in C+++.pdfRainer Grimm Functional programming? • Functional programming is programming with mathematical functions. • Mathematical functions are functions

Functional Programming Principles in Scalaansuman/sdv/ScalaTut.pdf · In a wider sense, a functional programming language enables the construction of elegant programs that focus on

Functional Programming - static.igstan.rostatic.igstan.ro/functional-programming.pdf · Functional Programming what why how. Functional Programming what why how. What is FP a programming

Principles of Programming Languages 2017ppl172/wiki.files/class/book/ppl17_chapter… · Functional Programming(FP) is a paradigm of programming that is most similar to evaluation

Principles of Programming Languages Declarative ...arusoaie.andrei/... · Principles of Programming Languages Declarative programming: functional programming Andrei Arusoaie1 1Department

Functional Reactive Programming · Functional Reactive! Programming (FRP) 1997 – Conal Elliott, Paul Hudak: “Functional Reactive Animation”!! functional reactive programming!

Functional Programming Element of Functional Programming