Functional Programming with Immutable Data Structures

Functional Programming withImmutable Data Structures

Why Imperative Languages are

Fundamentally Broken in a

Multi-Threaded Environment

Ivar Thorson

Italian Institute of Technology

November 2, 2010

Ivar Thorson

Research interests:

Compliant actuation

Hopping Robots

Rigid Body Dynamics Simulations

The Next 37 minutes:

Important abstractions of functional

programming

particularly for a 4-year old language

Rich Hickey

What I have learned from his work

Clojure: Blending theoretical abstractions +

practical know-how

Target Audience: C, C++, Java, Matlab

Programmers

Tempting to start with a feature tour!

But you won’t understand why Clojure is

cool without context

Actually, I’m going to try to knock the cup

out of your hand.

Three Outdated Concepts

1. Variables

2. Syntax

3. Object Orientation


1. Variables

2. Syntax



1. Variables

2. Syntax



1. Variables

2. Syntax


Goal is to convince you that

1. shared mutable data is now a

philosophically bankrupt idea.

2. code and data should be structured as

trees

3. OOP isn’t the best way to achieve

OOP’s goals





trees


OOP’s goals





trees


OOP’s goals





trees


OOP’s goals

Speaking bluntly

1. everything you know is wrong (15 min)

2. lisp parentheses are better than syntax

(10 min)

3. OOP inheritance sucks (5 min)

Speaking bluntly



(10 min)


Speaking bluntly



(10 min)


Speaking bluntly



(10 min)


disclaimer: some hyperbole in previous

statements

Oh noes, too many parentheses!

Good reasons for parentheses

1. Lisp is homoiconic

2. Parentheses uniquely define tree-shaped

computations

3. Parentheses enable structural editing




computations





computations





computations


For now, please be patient

Introduction: Motivation for multi-threaded

programming

1. Last 40 years: Moore’s Law

2. “Transistor count will double every 2

years”



years”



years”

# of transistors ≈ CPU performance

Constraining physical relationship between

power density, swiching time, oxide thickness

The future of hardware is increasingly parallel

The future of software will be ruled by

Amdahl’s law

Some things are sequential: Two women

cannot have a baby in 4.5 months.

1. Dividing up work is already a hard

design task

2. Resource contention makes this problem

harder


design task


harder


design task


harder

Common multi-threaded bugs:

I Invalid state

I Race conditions

I Deadlocks

I Livelocks

I Resource starvation


I Invalid state

I Race conditions

I Deadlocks

I Livelocks



I Invalid state

I Race conditions

I Deadlocks

I Livelocks



I Invalid state

I Race conditions

I Deadlocks

I Livelocks



I Invalid state

I Race conditions

I Deadlocks

I Livelocks



I Invalid state

I Race conditions

I Deadlocks

I Livelocks


What if most bugs were merely due to the

imperative programming model?

Part 1: The Functional Programming Style

Pure functions always return the same result

when they get the same input.

Pure functions don’t...

I ...look outside their box

I ...modify anything, anywhere

I ...print messages to the user

I ...write to disk





I ...write to disk





I ...write to disk





I ...write to disk





I ...write to disk

Pure functions have no side effects

Nothing would change if you ran the function

again – anywhere!

f (x) = x2 + 1

Pure functions just return a value, and do

nothing more.

Pure functions compose to other pure

functions

f (a, b, c) = (a + b)/(c ∗ 2)

Languages that emphasize the use of pure

functions are called functional languages

Imperative languages describe computation

in terms of changes to state.

C, C++, Java, and most engineering

languages are imperative.

Imperative languages describe memory

operations instead of purely functional

operations.

Imperative style: directly causing side effects

on memory.

The assignment operator changes

memory...often a side effect!

Could you write a C program...

I without any non-local variables?

I where = is only used for initialization?







Next: Variables are fundamentally a bad

abstraction in multithreaded environments.

Claim #1. Shared mutable data is now

philosophically bankrupt

I x = x + 1

I x = 3 (...last time I checked!)

I In my universe, 3 = 3 + 1 is never true

I x = x + 1



I x = x + 1



I x = x + 1



The Big Problem: the concept of variables

encourage us to forget about time.

x [t] = x0

x [t + 1] = x [t] + 1

The value of x for a given t is immutable

and unchanging!

The Most Important Slide

I x is a name, an identity of a sequence of

values

I x has different values at different times

I The values of x are related by pure

functions

I In this case, by the increment function



values



functions




values



functions




values



functions




values



functions


The idea of a variable confuses identity and

the most current value!

Locking: a tactic for winning a battle.

What we need is a strategy to win the war.

“What if all data was immutable?” – Rich

Hickey (not the first one to ask this question)

Keeping Old Immutable Data

I x@(t=0) → 5

I x@(t=1) → 6

I x@(t=13) → 7

I x@(t=15) → 8

I ...


I x@(t=0) → 5

I x@(t=1) → 6

I x@(t=13) → 7

I x@(t=15) → 8

I ...


I x@(t=0) → 5

I x@(t=1) → 6

I x@(t=13) → 7

I x@(t=15) → 8

I ...


I x@(t=0) → 5

I x@(t=1) → 6

I x@(t=13) → 7

I x@(t=15) → 8

I ...


I x@(t=0) → 5

I x@(t=1) → 6

I x@(t=13) → 7

I x@(t=15) → 8

I ...


I x@(t=0) → 5

I x@(t=1) → 6

I x@(t=13) → 7

I x@(t=15) → 8

I ...

I The old values of the data are kept and

indexed by time

I Data is immutable once created – we

cannot/will not change it!

I Values only destroyed when unneeded


indexed by time





indexed by time





indexed by time




Doesn’t keeping old copies of data consume

too much memory?

I (a b c) + d = (a b c d)

I What if the input and output shared

structure?

I Sharing structure is dangerous for

mutable data

I ...but sharing structure is safe if the

data is immutable.

I (a b c) + d = (a b c d)


structure?


mutable data


data is immutable.

I (a b c) + d = (a b c d)


structure?


mutable data


data is immutable.

I (a b c) + d = (a b c d)


structure?


mutable data


data is immutable.

I (a b c) + d = (a b c d)


structure?


mutable data


data is immutable.

The Trick: Represent the list as a tree

input tree → pure function → output tree

both trees are immutable but distinct

Same approach works also for insertions,

modifications, deletions, and all other list

operations.

a million threads, a million trees, a million

references to trees, zero locks

If you want a more current worldview, just

get its reference

Advantages of immutable trees:

I No locking required

I No ’stopping the world’ to see (readers

don’t block writers)

I Worldview never becomes corrupted

I Minimizes memory use while

maintaining multiple copies

I Unused nodes are garbage-collected









































We’ve gone a long way, but we’re only half

way to real concurrency

Immutability lets us read concurrently, but

not write concurrently to a single piece of

data

How can we coordinate the actions of

different threads working on the same data

at the same time?

”Treat changes in an identity’s value as a

database transaction.” – Rich Hickey

Database Details:

I Software Transactional Memory (STM)

I Multi-Version Concurrency Control

(MVCC)

Database Details:



(MVCC)

Database Details:



(MVCC)

The STM Guarantees:

I Atomicity (All or nothing)

I Consistency (Validation before commits)

I Isolation (Transactions can’t see each

other)

The STM Guarantees:




other)

The STM Guarantees:




other)

The STM Guarantees:




other)

Transactions are speculative and may be

retried if there is a collision.

For even more concurrency:

I Sometimes you don’t care about the

order of function application

I Commutative writers won’t need to retry

I Writers don’t interfere with other

writers!






writers!






writers!






writers!

I STMs exist for other languages...

I ... but Clojure is first to have built-in

STM with pervasive immutability







Part 1 Summary: In Clojure...

I ...Readers don’t block anybody

I ...Writers don’t block anybody

I ...Writers retry if conflicting

I ...Writers don’t retry if commutative





















Variables couldn’t do this because:

I Confuse identity and values

I Are mutable and can be corrupted

I Assume a single thread of control, no

interruptions

I Maintain only the last written copy





interruptions






interruptions






interruptions






interruptions


”Mutable stateful objects are the new

spaghetti code” – Rich Hickey

”We oppose the uncontrolled mutation of

variables.” – Stuart Halloway

”Mutable objects are a concurrency

disaster.” – Rich Hickey

”The future is a function of the past, but

doesn’t change it.” – Rich Hickey

”Many people can watch a baseball game,

but only one can be at bat.” – Rich Hickey

Part 2: Revenge of the Lisp

Claim #2: Your language’s syntax is

unneccesarily complex

Now we’ll explain why lisp has parentheses!

Pure functions represent computation as

trees

Reason 1: The tree structure is made

explicitly visible by the parentheses

Sorry, Haskell/Erlang/OCaml/ML/etc!

Infix Notation

1 + 1

Prefix Notation

(+ 1 1)

(fn arg1 arg2 arg3 ...)

1 + 2 + 3 + 4

(+ 1 2 3 4)

With prefix notation, you can forget about

the rules of precedence.

6 + 12 / 2 * 3

(6 + 12) / (2 * 3)

(/ (+ 6 12) (* 2 3))

(/ (+ 6p

12)

(* 2

3))

Triangular Tree Structure

Lisp code’s tree of computation is

exceptionally visible and regular.

Reason 2: Homoiconicity

Homoiconic = Homo + icon = ”same” +

”representation”

The property where code & a language

primitive look the same.

An example: Writing C with XML

for (i=0; i<100; i++) {

printf("%d\n", i);

dostuff();

}

<for>

<init>i = 0</init>

<test>i < 100</test>

<count>i++</count>

<body>

<print format="%d" args="i"/>

<dostuff/>

</body>

</for>

Imagine how simple it would be to use an

XML generator to emit compilable source

code.

We could modify our code programmatically.

In lisp, you can write programs that write

programs.

(list 1 2 3) -> (1 2 3)

(list ’+ 1 2 3) -> (+ 1 2 3)

(+ 1 2 3) -> 6

(defmacro and

([] true)

([x] x)

([x & rest]

‘(let [and# ~x]

(if and# (and ~@rest) and#))))

Why lispers go nuts:

I Macros in lisp are far more powerful

than in other languages.

I You can build new constructs that are

just as legitimate as existing constructs

like if

I You can abstract away boilerplate code






like if







like if







like if


Aside

I If Java used parentheses properly, XML

wouldn’t exist

I Lisp parentheses describe structure

I Most languages use ad-hoc syntax, data

formats

I Simplicity is elegance

Aside


wouldn’t exist



formats


Aside


wouldn’t exist



formats


Aside


wouldn’t exist



formats


Aside


wouldn’t exist



formats


Reason 3: Structural Editing

Good editors let you work with code in

blocks and forget about the parentheses.

Hard-to-show Examples

I When you type (, emacs adds the )

I Indentation is automatic

I You can easily navigate heirarchically

I Take next three expressions, apply them

to a function






to a function






to a function






to a function






to a function

Summary of Lisp Parentheses

I Parentheses render explicit the

tree-structure of your program

I Homoiconicity lets you write programs

with programs

I Structural Editing is fun and easy





with programs






with programs






with programs


Syntax is bad because

I It hides the program’s structure

I It destroys homoiconicity

I It is needlessly complex













”Things should be made as simple as

possible – but no simpler.” – Albert Einstein

Part 3: OOP isn’t the only path to

Polymorphism and Code Reuse

OOP has good goals

1. to group objects together

2. to encapsulate

3. to dispatch polymorphically

4. to reuse code

OOP has good goals


2. to encapsulate


4. to reuse code

OOP has good goals


2. to encapsulate


4. to reuse code

OOP has good goals


2. to encapsulate


4. to reuse code

OOP has good goals


2. to encapsulate


4. to reuse code

These are all good ideas and good goals.

However, OOP is not the only way to reach

these goals.

Claim #3: ”Functions compose better than

objects.”

The fundamental mechanism of OOP – the

inheritance of data, interfaces, type, or

methods from a parent – is often more

difficult to use in practice than techniques

that use functions to achieve the same effect.

Functions are simpler than objects.

Objects are semantic compounds of types,

data, and methods.

Implementation inheritance is bad:

I Forces “is-a” relationship instead of

“has-a”, and “has-a” is almost always

better

I Heirarchical nominalization is difficult

I Changes to a class affect all the

subclasses




better



subclasses




better



subclasses




better



subclasses

An example will help clarify.

Balls

I Ball class (presumably round)

I rollingBall subclass

I bouncingBall subclass

Balls




Balls




Balls




Problems

I What happens if we want to make a ball

that both rolls and bounces?

I Do/Can we inherit from both?

I What if our ball cracks and loses its

bounciness?

I Is a non-round rugby ball a subclass of

ball too?

Problems





bounciness?


ball too?

Problems





bounciness?


ball too?

Problems





bounciness?


ball too?

Problems





bounciness?


ball too?

Interfaces are Simpler

I Define functional interfaces, but don’t

inherit the implementation

I If you want to use another object’s

function to accomplish a task, just use it

I No need to encapsulate their function in

an object

I Multiple interfaces are simpler than

multiple inheritance







an object









an object









an object









an object



FREE THE VERBS!! Separate your object

methods from your objects!

Example: Single vs Multiple Dispatch

Making Drum Noises

I Drum, cymbal and stick classes

I When I hit something with the stick, it

makes a noise

I Single Dispatch:

drum.makeNoise(drumstick)

I cymbal.makeNoise(drumstick)

Making Drum Noises



makes a noise

I Single Dispatch:



Making Drum Noises



makes a noise

I Single Dispatch:



Making Drum Noises



makes a noise

I Single Dispatch:



Making Drum Noises



makes a noise

I Single Dispatch:



The verbs are owned by the nouns.

But what happens when I add a different

stick class?

Now I will need to modify the drum and

cymbal classes and add new methods to

handle the mallets!

When two objects hit, the sound is function

of both objects.

With multi-method functions

I hit(drumstick, cymbal) = crash

I hit(mallet, cymbal) = roar

I hit(drumstick, drum) = bam

I hit(mallet, drum) = bom

I hit(drumstick, drumstick) = click

I hit(cymbal, cymbal) = loud crash











































IMPORTANT: hit() is not a single function,

but a collection of functions. (It is called a

multi-method or generic function)

The particular function that is called is

determined by the type of both of its

arguments.

As you add more classes, just add more

definitions of hit().

Part 3 Conclusion

I Polymorphism is better done through

interfaces than subtype inheritance.

I Functions do not require a heirarchy

I Functions allow simple multiple dispatch

Part 3 Conclusion





Part 3 Conclusion





Part 3 Conclusion





The End

Any Questions?