Introduction to Functional Reactive Programming through...

Introduction to FunctionalReactive Programming through

Games and MusicBauhaus-Universität Weimar, 10–13 April 2017

Henrik Nilsson

Functional Programming Laboratory, School of Computer Science

University of Nottingham, UK

Introduction to Functional Reactive Programming through Games and Music – p.1/50

FP in the Real World (1)

Many eloquent and compelling cases forfunctional programming in general:

Introduction to Functional Reactive Programming through Games and Music – p.2/50

FP in the Real World (1)

Many eloquent and compelling cases forfunctional programming in general:

• John Backus, 1977 ACM Turing AwardLecture: Can Programming Be Liberated fromthe von Neumann Style?

Introduction to Functional Reactive Programming through Games and Music – p.2/50

FP in the Real World (1)

Many eloquent and compelling cases forfunctional programming in general:

• John Backus, 1977 ACM Turing AwardLecture: Can Programming Be Liberated fromthe von Neumann Style?

• John Hughes, recent retrospective: WhyFunctional Programming Matters(on YouTube, recommended)

Introduction to Functional Reactive Programming through Games and Music – p.2/50

FP in the Real World (1)

Many eloquent and compelling cases forfunctional programming in general:

• John Backus, 1977 ACM Turing AwardLecture: Can Programming Be Liberated fromthe von Neumann Style?

• John Hughes, recent retrospective: WhyFunctional Programming Matters(on YouTube, recommended)

One key point: Program with whole values, not aword-at-a-time. (Will come back to this.)

Introduction to Functional Reactive Programming through Games and Music – p.2/50

FP in the Real World (2)

But how can pure functions:

Introduction to Functional Reactive Programming through Games and Music – p.3/50

FP in the Real World (2)

But how can pure functions:

• Communicate with the outside world?

Introduction to Functional Reactive Programming through Games and Music – p.3/50

FP in the Real World (2)

But how can pure functions:

• Communicate with the outside world?

• React in a timely way to events?

Introduction to Functional Reactive Programming through Games and Music – p.3/50

FP in the Real World (2)

But how can pure functions:

• Communicate with the outside world?

• React in a timely way to events?

• Deal with state?

And so on.

Introduction to Functional Reactive Programming through Games and Music – p.3/50

FP in the Real World (3)

One answer:

Introduction to Functional Reactive Programming through Games and Music – p.4/50

FP in the Real World (3)

One answer: Forget about purity!

Introduction to Functional Reactive Programming through Games and Music – p.4/50

FP in the Real World (3)

One answer: Forget about purity!

• Allow functions to have side effects:state, exceptions, I/O, . . .

• Some benefits of functional style retained

Introduction to Functional Reactive Programming through Games and Music – p.4/50

FP in the Real World (3)

One answer: Forget about purity!

• Allow functions to have side effects:state, exceptions, I/O, . . .

• Some benefits of functional style retained

• Approach of languages like Lisp, Scheme,ML, OCaml, Erlang, Scala, Erlang

Introduction to Functional Reactive Programming through Games and Music – p.4/50

FP in the Real World (3)

One answer: Forget about purity!

• Allow functions to have side effects:state, exceptions, I/O, . . .

• Some benefits of functional style retained

• Approach of languages like Lisp, Scheme,ML, OCaml, Erlang, Scala, Erlang

However:

Introduction to Functional Reactive Programming through Games and Music – p.4/50

FP in the Real World (3)

One answer: Forget about purity!

• Allow functions to have side effects:state, exceptions, I/O, . . .

• Some benefits of functional style retained

• Approach of languages like Lisp, Scheme,ML, OCaml, Erlang, Scala, Erlang

However:

• Can make code harder to understand

Introduction to Functional Reactive Programming through Games and Music – p.4/50

FP in the Real World (3)

One answer: Forget about purity!

• Allow functions to have side effects:state, exceptions, I/O, . . .

• Some benefits of functional style retained

• Approach of languages like Lisp, Scheme,ML, OCaml, Erlang, Scala, Erlang

However:

• Can make code harder to understand

• Impairs formal reasoning

Introduction to Functional Reactive Programming through Games and Music – p.4/50

FP in the Real World (3)

One answer: Forget about purity!

• Allow functions to have side effects:state, exceptions, I/O, . . .

• Some benefits of functional style retained

• Approach of languages like Lisp, Scheme,ML, OCaml, Erlang, Scala, Erlang

However:

• Can make code harder to understand

• Impairs formal reasoning

• Very poor fit with lazy evaluationIntroduction to Functional Reactive Programming through Games and Music – p.4/50

FP in the Real World (3)

Another answer:

Introduction to Functional Reactive Programming through Games and Music – p.5/50

FP in the Real World (3)

Another answer: Monads!

Introduction to Functional Reactive Programming through Games and Music – p.5/50

FP in the Real World (3)

Another answer: Monads!

• Allows effects in a pure setting

Introduction to Functional Reactive Programming through Games and Music – p.5/50

FP in the Real World (3)

Another answer: Monads!

• Allows effects in a pure setting

• Flexible: any kind of interfacing or imperativealgorithms can be expressed

Introduction to Functional Reactive Programming through Games and Music – p.5/50

FP in the Real World (3)

Another answer: Monads!

• Allows effects in a pure setting

• Flexible: any kind of interfacing or imperativealgorithms can be expressed

• Crucial to the success of pure languages likeHaskell

Introduction to Functional Reactive Programming through Games and Music – p.5/50

FP in the Real World (3)

Another answer: Monads!

• Allows effects in a pure setting

• Flexible: any kind of interfacing or imperativealgorithms can be expressed

• Crucial to the success of pure languages likeHaskell

However, whatever fragments of the code areexpressed in an effectful manner comes with theusual caveats: in particular, often an imperative,“word-at-a-time” programming style.

Introduction to Functional Reactive Programming through Games and Music – p.5/50

FP in the Real World (4)

Here, we are going to consider a third option:

• Reactive systems described through purefunctions over time-varying entities.

• Whole-value approach, as opposed toword-at-a-time.

Introduction to Functional Reactive Programming through Games and Music – p.6/50

FP in the Real World (4)

Here, we are going to consider a third option:

• Reactive systems described through purefunctions over time-varying entities.

• Whole-value approach, as opposed toword-at-a-time.

Reactive systems:

• Input arrives incrementally while system running.

• Output is generated in response to input in aninterleaved and timely fashion.

Contrast transformational systems.

Introduction to Functional Reactive Programming through Games and Music – p.6/50

Functional Reactive Programming

Introduction to Functional Reactive Programming through Games and Music – p.7/50

Functional Reactive Programming

• Key idea: Don’t program one-time-step-at-a-time,but describe an evolving entity as whole.

Introduction to Functional Reactive Programming through Games and Music – p.7/50

Functional Reactive Programming

• Key idea: Don’t program one-time-step-at-a-time,but describe an evolving entity as whole.

• FRP originated in Conal Elliott and Paul Hudak’swork on Functional Reactive Animation (Fran).

Introduction to Functional Reactive Programming through Games and Music – p.7/50

Functional Reactive Programming

• Key idea: Don’t program one-time-step-at-a-time,but describe an evolving entity as whole.

• FRP originated in Conal Elliott and Paul Hudak’swork on Functional Reactive Animation (Fran).Highly cited 1997 ICFP paper; ICFP award formost influential paper in 2007.

Introduction to Functional Reactive Programming through Games and Music – p.7/50

Functional Reactive Programming

• Key idea: Don’t program one-time-step-at-a-time,but describe an evolving entity as whole.

• FRP originated in Conal Elliott and Paul Hudak’swork on Functional Reactive Animation (Fran).Highly cited 1997 ICFP paper; ICFP award formost influential paper in 2007.

• FRP has evolved in a number of directionsand into different concrete implementations.

Introduction to Functional Reactive Programming through Games and Music – p.7/50

Functional Reactive Programming

• Key idea: Don’t program one-time-step-at-a-time,but describe an evolving entity as whole.

• FRP originated in Conal Elliott and Paul Hudak’swork on Functional Reactive Animation (Fran).Highly cited 1997 ICFP paper; ICFP award formost influential paper in 2007.

• FRP has evolved in a number of directionsand into different concrete implementations.

• We will use Yampa: an FRP systemembedded in Haskell.

Introduction to Functional Reactive Programming through Games and Music – p.7/50

FRP Applications

Some domains where FRP or FRP-inspiredapproaches have been used:

• Graphical Animation

• Robotics

• Vision

• Sound synthesis

• GUIs

• Virtual Reality Environments

• Games

Introduction to Functional Reactive Programming through Games and Music – p.8/50

FRP Applications

Some domains where FRP or FRP-inspiredapproaches have been used:

• Graphical Animation

• Robotics

• Vision

• Sound synthesis

• GUIs

• Virtual Reality Environments

• GAMES

Introduction to Functional Reactive Programming through Games and Music – p.8/50

Games in FRP (1)

We will illustrate the FRP approach and Yampaby considering simple video games. Our wholevalues are things like:

• The totality of input from the player

• The animated graphics output

• The entire life of a game object

Introduction to Functional Reactive Programming through Games and Music – p.9/50

Games in FRP (1)

We will illustrate the FRP approach and Yampaby considering simple video games. Our wholevalues are things like:

• The totality of input from the player

• The animated graphics output

• The entire life of a game object

We construct and work with pure functions on these:

• The game: function from input to animation

• In the game: fixed point of function oncollection of game objects

Introduction to Functional Reactive Programming through Games and Music – p.9/50

Games in FRP (2)

Play Store: Pang-a-lambda (Keera Studios)

Introduction to Functional Reactive Programming through Games and Music – p.10/50

Games in FRP (3)

Lots of bouncing balls in Pang-a-lambda!

y = y0 +

∫v dt

v = v0 +

∫−9.81

On impact (fully elastic collision):

v = −v(t−)

We are used to describing behaviours in theirtotallity over time in mathematical models. Whynot program in the same way?

Introduction to Functional Reactive Programming through Games and Music – p.11/50

Key FRP Features

Combines conceptual simplicity of the synchronousdata flow approach with the flexibility and abstractionpower of higher-order functional programming:

Introduction to Functional Reactive Programming through Games and Music – p.12/50

Key FRP Features

Combines conceptual simplicity of the synchronousdata flow approach with the flexibility and abstractionpower of higher-order functional programming:

• Synchronous

Introduction to Functional Reactive Programming through Games and Music – p.12/50

Key FRP Features

Combines conceptual simplicity of the synchronousdata flow approach with the flexibility and abstractionpower of higher-order functional programming:

• Synchronous

• First class temporal abstractions

Introduction to Functional Reactive Programming through Games and Music – p.12/50

Key FRP Features

Combines conceptual simplicity of the synchronousdata flow approach with the flexibility and abstractionpower of higher-order functional programming:

• Synchronous

• First class temporal abstractions

• Hybrid: mixed continuous and discrete time

Introduction to Functional Reactive Programming through Games and Music – p.12/50

Key FRP Features

Combines conceptual simplicity of the synchronousdata flow approach with the flexibility and abstractionpower of higher-order functional programming:

• Synchronous

• First class temporal abstractions

• Hybrid: mixed continuous and discrete time

• Dynamic system structure

Introduction to Functional Reactive Programming through Games and Music – p.12/50

Key FRP Features

Combines conceptual simplicity of the synchronousdata flow approach with the flexibility and abstractionpower of higher-order functional programming:

• Synchronous

• First class temporal abstractions

• Hybrid: mixed continuous and discrete time

• Dynamic system structure

Good fit for typical video games, inter alia,(but not everything labelled “FRP” supports them all).

Introduction to Functional Reactive Programming through Games and Music – p.12/50

Yampa

Introduction to Functional Reactive Programming through Games and Music – p.13/50

Yampa

• FRP implementation embedded in Haskell

Introduction to Functional Reactive Programming through Games and Music – p.13/50

Yampa

• FRP implementation embedded in Haskell

• Key notions:

- Signals: time-varying values

- Signal Functions: pure functions on signals

- Switching: temporal composition of signalfunctions

Introduction to Functional Reactive Programming through Games and Music – p.13/50

Yampa

• FRP implementation embedded in Haskell

• Key notions:

- Signals: time-varying values

- Signal Functions: pure functions on signals

- Switching: temporal composition of signalfunctions

• Programming model:

Introduction to Functional Reactive Programming through Games and Music – p.13/50

Yampa?

Introduction to Functional Reactive Programming through Games and Music – p.14/50

Yampa?

Yampa is a river with long calmly flowing sectionsand abrupt whitewater transitions in between.

A good metaphor for hybrid systems!Introduction to Functional Reactive Programming through Games and Music – p.14/50

Signal Functions

Introduction to Functional Reactive Programming through Games and Music – p.15/50

Signal Functions

Intuition:

Introduction to Functional Reactive Programming through Games and Music – p.15/50

Signal Functions

Intuition:

Time ≈ R

Introduction to Functional Reactive Programming through Games and Music – p.15/50

Signal Functions

Intuition:

Time ≈ R

Signal a ≈ Time → a

x :: Signal T1y :: Signal T2

Introduction to Functional Reactive Programming through Games and Music – p.15/50

Signal Functions

Intuition:

Time ≈ R

Signal a ≈ Time → a

x :: Signal T1y :: Signal T2SF a b ≈ Signal a → Signal bf :: SF T1 T2

Introduction to Functional Reactive Programming through Games and Music – p.15/50

Signal Functions

Intuition:

Time ≈ R

Signal a ≈ Time → a

x :: Signal T1y :: Signal T2SF a b ≈ Signal a → Signal bf :: SF T1 T2

Additionally, causality required: output at time t

must be determined by input on interval [0, t].

Introduction to Functional Reactive Programming through Games and Music – p.15/50

Signal Functions and State

Alternative view: Signal functions canencapsulate state:

state(t) summarizes input history x(t′), t′ ∈ [0, t].

Introduction to Functional Reactive Programming through Games and Music – p.16/50

Signal Functions and State

Alternative view: Signal functions canencapsulate state:

state(t) summarizes input history x(t′), t′ ∈ [0, t].

From this perspective, signal functions are:

• state(t)

• stateless if y(t) depends only on x(t)

Introduction to Functional Reactive Programming through Games and Music – p.16/50

Some Basic Signal Functions

identity :: SF a a

Introduction to Functional Reactive Programming through Games and Music – p.17/50

Some Basic Signal Functions

identity :: SF a a

constant :: b → SF a b

Introduction to Functional Reactive Programming through Games and Music – p.17/50

Some Basic Signal Functions

identity :: SF a a

constant :: b → SF a b

iPre :: a → SF a a

Introduction to Functional Reactive Programming through Games and Music – p.17/50

Some Basic Signal Functions

identity :: SF a a

constant :: b → SF a b

iPre :: a → SF a a

integral :: VectorSpace a s ⇒ SF a a

y(t) =

t∫

0

x(τ) dτ

Introduction to Functional Reactive Programming through Games and Music – p.17/50

Composition

In Yampa, systems are described by combiningsignal functions (forming new signal functions).

Introduction to Functional Reactive Programming through Games and Music – p.18/50

Composition

In Yampa, systems are described by combiningsignal functions (forming new signal functions).

For example, serial composition:

Introduction to Functional Reactive Programming through Games and Music – p.18/50

Composition

In Yampa, systems are described by combiningsignal functions (forming new signal functions).

For example, serial composition:

A combinator that captures this idea:

(≫) :: SF a b → SF b c → SF a c

Introduction to Functional Reactive Programming through Games and Music – p.18/50

Composition

In Yampa, systems are described by combiningsignal functions (forming new signal functions).

For example, serial composition:

A combinator that captures this idea:

(≫) :: SF a b → SF b c → SF a c

Signal functions are the primary notion; signals asecondary one, only existing indirectly.

Introduction to Functional Reactive Programming through Games and Music – p.18/50

Systems

What about larger, more complicated networks?How many combinators are needed?

Introduction to Functional Reactive Programming through Games and Music – p.19/50

Systems

What about larger, more complicated networks?How many combinators are needed?

John Hughes’s Arrow framework provides agood answer!

Introduction to Functional Reactive Programming through Games and Music – p.19/50

The Arrow framework (1)

arr f f ≫ g

first f loop f

arr :: (a → b)→ SF a b

(≫) :: SF a b → SF b c → SF a c

first :: SF a b → SF (a, c) (b, c)

loop :: SF (a, c) (b, c)→ SF a bIntroduction to Functional Reactive Programming through Games and Music – p.20/50

Arrow notation

Introduction to Functional Reactive Programming through Games and Music – p.21/50

Arrow notation

Introduction to Functional Reactive Programming through Games and Music – p.21/50

Arrow notation

proc x → do

rec

u ← f −≺ (x , v)

y ← g−≺ u

v ← h−≺ (u, x )

returnA−≺ y

Introduction to Functional Reactive Programming through Games and Music – p.21/50

Arrow notation

proc x → do

rec

u ← f −≺ (x , v)

y ← g−≺ u

v ← h−≺ (u, x )

returnA−≺ y

Only syntactic sugar:

everything translated into a

combinator expression.

Introduction to Functional Reactive Programming through Games and Music – p.21/50

A Bouncing Ball

Lots of bouncing balls in Pang-a-lambda!Consider 1D case:

y = y0 +

∫v dt

v = v0 +

∫−9.81

On impact:

v = −v(t−)

(fully elastic collision)

Introduction to Functional Reactive Programming through Games and Music – p.22/50

Modelling the Bouncing Ball: Part 1

Free-falling ball:

fallingBall :: Pos → Vel → SF () (Pos ,Vel)

fallingBall y0 v0 = proc ()→ do

v ← (v0+) <̂< integral−≺ − 9.81

y ← (y0+) <̂< integral−≺ v

returnA−≺ (y , v)

Introduction to Functional Reactive Programming through Games and Music – p.23/50

Modelling the Bouncing Ball: Part 1

Free-falling ball:

fallingBall :: Pos → Vel → SF () (Pos ,Vel)

fallingBall y0 v0 = proc ()→ do

v ← (v0+) <̂< integral−≺ − 9.81

y ← (y0+) <̂< integral−≺ v

returnA−≺ (y , v)

Some different and extra symbols, but just superficialsyntactic details: the structure remains the same.We have turned the mathematical model into adeclarative program!

Introduction to Functional Reactive Programming through Games and Music – p.23/50

Events

Yampa models discrete-time signals by lifting theco-domain of signals using an option-type:

data Event a = NoEvent | Event a

Discrete-time signal = Signal (Eventα).

Introduction to Functional Reactive Programming through Games and Music – p.24/50

Events

Yampa models discrete-time signals by lifting theco-domain of signals using an option-type:

data Event a = NoEvent | Event a

Discrete-time signal = Signal (Eventα).

Some functions and event sources:

tag :: Event a → b → Event b

after :: Time → b → SF a (Event b)

edge :: SF Bool (Event ())

Introduction to Functional Reactive Programming through Games and Music – p.24/50

Modelling the Bouncing Ball: Part 2

Detecting when the ball goes through the floor:

fallingBall ′ ::

Pos → Vel → SF () ((Pos ,Vel),Event (Pos ,Vel))

fallingBall ′ y0 v0 = proc ()→ do

yv@(y , )← fallingBall y0 v0−≺ ()

hit ← edge −≺ y 6 0

returnA−≺ (yv , hit ‘tag ‘ yv)

Introduction to Functional Reactive Programming through Games and Music – p.25/50

Switching

Q: How and when do signal functions “start”?

Introduction to Functional Reactive Programming through Games and Music – p.26/50

Switching

Q: How and when do signal functions “start”?

A: • Switchers apply a signal functions to itsinput signal at some point in time.

Introduction to Functional Reactive Programming through Games and Music – p.26/50

Switching

Q: How and when do signal functions “start”?

A: • Switchers apply a signal functions to itsinput signal at some point in time.

• This is temporal composition of signalfunctions.

Introduction to Functional Reactive Programming through Games and Music – p.26/50

Switching

Q: How and when do signal functions “start”?

A: • Switchers apply a signal functions to itsinput signal at some point in time.

• This is temporal composition of signalfunctions.

Switchers thus allow systems with varyingstructure to be described.

Introduction to Functional Reactive Programming through Games and Music – p.26/50

Switching

Q: How and when do signal functions “start”?

A: • Switchers apply a signal functions to itsinput signal at some point in time.

• This is temporal composition of signalfunctions.

Switchers thus allow systems with varyingstructure to be described.

Generalised switches allow composition ofcollections of signal functions. Can be used tomodel e.g. varying number of objects in a game.

Introduction to Functional Reactive Programming through Games and Music – p.26/50

The Basic Switch

Idea:

• Allows one signal function to be replaced byanother.

• Switching takes place on the first occurrenceof the switching event source.

switch::

SF a (b,Event c)

→ (c → SF a b)

→ SF a b

Introduction to Functional Reactive Programming through Games and Music – p.27/50

The Basic Switch

Idea:

• Allows one signal function to be replaced byanother.

• Switching takes place on the first occurrenceof the switching event source.

switch:: Initial SF with event source

SF a (b,Event c)

→ (c → SF a b)

→ SF a b

Introduction to Functional Reactive Programming through Games and Music – p.27/50

The Basic Switch

Idea:

• Allows one signal function to be replaced byanother.

• Switching takes place on the first occurrenceof the switching event source.

switch:: Function yielding SF to switch into

SF a (b,Event c)

→ (c → SF a b)

→ SF a b

Introduction to Functional Reactive Programming through Games and Music – p.27/50

Modelling the Bouncing Ball: Part 3

Making the ball bounce:

bouncingBall :: Pos → SF () (Pos ,Vel)

bouncingBall y0 = bbAux y0 0.0

where

bbAux y0 v0 =

switch (fallingBall ′ y0 v0 ) $ λ(y , v)→

bbAux y (−v)

Introduction to Functional Reactive Programming through Games and Music – p.28/50

Game Objects

data Object = Object {objectName :: ObjectName

, objectKind :: ObjectKind

, objectPos :: Pos2D

, objectVel :: Vel2D

. . .

}

data ObjectKind = Ball . . . | Player . . . | . . .

data ObjectInput = ObjectInput

{userInput :: Controller

, collisions :: Collisions

}Introduction to Functional Reactive Programming through Games and Music – p.29/50

Overall Game Structure

gamePlay :: [ListSF ObjectInput Object ]

→ SF Controller ([Object ],Time)

gamePlay objs = loopPre [ ] $

proc (input , cs)→ do

let oi = ObjectInput input cs

ol ← dlSwitch objs−≺ oi

let cs ′ = detectCollisions ol

tLeft ← time−≺ ()

returnA−≺ ((ol , tLeft), cs ′)

ListSF and dlSwitch are related abstractions thatallow objects to die or spawn new ones.

Introduction to Functional Reactive Programming through Games and Music – p.30/50

And now for something different . . .

Switched-on Yampa: Programming ModularSynthesizers in Haskell

Introduction to Functional Reactive Programming through Games and Music – p.31/50

And now for something different . . .

Switched-on Yampa: Programming ModularSynthesizers in Haskell

Introduction to Functional Reactive Programming through Games and Music – p.31/50

Modular synthesizers?

Introduction to Functional Reactive Programming through Games and Music – p.32/50

Modular synthesizers?

Introduction to Functional Reactive Programming through Games and Music – p.32/50

Modular synthesizers?

Steve Pocaro, Toto,with Polyfusion Syn-thesizer

Introduction to Functional Reactive Programming through Games and Music – p.32/50

Modern Modular Synthesizers

Introduction to Functional Reactive Programming through Games and Music – p.33/50

Where does Yampa enter the picture?

Introduction to Functional Reactive Programming through Games and Music – p.34/50

Where does Yampa enter the picture?

• Music can be seen as a hybrid phenomenon.Thus interesting to explore a hybrid approachto programming music and musical applications.

Introduction to Functional Reactive Programming through Games and Music – p.34/50

Where does Yampa enter the picture?

• Music can be seen as a hybrid phenomenon.Thus interesting to explore a hybrid approachto programming music and musical applications.

• Yampa’s programming model is very reminiscentof programming modular synthesizers:

Introduction to Functional Reactive Programming through Games and Music – p.34/50

Where does Yampa enter the picture?

• Music can be seen as a hybrid phenomenon.Thus interesting to explore a hybrid approachto programming music and musical applications.

• Yampa’s programming model is very reminiscentof programming modular synthesizers:

• Fun application! Useful for teaching?Introduction to Functional Reactive Programming through Games and Music – p.34/50

What have we done?

Introduction to Functional Reactive Programming through Games and Music – p.35/50

What have we done?

Framework for programming modularsynthesizers in Yampa:

Introduction to Functional Reactive Programming through Games and Music – p.35/50

What have we done?

Framework for programming modularsynthesizers in Yampa:

• Sound-generating and sound-shapingmodules

Introduction to Functional Reactive Programming through Games and Music – p.35/50

What have we done?

Framework for programming modularsynthesizers in Yampa:

• Sound-generating and sound-shapingmodules

• Additional supporting infrastructure:

- Input: MIDI files (musical scores), keyboard

- Output: audio files (.wav), sound card

- Reading SoundFont files (instrumentdefinitions)

Introduction to Functional Reactive Programming through Games and Music – p.35/50

What have we done?

Framework for programming modularsynthesizers in Yampa:

• Sound-generating and sound-shapingmodules

• Additional supporting infrastructure:

- Input: MIDI files (musical scores), keyboard

- Output: audio files (.wav), sound card

- Reading SoundFont files (instrumentdefinitions)

• Status: proof-of-concept, but decent performance.

Introduction to Functional Reactive Programming through Games and Music – p.35/50

Example 1: Sine oscillator

oscSine fcv

oscSine :: Frequency → SF CV Sample

oscSine f0 = proc cv → do

let f = f0 ∗ (2 ∗∗ cv)

phi ← integral−≺ 2 ∗ pi ∗ f

returnA−≺ sin phi

constant 0 ≫ oscSine 440

Introduction to Functional Reactive Programming through Games and Music – p.36/50

Example 2: Vibrato

0oscSine 5.0 oscSine f*0.05

constant 0

≫ oscSine 5.0

≫ arr (∗0.05)

≫ oscSine 440

Introduction to Functional Reactive Programming through Games and Music – p.37/50

Example 3: 50’s Sci Fi

0oscSine 3.0

oscSine f

*0.2

-0.25+1.0

+

sciFi :: SF () Sample

sciFi = proc ()→ do

und ← arr (∗0.2) ≪ oscSine 3.0−≺ 0

swp ← arr (+1.0) ≪ integral −≺ −0.25

audio ← oscSine 440 −≺ und + swp

returnA−≺ audioIntroduction to Functional Reactive Programming through Games and Music – p.38/50

Envelope Generators (1)

A D S R

key on key off t

envGen :: CV → [(Time,CV )]→ (Maybe Int)

→ SF (Event ()) (CV ,Event ())

envEx = envGen 0 [(0.5, 1), (0.5, 0.5), (1.0, 0.5), (0.7, 0)]

(Just 3)

Introduction to Functional Reactive Programming through Games and Music – p.39/50

Envelope Generators (2)

How to implement?

Introduction to Functional Reactive Programming through Games and Music – p.40/50

Envelope Generators (2)

How to implement?

Integration of a step function yields suitableshapes:

t

∫−→

A D S R

key on key off t

Introduction to Functional Reactive Programming through Games and Music – p.40/50

Envelope Generators (3)

t

afterEach :: [(Time, b)]→ SF a (Event b)

hold :: a → SF (Event a) a

steps = afterEach [(0.7, 2), (0.5,−1), (0.5, 0), (1,−0.7), (0.7, 0)]

≫ hold 0

Introduction to Functional Reactive Programming through Games and Music – p.41/50

Envelope Generators (4)

Envelope generator with predetermined shape:

envGenAux :: CV → [(Time,CV )]→ SF a CV

envGenAux l0 tls = afterEach trs ≫ hold r0

≫ integral ≫ arr (+l0 )

where

(r0 , trs) = toRates l0 tls

Introduction to Functional Reactive Programming through Games and Music – p.42/50

Example 4: Bell

0

*oscSine f*2.0oscSine (f*2.33)

envBell

bell :: Frequency → SF () (Sample,Event)

bell f = proc ()→ do

m ← oscSine (2.33 ∗ f )−≺ 0

audio ← oscSine f −≺ 2.0 ∗m

(ampl , end)← envBell −≺ noEvent

returnA−≺ (audio ∗ ampl , end)

Introduction to Functional Reactive Programming through Games and Music – p.43/50

Example 5: Tinkling Bell

tinkle :: SF () Sample

tinkle = (repeatedly 0.25 84

≫ constant ()

&&&arr (fmap (bell ◦midiNoteToFreq))

≫ rSwitch (constant 0))

Introduction to Functional Reactive Programming through Games and Music – p.44/50

Example 6: Playing a C-major scale

scale :: SF () Sample

scale = (afterEach [(0.0, 60), (2.0, 62), (2.0, 64),

(2.0, 65), (2.0, 67), (2.0, 69),

(2.0, 71), (2.0, 72)]

≫ constant ()

&&&arr (fmap (bell ◦midiNoteToFreq))

≫ rSwitch (constant 0))

&&&after 16 ()

Introduction to Functional Reactive Programming through Games and Music – p.45/50

Example 7: Playing simultaneous notes

mysterySong :: SF () (Sample,Event ())

mysterySong = proc → do

t ← tinkle −≺ ()

m ← mystery−≺ ()

returnA−≺ (0.4 ∗ t + 0.6 ∗m)

Introduction to Functional Reactive Programming through Games and Music – p.46/50

A polyphonic synthesizer (1)

Sample-playing monophnic synthesizer:• Read samples (instrument recordings) from

SoundFont file into internal table.• Oscillator similar to sine oscillator, except

table lookup and interpolation instead ofcomputing the sine.

SoundFont synthesizer structure:

Envelopes

LFO

Modulators

Oscillator Lowpass filter Amplifier

FrequencyReverb

ChorusVolume

Fc

Introduction to Functional Reactive Programming through Games and Music – p.47/50

A polyphonic synthesizer (2)

Exploit Yampa’s switching capabilities to:

• create and switch in a mono synth instance isresponse to each note on event;

• switch out the instance in response to acorresponding note off event.

Introduction to Functional Reactive Programming through Games and Music – p.48/50

Switched-on Yampa?

Introduction to Functional Reactive Programming through Games and Music – p.49/50

Switched-on Yampa?

Introduction to Functional Reactive Programming through Games and Music – p.49/50

Conclusions

Introduction to Functional Reactive Programming through Games and Music – p.50/50

Conclusions

• FRP offers one way to write interactive gamesand similar software in a declarative way.

Introduction to Functional Reactive Programming through Games and Music – p.50/50

Conclusions

• FRP offers one way to write interactive gamesand similar software in a declarative way.

• It allows systems to be described in terms ofwhole values varying over time.

Introduction to Functional Reactive Programming through Games and Music – p.50/50

Conclusions

• FRP offers one way to write interactive gamesand similar software in a declarative way.

• It allows systems to be described in terms ofwhole values varying over time.

• Not covered in this talk:

Introduction to Functional Reactive Programming through Games and Music – p.50/50

Conclusions

• FRP offers one way to write interactive gamesand similar software in a declarative way.

• It allows systems to be described in terms ofwhole values varying over time.

• Not covered in this talk:

- Not everything fit easily into the FRPparadigm: e.g., interfacing to existing GUItoolkits, other imperative APIs.

Introduction to Functional Reactive Programming through Games and Music – p.50/50

Conclusions

• FRP offers one way to write interactive gamesand similar software in a declarative way.

• It allows systems to be described in terms ofwhole values varying over time.

• Not covered in this talk:

- Not everything fit easily into the FRPparadigm: e.g., interfacing to existing GUItoolkits, other imperative APIs.

- But also such APIs can be given a “whole-valuetreatment” to improve the fit within a declarativesetting. E.g. Reactive Values and Relations.

Introduction to Functional Reactive Programming through Games and Music – p.50/50