Advance Scala - Oleg Mürk

Advanced ScalaApril 28th, 2016

OLEG MÜRK, CHIEF ARCHITECT

PLANET OS - APRIL 2016

Advanced Scala

• Advanced features are mostly meant for writing new • Libraries • Frameworks • Domain Specific Languages (DSLs) • Language extensions

• Application developer mostly needs to know these features • To efficiently use the libraries • To efficiently debug issues • To understand weird typing errors


• Beginner (A1) • Java-like expressions • Class, def, val, var • Closures • Collections • For-expressions

• Intermediate (A2) • Pattern matching • Trait composition • (Tail) recursion

• Expert (A3) • Folds • Streams • Actors • Parser combinators

Scala Levels: Application Developer


• Beginner (Library: L1) • Type parameters • Traits • Lazy vals • Currying • Call-by-name • Intermediate (L2)

• Variance • Existential types • Self types • Monads • Extractors

• Expert (L3) • Abstract types • Implicit definitions • Higher-kinded types

Scala Levels: Library Developer


Higher Order Functions

class List[A] {

def filter(f : A => Boolean) : List[A]

def map[B](f : A => B) : List[B]

def foldLeft[B](z: B)(f: (B, A) => B): B

def foldRight[B](z: B)(f: (A, B) => B): B

}

• Higher order functions take/return functions as arguments


Call-by-Name Parameters

• Call-by-name parameters are not executed at the time of call

def runLater[R](f: => R) = { new Thread( new Runnable { def run { val result = f; … } } ).start()}

runLater { blockingExpensiveFun() }


Implicit Conversions

• Implicit conversions are implemented with implicit methods

object A2BUtils { implicit def convertA2B(a:A):B = …}

import A2BUtils._

val a:A = …val b:B = a // convertA2B(a)


Extension Methods

• Extensions methods can be implemented with implicit classes

implicit class DurationInt(private val n: Int) { def seconds = { Duration(n.toLong, TimeUnit.SECONDS) }}

5.seconds // new DurationInt(5).seconds


Implicit Parameters

• Methods & classes can have implicit parameters

def work(…)(implicit timeout:Duration) = { …}

implicit val myDuration = 5.seconds

work(…) // work(…)(myDuration)


Type Classes aka Ad-Hoc Polymorphism

• Type classes can be implemented using implicitstrait Ordering[T] { def compare(x: T, y: T): Int}def min[T](a:T, b:T)(implicit o:Ordering[T]) = …

// Define ordering of Intsimplicit object IntOrdering extends Ordering[Int] { def compare(x: Int, y: Int) = x - y}

min(5,6) // min(5,6)(IntOrdering)


Type Classes aka Ad-Hoc Polymorphism

• Type classes can be implemented using implicitstrait Ordering[T] { def compare(x: T, y: T): Int}

// These two are equivalentdef min[T](a:T, b:T)(implicit o:Ordering[T]) = …def min[T:Ordering](a:T, b:T) = if (implicitly[Ordering[T]].compare(a,b)) a else b}


Futures

• Is a way to asynchronously retrieve the result of a concurrent operation

def blockingExpensiveFun() : Result = { … }

val resultFuture : Future[Result] = Future { blockingExpensiveFun() : Result }


Execution Context

• Is an abstraction for running Runnable • It is usually passed around implicitly

implicit val executionContext: ExecutionContext = …

val resultFuture: Future[Result] = Future { blockingExpensiveFun() }//(executionContext)


Awaiting Future Results

• Synchronously await future result with timeout import scala.concurrent._import scala.concurrent.duration._ val resultFuture = Future { blockingExpensiveFun() }

// In most cases You shouldn’t do that :)val result = Await.result(resutFuture, 15.seconds)


Future Results & Callbacks

• Futures can succeed or fail

val resultFuture: Future[Result] = Future { blockingExpensiveFun()} resultFuture onComplete { case Success(result) => … case Failure(throwable) => …}


Promises

• Promises are used to communicate values between threads

trait Promise[R] {

def future : Future[R]

def success(result:R)

}


Promises

• Promises are used to communicate values between threadsdef runLater[R](f: => R) : Future[R] = { val promise = Promise[R]() new Thread(new Runnable { def run { val result = f; promise.success(result) }}).start() promise.future}

val resultFuture:Future[R] = runLater{ blockingExpensiveFun() }


Futures Can be Composed: … using Callback Hell

val rateQuote = Future { connection.getCurrentValue(USD)}

rateQuote onSuccess { case quote => val purchase = Future { if (isProfitable(quote)) connection.buy(amount, quote) else throw new Exception("not profitable") } purchase onSuccess { case _ => println("Purchased " + amount + " USD") }}


Futures Can be Composed: … or using Monad nirvana

val usdQuote = Future { connection.getCurrentValue(USD) }val chfQuote = Future { connection.getCurrentValue(CHF) }

val purchase = for { usd <- usdQuote chf <- chfQuote if isProfitable(usd, chf)} yield connection.buy(amount, chf)

purchase onSuccess { case _ => println("Purchased " + amount + " CHF")}


For-Comprehensions are Syntactic Sugar for Monads

val purchase = usdQuote.flatMap { usd => chfQuote .withFilter(chf => isProfitable(usd, chf)) .map(chf => connection.buy(amount, chf))}

val purchase = for { usd <- usdQuote chf <- chfQuote if isProfitable(usd, chf)} yield connection.buy(amount, chf)


For Comprehensions

• For comprehensions work with any Monad • Examples of monads:

• Seq • List • Option • Either • Try • …


For Comprehensions: Seq Example

val list : Seq[(Int,Int)] = for( i <- 0 to 100 j <- i to 2*i if (i*j % 7 == 0) ) yield (i,j)


For Comprehensions: Option Example

sealed trait Option[+T]case class Some[+T](v:T) extends Option[T]case object None extends Option[Nothing]

val option : Option[Result] = for( i <- mayBeIntermediate() : Option[Intermediate] r <- mayBeResult(r) : Option[Result]

if check(i, r) ) yield r


For-Comprehensions: Desugaring

genA().map(a => f(a))

for{ a <- genA()} yield f(a)



genA().flatMap{ a => genB(a).map(b => f(a,b)) }

for{ a <- genA() b <- gebB(a)} yield fun(a,b)



genA().flatMap{ a => genB(a) .withFilter{ b => check(a,b) } .map(b => f(a,b)) ) } }

for{ a <- genA() b <- gebB(a) if check(a,b) } yield f(a,b)


For Comprehensions & Monads

• For comprehensions work with any Monad • Examples of monads:

• Seq, List, Option, Either, Try • Futures • IO, (Side-)Effects • Query DSLs (Slick) • Testing DSLs (ScalaCheck)


Slick - Functional Relational Mapping (FRM)

select c.name, c.price, s.name from coffees c join suppliers s on (c.supID == s.id)where c.name = "Expresso"

for { (c, s) <- coffees join suppliers on (_.supID === _.id) if c.name === "Espresso"} yield (c.name, c.price, s.name)


ScalaCheck - Random Testing of Program Properties

val ints = Gen.choose(-100, 100)

def leafs: Gen[Leaf] = for { x <- ints} yield Leaf(x)

def nodes: Gen[Node] = for { left <- trees right <- trees} yield Node(left, right)

def trees: Gen[Tree] = Gen.oneOf(leafs, nodes)


Monads, Monads, Monads, Monads!


Monads

All told, a monad in X is just a monoid in the category of endofunctors of X, with product × replaced by composition of endofunctors and unit set by the identity endofunctor.

Saunders Mac Lane Categories for the Working Mathematician


Monads in Scala

• Monad is a trait parametrized by higher kinded type, eg • List[ _ ] • Option[ _ ] • …

trait Monad[M[_]] { def unit[A](a: A): M[A] def flatMap[A, B](m: M[A])(f: A => M[B]): M[B]}


Monads Laws

• Left identity: unit(a) >>= f ≡ f(a)

• Right identity: m >>= (x => unit(x)) ≡ m

• Associativity: (m >>= f) >>= g ≡ m >>= (x => (f(x) >>= g))

• Operator >>= denotes flatMap


List Monad Exampleimplicit object MonadicList extends Monad[List] {

def unit[A](a:A) : List[A] = List(a)

def flatMap[A,B](l: List[A]) (f: A => List[B]) : List[B] = {

val i:List[List[B]] = l.map(f) i.flatten

}

}


Option Monad Exampleimplicit object MonadicOption extends Monad[Option] {

def unit[A](a:A) : Option[A] = Some(a)

def flatMap[A,B](opt : Option[A]) (f: A => Option[B]) : Option[B] = { opt match { case Some(a) => f(a) : Option[B] case None => None } }}


Advanced Scala Concepts

• Implicits • Implicit parameters • Implicit conversions • Extension methods

• Type-classes aka Adhoc Polymorphism

• For-comprehensions and Monads


Very Advanced Scala Concepts

• Algebra based programming • Monoids, groups, rings

• Category theory based programming • Functors, applicative functors, monads, arrows

• Generic programming • Type-level programming • Meta-programming

• Scala Reflect • Scala Meta


Scala Libraries / DSLs

• Async - simpler way to write async code • Parser Combinators - grammar parsing DSL • ScalaCheck - random testing of program properties • Algebird - abstract algebra for scala • Scalaz/Cats - category theory based programming • Shapeless - generic programming • Spire - generic numeric computations


Projects in Scala

• Slick - functional relational mapping for Scala • Akka - actor-based reactive programming tookit • Finagle - fault tolerant, protocol-agnostic RPC system • Spray - high-performance REST/HTTP for Akka • Play - high velocity web framework for Java and Scala • Scalding - Scala library to specify MapReduce jobs • Spark - in-memory cluster computing • Kafka - publish-subscribe messaging • Samza - distributed stream processing framework


Scala Advantages

• FP gateway drug for Java developers • You can treat Scala as Java with prettier syntax • Runs on JVM, integrates with libraries • Higher productivity, less lines of code (with type-inference) • Very suitable for:

• Data processing • Concurrent/distributed programming • Libraries/DSLs

• Many errors are caught at compile time (by the type system)


Development in Scala


Scala Downsides: “Modern C++”

• Shoehorning Scala type system into JVM • Slowish compile-time forces to modularize • Brittle across Scala and library versions • Sometimes scary compiler messages • Compulsion to use all possible Scala features • Doesn't prevent from creating write-only code • Libraries/DSLs can have cryptic operators and types • Optimized inner loops cannot use any advanced features • Tooling got much better lately


Scala Downsides: Cultural

• Too many different programming styles possible • Object-oriented • Basic functional programming • Advanced functional programming • Category theory with unicode symbols • Generic programming • Type-level programming

• Need to enforce cultural/coding standards in organization


Obligatory Moralistic Slide

49 SEPTEMBER 2015

[email protected]

Technology

Advance Scala - Oleg Mürk