A Slice of Scala

Melbourne Scala User Group - August 2013Kevin O’Neill - CTO PlayUp - @kevinoneill

Kevin O’Neill CTO PlayUp

A Slice of Scala

@kevinoneill

August 2013

A little History

Engage & Entertain Around Live Sport

Mobile Games

SMS Based

Number of Runs

Complex

Carrier Integration

Smart Phone Revolution

Move from a Content Creator to a Content

Enabler

Top 10 in Engagement

Over 3 Million Likes

30% - 40% Active

Manually Curated

We Know there's an Opportunity

October 2011

Technologist at Large

Version 2

Rebuild Based on Hypermedia*

Same Feature Set

Different Skin

Android

Version 3

Platform Extensions

Third Party

Ticketing

Version 4

Engagement Too Deep

Streams

Graph Storage

League

Region

Team Story

League

TeamStory

aboutabout

follows competes in

competes in

follows

competes in

Region

follows

competes in

follows

Contest

plays in

Value is in the Connections

50 Seconds

Cypher Processing too much Data

Java plugin

5 Seconds

10 Times too Long*

Aiming Around 200ms

Time to Change

If you hear a voice within you say 'you cannot paint,' then by all means paint, and that voice will be silenced.

– Vincent Van Gogh

Just Couldn't Face Java

Some Experience with it on a Large Project

The Stack

Scala 2.10.x

akka 2.1.x*

Spray M8+

Play 2.1.x*

Other Bits

Neo4j 1.9.x

Scalaz 7.0.x

Blueprints 2.4.x

Scala Test 2.0.x

Scala Check 1.10.x

SBT 1.12.x*

Systems

Two akka Services*

Graph Service

Sports Connect

Fanbase Connect*

One Spray Server

Hypermedia API

One Play Application*

HTML Front End

Admin Front End*

JSON Generation

Problem : How do you isolate environment

Lambda: The Ultimate Dependency Injection

Framework

Did my Head in

So what if …

Environment Readers

type Settings = Set[EnvironmentSetting[_]] !type EnvironmentReader[+A] = scalaz.Reader[Settings, A] object EnvironmentReader { def apply[A](f: Settings => A):EnvironmentReader[A] = scalaz.Reader(f) }

Writing an Identity

protected def writeResourcePointer(identity: ResourceIdentity) : EnvironmentReader[JObject] = for { ! href <-‐ urlForIdentity(identity) contentType = identity.documentType representations <-‐ writeRepresentations(identity) views <-‐ viewsForIdentity(identity) ! } yield (":href" -‐> href) ~ (":type" -‐> contentType) ~ (":uri" -‐> identity.uri) ~ representations ~ (":views" -‐> views)

urlForIdentity

protected def urlForIdentity(identity:ResourceIdentity) : EnvironmentReader[JValue] = for { ! r <-‐ URLResolverSetting.resolve(identity) href <-‐ mapOrJNothing(r) { ref : String => pure(JString(ref)) } !} yield href

URLResolverSetting

case class URLResolverSetting(value: Resolver[String]) extends EnvironmentSetting[Resolver[String]] { val key = "url-‐resolver" } !object URLResolverSetting extends ResolverSetting ({ case setting: URLResolverSetting => setting.value })

ResolverSetting

class ResolverSetting[T]( finder : PartialFunction[ EnvironmentSetting[_], Resolver[T]]) { ! def resolve(identity : ResourceIdentity) : EnvironmentReader[Option[T]] = EnvironmentReader { settings => for { resolver <-‐ settings.collectFirst(finder) result <-‐ resolver.resolve(identity) } yield result } }

Wooh … That's far more

complex than my … blah blah in blah

Remember I mentioned Runars talk

did my head in.*

It's actually really easy to work with

Each components is responsible for one

Combining components is trivial

You spend most of your time doing

simple, safe, composition

Elephants all the way down

The Language

val route = { language { lang => implicit val env = RequestEnvironmentProvider( EnvironmentConfig(language = Some(lang)) ) … !}

The Directive*

def language: Directive1[Lang] = { headerValue { ! case HttpHeader("accept-‐language", langString) => langString.split(',').headOption.flatMap(firstLanguage => Lang.get(firstLanguage)) ! case _ => None ! } | provide(Lang.defaultLang) }

Creating a Responsedef getStream(implicit env: RequestEnvironmentProvider): Route = { path("stream" / StreamPath) { case (stream, afterStory) => read(FetchStream(stream, afterStory)) { stream => val streamEnv = environmentForStreamWithTopics( env, stream.identity.references ) respondWithHeaders( `Cache-‐Control`(`public`, `max-‐age`(60))) { complete { implicit val criteriaMarshaller: Marshaller[StoryStream] = resourceMarshaller(streamEnv) stream } } } } }

A Little akka Helper

Reading the Graph

def read[T](message : GraphRequest[T])(f : T => Route):Route = { ctx => val response: Future[T] = Graph.reader.ask(message).asInstanceOf[Future[T]] response.map { case Error(messages) => { ctx.complete( InternalServerError, messages.mkString("\n") ) } case result => f(result)(ctx) } }

Topic Management

Topic Actor

private class TopicActor (system : GraphSystem, topic : StreamComponentIdentity) extends GraphWorker(system) with Stash { ! override def preStart() { available(topic) onAvailable { become(active) } onMissing { topicManager ! UpdateTopic(topic) become(pending) } } !… }

Receive

def receive = PartialFunction.empty

Pending

def pending : Actor.Receive = { case message @ LinkStory(story, identity) if identity == topic => { stash() } ! case identity if identity == topic => { unstashAll() become(active) } }

Active

def active : Actor.Receive = { case LinkStory(story, identity) if identity == topic => { gs.write(associateWithSubject(story, topic)) } }

Baby Steps

Challenges

As the Carpenters said …

We've Only Just Begun

Data Load

Sparsity

Propagation

Friends

500 Topics

10,000,000 Nodes

1,000,000 Topics

10,000,000,000 Nodes

Time Series Data

Distribution

Need to change the way you think about

services

Actors are not Queues

Monitoring is Hard

"Real Time"

Balancing Load vs Caching

Two Hard Problems

Naming

Cache Invalidation

Off by One Errors

Caching Stuff is Easy

Invalidation is Hard

Limited Use

Just be Fast

Payments

Ticketing

À la carte

Cross Platform

Come Join Us

Disrupting

Looking for the 'X' factor

Hiring Selectively

Talk to Me or Andrea

You miss 100 percent of the the shots you don't take

– Wayne Gretzky

Kevin O’NeillCTO - PlayUp

kevin@playup.com@kevinoneill

Questions?

A Slice of Scala

Technology

Scala Days 2018 You Are a - Lightbend...scala/scala-parallel-collections scala/scala-collection-compat scala/scala-java8-compat scala/scala-swing scala/scala-async scala/scala-continuations

Scala & Spark Msc. Johannes Härtel PTT18/19 Msc. Marcel ... · What is Scala? - Scala is a general purpose programming language. - Scala provides support for functional programming

Scala Tutorial - UFSCbosco/ensino/ine5645/Scala_tutorial.pdf · Scala Tutorial Scala is a modern ... Basic Operations on Set: ... downloaded scala-2.9.0.1-installer.jar and put it

An Introduction to Scala · 2015-07-29 · Scala is functional Scala is also a functional language in the sense that every function is a value. Scala provides a lightweight syntax

Grab A Slice

Scala - A Scalable Language - oracle.com · Scala compared + abstract types-raw types Scala adds Scala removes + existential types - wildcards + pattern matching-enums + mixin composition

A Slice of Neurons

Scala 2013: a Pragmatic guide to Scala adoPtionpages.zeroturnaround.com/.../images/Scala2013-A-Pragmatic-Guide-To... · Scala 2013: a Pragmatic guide to Scala ... Numerous blog posts

Scala Tutorial - people.cs.ksu.eduschmidt/705a/Scala/scala_tutorial.pdf · Scala is also a functional language in the sense that every function is a value and because every value

Senior Project with the SPP Michael Williamson. Communicating with a Slice Slice-RMP library using a Unix Domain Socket ◦ RPC-Like ◦ Slice application

A SLICE OF EUROPE

Scala in a nutshell

Scala as a Declarative Language

Introducción a scala

Scala Abide: A lint tool for Scala

LONDON COLLECTION 2 SLICE & 4 SLICE TOASTER COLLECTION 2 SLICE & 4 SLICE TOASTER ... Sunbeam Corporation is a division of GUD Holdings Limited. ... London Collection 2 Slice

A “SLICE OF ME”

Optimal Slice Allocation in 5G Core Networks · Optimal Slice Allocation in 5G Core Networks ... also ensures the end-to-end delay required by a core network slice. A 5G network slice

Scala Next SF Scala meetup Dec 8 th, 2011. 2 Scala Today

A Slice of PIE