ScalaDays Amsterdam - Don't block yourself

DONT BLOCK YOURSELFFlavio W. Brasil - SoundCloud

@flaviowbrasil

@fwbrasil

backend engineer

Service composition

getclump/clump

Service composition Distributed Durable STM

getclump/clump fwbrasil/activate

Service composition Distributed Durable STM Sane scala reflection

getclump/clump fwbrasil/activate fwbrasil/smirror

Service composition Distributed Durable STM Sane scala reflection Type-level validation


fwbrasil/bond

Service composition Distributed Durable STM Sane scala reflection Type-level validation Type-safe REST


fwbrasil/bond fwbrasil/zoot

12 hours of new content/minute

12 hours of new content/minute 350 million users/month

12 hours of new content/minute 350 million users/month

1 outer space user

1) WHY NON-BLOCKING? !

2) FUTURES AND PROMISES !

3) YOUR SERVER AS A FUNCTION

PART ONE !

WHY NON-BLOCKING?

How many requests?

TimelineAPI

TimelineAPI

Blocking IO

Threads aren't free!

Threads aren't free!

3 facts

Java uses native threads1

The stack uses memory2

Garbage collection pressure

3

GC pressure

Java uses native threads

The stack uses memory

Multiplexing!

Waiters aren't free!

Thread

Thread

EventLoop

TimelineAPI

TimelineAPI

timeline.stream(u): List[Event]

TimelineAPI

TimelineAPI

timeline.stream(u, callback)

TimelineAPI

timeline.stream(u): Future[List[Event]]

PART TWO !

FUTURES AND PROMISES

Future[List[Event]]

Reference for an asynchronous operation completion

Future[List[Event]]

Returns immediately

timeline .stream(u): Future[List[Event]]

future .poll: Option[Try[List[Event]]]

!

val filtered: Future[List[Event]] = timeline.stream(u).map { _.filter(_.isTrackPost) }

Compose a new future

Compose a new future using another future

timeline.stream(u).flatMap { fetchTrackMetadata(_)}

val session = authenticate(request)!

val geolocation = geolocate(request)

Parallel

val session = authenticate(request)!

val geolocation = geolocate(request)

!

session.join(geolocation).map { case (session, geo) => doSomething}

Compose parallel futures

!

val tracks: Seq[Future[Track]] = ids.map(tracksService.fetch)!

val collect: Future[Seq[Track]] = Future.collect(tracks)

Compose parallel futures

Recursive composition

Recursive composition

def likes(user: Urn, offset: Int = 0) = likesService.fetch(user, offset).flatMap { case page if(page.hasNext) => likes(user, page.nextOffset).map { page.events ++ _ } case page => Future.value(page.events) }

!

timeline.stream(u).flatMap { events => Future.collect(Seq( fetchTracks(events), fetchPlaylists(events), fetchComments(events), fetchUsers(events)))}

Timeout

!

timeline.stream(u).flatMap { events => Future.collect(Seq( fetchTracks(events), fetchPlaylists(events), fetchComments(events), fetchUsers(events)))}.within(2.seconds)

Cancellation

!

timeline.stream(u).flatMap { events => Future.collect(Seq( fetchTracks(events), fetchPlaylists(events), fetchComments(events), fetchUsers(events)))}.within(2.seconds)

Local values

!

object Context { val requestId = new Local[Int]}!

def endpoint(request: Request) = callService1 .map(applyATransformation(_)) .flatMap(callService2) timeline.stream(u, Context.requestId())

Local values

Promise[Stream]

A promise is a writable future

Promise[Stream]

val promise = Promise[List[Event]]()promise.setValue(List())

Not common to use

val promise = Promise[List[Event]]()promise.setValue(List())

PART THREE !

YOUR SERVER AS A FUNCTION

The tripod

Futures

Services

Filters

Futures

Services

Servicestype Service[Req, Rep] = Req => Future[Rep]

Services

Asynchronous functions that represent systems boundaries

type Service[Req, Rep] = Req => Future[Rep]

Services

Symmetric and uniform API for clients and servers

type Service[Req, Rep] = Req => Future[Rep]

Services

val client: Service[HttpReq, HttpRep] = !val f: Future[HttpRep] = client(HttpReq("/"))

Services

val client: Service[HttpReq, HttpRep] = !val f: Future[HttpRep] = client(HttpReq("/"))

val server: Service[HttpReq, HttpRep] = !Http.serve(":80", service)

one-liner proxy

Http.serve(:81", Http.newClient(127.0.0.1:80))

one-liner proxy

Filterstype Filter[Req, Rep] = (Req, Service[Req, Rep]) => Future[Rep]

Filterstype Filter[Req, Rep] = (Req, Service[Req, Rep]) => Future[Rep]

Application agnostic concerns (timeout, retry, etc)

Filtersclass TimeoutFilter[Req, Rep] extends SimpleFilter[Req, Rep] { def apply(req: Req, service: Service[Req, Rep]) = service(req).within(2.seconds)}

Filters

val serviceWithTimeout = new TimeoutFilter andThen service

class TimeoutFilter[Req, Rep] extends SimpleFilter[Req, Rep] { def apply(req: Req, service: Service[Req, Rep]) = service(req).within(2.seconds)}

Not only a tripod

Not only NIO

- Connection pooling

- Connection pooling - Load-balancers

- Connection pooling - Load-balancers - Distributed tracing

- Connection pooling - Load-balancers - Distributed tracing - Service discovery

- Connection pooling - Load-balancers - Distributed tracing - Service discovery - Metrics

REACTIVE!!!!!!!1111

- Failover strategies

- Failover strategies - Failure detectors

- Failover strategies - Failure detectors - Dynamic cluster

- Failover strategies - Failure detectors - Dynamic cluster - Back-pressure

MUX

GC avoidance

Failure detector

kraken example

Errors / second

Failure detector

CONCLUSIONS

- Can't use scala futures !

!

- Can't use scala futures - slow adoption of new scala versions

- Can't use scala futures - slow adoption of new scala versions - netty 3

- Can't use scala futures - slow adoption of new scala versions - netty 3 - Internals are complex

- Can't use scala futures - slow adoption of new scala versions - netty 3 - Internals are complex - Mysql driver is limited

Happy user

- Really good toolchain - - -

- Really good toolchain - Many protocols - -

- Really good toolchain - Many protocols - Active community -

- Really good toolchain - Many protocols - Active community - Battle tested

DONT BLOCK YOURSELFFlavio W. Brasil - SoundCloud

Technology

ScalaDays Amsterdam - Don't block yourself