Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013)

@crichardson

Consuming web services asynchronously with Futures

and Rx ObservablesChris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

@[email protected]://plainoldobjects.com

@crichardson

Presentation goal

Learn how to use (Scala) Futures and Rx

Observables to write simple yet robust and scalable

concurrent code

@crichardson

About Chris

@crichardson

(About Chris)

@crichardson

About Chris()

@crichardson

About Chris

@crichardson

About Chris

http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/

@crichardson

About Chris

@crichardson

Agenda

The need for concurrency

Simplifying concurrent code with Futures

Taming callback hell with JavaScript promises

Consuming asynchronous streams with Reactive Extensions

@crichardson

Let’s imagine you are building an online store

@crichardson

Shipping

Recomendations

ProductInfo

Reviews

@crichardson

Reviews

ProductInfo Sales

ranking

@crichardson

Related books

Viewing history

Forum

@crichardson

+ mobile apps

@crichardson

Application architecture

Product Info ServiceDesktop browser

Native mobile client

REST

REST

REST

Recomendation Service

Review Service

Front end server

API gatewayREST

Mobile browser

Web Application

HTML/JSON

JSON

@crichardson

Handling getProductDetails()

Front-end server

Product Info Service

RecommendationsService

ReviewService

getProductInfo()

getRecommendations()

getReviews()

getProductDetails()

@crichardson

Handling getProductDetails() - sequentially

Front-end server



ReviewService

getProductInfo()


getReviews()

getProductDetails()

Higher response time :-(

@crichardson

Handling getProductDetails() - in parallel

Front-end server



ReviewService

getProductInfo()


getReviews()

getProductDetails()

Lower response time :-)

@crichardson

Implementing a concurrent REST client

Thread-pool based approach

executorService.submit(new Callable(...))

Simpler but less scalable - lots of idle threads consuming memory

Event-driven approach

NIO with completion callbacks

More complex but more scalable

@crichardson

The front-end server must handle partial failure of backend services

Front-end server



ReviewService

getProductInfo()


getReviews()

getProductDetails() XHow to provide a good user experience?

@crichardson

Agenda





@crichardson

Futures are a great concurrency abstraction

http://en.wikipedia.org/wiki/Futures_and_promises

@crichardson

Worker threadMain threadHow futures work

Outcome

Future

Client

get

Asynchronous operation

set

initiates

@crichardson

Benefits

Client does not know how the asynchronous operation is implemented

Client can invoke multiple asynchronous operations and gets a Future for each one.

@crichardson

Handling ProductDetails request

ProductDetailsController

ProductDetailsService

getProductDetails()

ProductInfoService

getProductInfo()

ReviewService

getReviews()

RecommendationService


RestTemplate

@crichardson

REST client using Spring @Async

@Componentclass ProductInfoServiceImpl extends ProducInfoService { val restTemplate : RestTemplate = ...

@Async def getProductInfo(productId: Long) = { new AsyncResult(restTemplate.getForObject(....)...) }

}

Execute asynchronously in

thread pool

A fulfilled Future

trait ProductInfoService { def getProductInfo(productId: Long):

java.util.concurrent.Future[ProductInfo]}

@crichardson

ProductDetailsService@Componentclass ProductDetailsService @Autowired()(productInfoService: ProductInfoService, reviewService: ReviewService, recommendationService: RecommendationService) {

def getProductDetails(productId: Long): ProductDetails = { val productInfoFuture = productInfoService.getProductInfo(productId)

}

}

val recommendationsFuture = recommendationService.getRecommendations(productId)val reviewsFuture = reviewService.getReviews(productId)

val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS) val recommendations = recommendationsFuture.get(10, TimeUnit.MILLISECONDS) val reviews = reviewsFuture.get(10, TimeUnit.MILLISECONDS)

ProductDetails(productInfo, recommendations, reviews)

@crichardson

ProductController

@Controllerclass ProductController @Autowired()(productDetailsService : ProductDetailsService) {

@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) =

productDetailsService.getProductDetails(productId)

@crichardson

Blocks Tomcat thread until Future completes

Not bad but...

class ProductDetailsService def getProductDetails(productId: Long): ProductDetails = {

val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS)

Not so scalable :-(

@crichardson

... and also...

Java Futures work well for a single-level of asynchronous execution

BUT #fail for more complex, scalable scenarios

Difficult to compose and coordinate multiple concurrent operations

See this blog post for more details:http://techblog.netflix.com/2013/02/rxjava-netflix-api.html

@crichardson

Better: Futures with callbacks ⇒ no blocking! val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") }

Guava ListenableFutures, Spring 4 ListenableFutureJava 8 CompletableFuture, Scala Futures

@crichardson

Even better: composable futures

def asyncOperation() = Future { ... ; 1 }

val r = asyncOperation().map{ _ * 2 }

assertEquals(4, Await.result(r, 1 second))

Scala, Java 8 CompletableFuture (partially)

Transforms Future

@crichardson

map() is asynchronous - uses callbacks

class Future[T] {

def map[S](f: T => S): Future[S] = { val p = Promise[S]()

onSuccess { case r => p success f(r) } onFailure { case t => p failure t }

p.future }

When ‘this’ completes propagate outcome to ‘p’

Return new Future

@crichardson

Chaining using flatmap()

def asyncOperation() = Future { ... ; 3 }

def asyncSquare(x : Int) : Future[Int] = Future { x * x}

val r = asyncOperation().flatMap { x => asyncSquare(x) }

assertEquals(9, Await.result(r, 1 second))


Chaining asynchronous operations

@crichardson

Combining futuresval f1 = Future { ... ; 1}val f2 = Future { .... ; 2}

val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))

def asyncOp(x : Int) = Future { x * x}val f = Future.sequence((1 to 5).map { x => asyncOp(x) })assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second))


Combines two futures

Transforms list of futures to a future containing a list

@crichardson

Scala futures are Monadsdef callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...

val result = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d

result onSuccess { .... }

Two calls execute in parallel

And then invokes D

Get the result of D

@crichardson

‘for’ is shorthand for map() and flatMap()

(callB() zip callC()) flatMap { r1 => val (b, c) = r1 callD(b, c) map { d => d }}

for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d

@crichardson

Scala Future + RestTemplateimport scala.concurrent.Future

@Componentclass ProductInfoService {

def getProductInfo(productId: Long): Future[ProductInfo] = { Future { restTemplate.getForObject(....) } }

}

Executes in thread pool

Scala Future

@crichardson

Scala Future + RestTemplateclass ProductDetailsService @Autowired()(....) {

def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId)

for (((productInfo, recommendations), reviews) <- productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) }

}

Non-blocking!

@crichardson

Async Spring MVC + Scala Futures

@Controllerclass ProductController ... {

@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = { val productDetails = productDetailsService.getProductDetails(productId)

val result = new DeferredResult[ProductDetails] productDetails onSuccess { case r => result.setResult(r) } productDetails onFailure { case t => result.setErrorResult(t) } result }

Convert Scala Future to

Spring MVCDeferredResult

@crichardson

Servlet layer is asynchronous but the backend uses thread

pools⇒

Need event-driven REST client

@crichardson

New in Spring 4

Mirrors RestTemplate

Methods return a ListenableFuture

ListenableFuture = JDK 7 Future + callback methods

Can use HttpComponents NIO-based AsyncHttpClient

Spring AsyncRestTemplate

@crichardson

Using the AsyncRestTemplate

http://hc.apache.org/httpcomponents-asyncclient-dev/

val asyncRestTemplate = new AsyncRestTemplate( new HttpComponentsAsyncClientHttpRequestFactory())

override def getProductInfo(productId: Long) = {

val listenableFuture = asyncRestTemplate.getForEntity("{baseUrl}/productinfo/{productId}", classOf[ProductInfo], baseUrl, productId)

toScalaFuture(listenableFuture).map { _.getBody }}

Convert to Scala Future and get entity

@crichardson

Converting ListenableFuture to Scala Future

implicit def toScalaFuture[T](f : ListenableFuture[T]) : Future[T] = { val p = promise[T]()

f.addCallback(new ListenableFutureCallback[T] { def onSuccess(result: T) { p.success(result)} def onFailure(t: Throwable) { p.failure(t) } }) p.future }

The producer side of Scala Futures

Supply outcome

Return future

@crichardson

Now everything is non-blocking :-)

@crichardson

If recommendation service is down...

Never responds ⇒ front-end server waits indefinitely

Consumes valuable front-end server resources

Page never displayed and customer gives up

Returns an error

Error returned to the front-end server ⇒ error page is displayed

Customer gives up

@crichardson

Fault tolerance at NetflixNetwork timeouts and retries

Invoke remote services via a bounded thread pool

Use the Circuit Breaker pattern

On failure:

return default/cached data

return error to caller

Implementation: https://github.com/Netflix/Hystrix

http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

@crichardson

How to handling partial failures?

productDetailsFuture zip recommendationsFuture zip reviewsFuture

Fails if any Future has failed

@crichardson

Handling partial failures

recommendationService. getRecommendations(userId,productId) .recover { case _ => Recommendations(List()) }

“catch-like” Maps Throwable to value

@crichardson

Agenda





@crichardson

Why solve this problem for JavaScript?

Browser invokes web services

Implement front-end server/API gateway using NodeJS

@crichardson

What’s NodeJS?

Designed for DIRTy apps

@crichardson

NodeJS

JavaScript

Reactor pattern

Modules

@crichardson

Asynchronous JavaScript code = callback hell

Scenarios:

Sequential: A ⇒ B ⇒ C

Fork and join: A and B ⇒ C

Code quickly becomes very messy

@crichardson

Callback-based HTTP client

request = require("request") handler = (error, clientResponse, body) -> if clientResponse.statusCode != 200 // ERROR else // SUCCESS

request("http://.../productdetails/" + productId, handler)

@crichardson

Messy callback codegetProductDetails = (req, res) -> productId = req.params.productId result = {productId: productId} makeHandler = (key) -> (error, clientResponse, body) -> if clientResponse.statusCode != 200 res.status(clientResponse.statusCode) res.write(body) res.end() else result[key] = JSON.parse(body) if (result.productInfo and result.recommendations and result.reviews) res.json(result)

request("http://localhost:3000/productdetails/" + productId, makeHandler("productInfo")) request("http://localhost:3000/recommendations/" + productId,

makeHandler('recommendations')) request("http://localhost:3000/reviews/" + productId, makeHandler('reviews'))

app.get('/productinfo/:productId', getProductInfo)

Returns a callback function

Have all three requests completed?

@crichardson

Simplifying code with Promises (a.k.a. Futures)

Functions return a promise - no callback parameter

A promise represents an eventual outcome

Use a library of functions for transforming and composing promises

Promises/A+ specification - http://promises-aplus.github.io/promises-spec

@crichardson

Basic promise API

promise2 = promise1.then(fulfilledHandler, errorHandler, ...)

called when promise1 is fulfilledreturns a promise or value

resolved with outcome of callback

called when promise1 is rejected

About when.jsRich API = Promises spec + use full extensions

Creation:

when.defer()

when.reject()...

Combinators:

all, some, join, map, reduce

Other:

Calling non-promise code

timeout

....

https://github.com/cujojs/when

@crichardson

Simpler promise-based code

rest = require("rest")

@httpClient = rest.chain(mime).chain(errorCode);

getProductInfo : (productId) -> @httpClient path: "http://.../productinfo/" + productId

Returns a promise

No ugly callbacks

@crichardson

Simpler promise-based code class ProductDetailsService getProductDetails: (productId) -> makeProductDetails = (productInfo, recommendations, reviews) -> details = productId: productId productDetails: productInfo.entity recommendations: recommendations.entity reviews: reviews.entity details responses = [@getProductInfo(productId), @getRecommendations(productId),

@getReviews(productId)] all(responses).spread(makeProductDetails)

Array[Promise] ⇒ Promise[Array]

@crichardson

Simpler promise-based code: HTTP handler

getProductDetails = (req, res) -> productId = req.params.productId

succeed = (productDetails) -> res.json(productDetails)

fail = (something) -> res.send(500, JSON.stringify(something.entity || something))

productDetailsService. getDetails(productId). then(succeed, fail)

app.get('/productdetails/:productId', getProductDetails)

@crichardson

Writing robust client code

@crichardson

Recovering from failures

getRecommendations(productId) .otherwise( -> {})

Invoked to return default value if getRecommendations() fails

@crichardson

Agenda





@crichardson

Let’s imagine you have a stream of trades

and you need to calculate the rolling

average price of each stock

@crichardson

Spring Integration + Complex event processing (CEP) engine is a good choice

@crichardson

But where is the high-level abstraction that solves this

problem?

@crichardson

Future[List[T]]

Not applicable to infinite streams

@crichardson

Introducing Reactive Extensions (Rx)

The Reactive Extensions (Rx) is a library for composing asynchronous and event-

based programs using observable sequences and LINQ-style query operators. Using Rx, developers

represent asynchronous data streams with Observables , query

asynchronous data streams using LINQ operators , and .....

https://rx.codeplex.com/

@crichardson

About RxJava

Reactive Extensions (Rx) for the JVM

Implemented in Java

Adaptors for Scala, Groovy and Clojure

https://github.com/Netflix/RxJava

@crichardson

RxJava core concepts

class Observable<T> { Subscription subscribe(Observer<T> observer) ...}

interface Observer<T> {void onNext(T args)void onCompleted()void onError(Throwable e)

}

Notifies

An asynchronous stream of items

Used to unsubscribe

@crichardson

Comparing Observable to...Observer pattern - similar but adds

Observer.onComplete()

Observer.onError()

Iterator pattern - mirror image

Push rather than pull

Future - similar but

Represents a stream of multiple values

@crichardson

So what?

@crichardson

Transforming Observables

class Observable<T> {

Observable<T> take(int n); Observable<T> skip(int n); <R> Observable<R> map(Func1<T,R> func) <R> Observable<R> flatMap(Func1<T,Observable<R>> func) Observable<T> filter(Func1<T,java.lang.Boolean> predicate)

...}

Scala-collection style methods

@crichardson

Transforming observables


<K> Observable<GroupedObservable<K,T>> groupBy(Func1<T,K> keySelector) ...}

Similar to Scala groupBy except results are emitted as items arrive

Stream of streams!

@crichardson

Combining observables


static <R,T1,T2> Observable<R> zip(Observable<T0> o1, Observable<T1> o2, Func2<T1,T2,R> reduceFunction)

...}

Invoked with pairs of items from o1 and o2

Stream of results from reduceFunction

@crichardson

Creating observables from data

class Observable<T> { static Observable<T> from(Iterable<T> iterable]); static Observable<T> from(T items ...]); static Observable<T> from(Future<T> future]); ...}

@crichardson

Arranges to call obs.onNext/onComplete/...

Creating observables from event sources

Observable<T> o = Observable.create( new SubscriberFunc<T> () { Subscription onSubscribe(Observer<T> obs) {

... connect to event source....

return new Subscriber () { void unsubscribe() { ... };

}; });

Called once for

each Observer

Called when unsubscribing

@crichardson

Using Rx Observables instead of Futures

@crichardson

Rx-based ProductInfoService@Componentclass ProductInfoService override def getProductInfo(productId: Long) = {

val baseUrl = ...

val responseEntity = asyncRestTemplate.getForEntity(baseUrl + "/productinfo/{productId}", classOf[ProductInfo], productId)

toRxObservable(responseEntity).map { (r : ResponseEntity[ProductInfo]) => r.getBody }

}

@crichardson

ListenableFuture ⇒ Observable

implicit def toRxObservable[T](future: ListenableFuture[T]) : Observable[T] = { def create(o: Observer[T]) = { future.addCallback(new ListenableFutureCallback[T] { def onSuccess(result: T) { o.onNext(result) o.onCompleted() }

def onFailure(t: Throwable) { o.onError(t) } }) new Subscription { def unsubscribe() {} } }

Observable.create(create _) }

Supply single value

Indicate failure

Do nothing

@crichardson

Rx - ProductDetailsService@Componentclass ProductDetailsService @Autowired() (productInfoService: ProductInfoService, reviewService: ReviewService, ecommendationService: RecommendationService) {

def getProductDetails(productId: Long) = { val productInfo = productInfoService.getProductInfo(productId) val recommendations =

recommendationService.getRecommendations(productId) val reviews = reviewService.getReviews(productId)

Observable.zip(productInfo, recommendations, reviews, (p : ProductInfo, r : Recommendations, rv : Reviews) =>

ProductDetails(p, r, rv)) }

}Just like Scala Futures

@crichardson

Rx - ProductController

@Controllerclass ProductController

@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = val pd = productDetailsService.getProductDetails(productId) toDeferredResult(pd)

@crichardson

Rx - Observable ⇒ DeferredResult

implicit def toDeferredResult[T](observable : Observable[T]) = { val result = new DeferredResult[T] observable.subscribe(new Observer[T] { def onCompleted() {}

def onError(e: Throwable) { result.setErrorResult(e) }

def onNext(r: T) { result.setResult(r.asInstanceOf[T]) } }) result }

@crichardson

RxObservables vs. Futures

Much better than JDK 7 futures

Comparable to Scala Futures

Rx Scala code is slightly more verbose

@crichardson

Making this code robust

Hystrix works with Observables (and thread pools)

But

For event-driven code we are on our own

@crichardson

Back to the stream of Trades averaging example...

@crichardson

Calculating averages

Observable<AveragePrice> calculateAverages(Observable<Trade> trades) { ...}

class Trade { private String symbol; private double price;

class AveragePrice { private String symbol; private double averagePrice;

@crichardson

Using groupBy()APPL : 401 IBM : 405 CAT : 405 APPL: 403

groupBy( (trade) => trade.symbol)

APPL : 401

IBM : 405

CAT : 405 ...

APPL: 403

Observable<GroupedObservable<String, Trade>>

Observable<Trade>

@crichardson

Using window()APPL : 401 APPL : 405 APPL : 405 ...

APPL : 401 APPL : 405 APPL : 405

APPL : 405 APPL : 405 APPL : 403

APPL : 405 ...

window(...)

Observable<Trade>

Observable<Observable<Trade>>

N secs

N secs

M secs

@crichardson

Using fold()

APPL : 402 APPL : 405 APPL : 405

APPL : 404

fold(0)(sumCalc) / length

Observable<Trade>

Observable<AveragePrice> Singleton

@crichardson

Using merge()

merge()

APPL : 401

IBM : 405

CAT : 405 ...

APPL: 403

APPL : 401 IBM : 405 CAT : 405 APPL: 403

Observable<Observable<AveragePrice>>

Observable<AveragePrice>

@crichardson

Calculating average pricesdef calculateAverages(trades: Observable[Trade]): Observable[AveragePrice] = {

trades.groupBy(_.symbol).map { symbolAndTrades => val (symbol, tradesForSymbol) = symbolAndTrades ...

tradesForSymbol.window(...).map { windowOfTradesForSymbol => windowOfTradesForSymbol.fold((0.0, 0, List[Double]())) { (soFar, trade) => val (sum, count, prices) = soFar (sum + trade.price, count + 1, trade.price +: prices) } map { x => val (sum, length, prices) = x AveragePrice(symbol, sum / length, prices) } }.flatMap(identity) }.flatMap(identity)}

@crichardson

RxJS / NodeJS examplevar rx = require("rx");

function tailFile (fileName) { var self = this;

var o = rx.Observable.create(function (observer) { var watcher = self.tail(fileName, function (line) { observer.onNext(line); }); return function () { watcher.close(); } }); return o.map(parseLogLine);}

function parseLogLine(line) { ... }

@crichardson

Rx in the browser - implementing completion

TextChanges(input) .DistinctUntilChanged() .Throttle(TimeSpan.FromMilliSeconds(10)) .Select(word=>Completions(word)) .Switch() .Subscribe(ObserveChanges(output));

Your Mouse is a Databasehttp://queue.acm.org/detail.cfm?id=2169076

Ajax call returning Observable

Change events ⇒ Observable

Display completions

@crichardson

Summary

Consuming web services asynchronously is essential

Scala-style are a powerful concurrency abstraction

Rx Observables are even more powerful

Rx Observables are a unifying abstraction for a wide variety of use cases

@crichardson

Questions?

@crichardson [email protected]

http://plainoldobjects.com

Technology

Consuming web services asynchronously with Futures and Rx Observables (svcc, svcc2013)