SpringOne2GX 2014 Day 2 Keynote: Platform Reinvented

Unless otherwise indicated, these slides are © 2013-2014 Pivotal Software, Inc. and licensed under a

Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/



Day 2 Keynote: Platform Reinvented

Juergen Hoeller, Mark Fisher, Dave Syer, Jon Brisbin

1



Spring IO Platform

2

Jobs, Steps,

Readers, Writers

Ingestion, Export,

Orchestration, Hadoop

Controllers, REST,

WebSocket

Channels, Adapters,

Filters, Transformers

WEB INTEGRATION BATCH BIG DATA

SPRING CORE

FRAMEWORK SECURITY GROOVY REACTOR

DATA

RELATIONAL

DATA ACCESS

NON-RELATIONAL

DATA ACCESS

BOOT

Bootable, Minimal, Ops-Ready

GRAILS Full-stack,

Web

XD Stream, Taps,

Jobs

IO EXECUTION

IO FOUNDATION

IO COORDINATION SPRING CLOUD



Spring IO Core: Spring Framework 4.1 GA

• Annotated JMS listener methods

• Comprehensive support for JCache (JSR-107) annotations

• Flexible resolution and transformation of static web resources

• MVC views: declarative resolution + Groovy markup templates

• Performance: SpEL compiler mode + concurrency fine-tuning

3



Spring 4.1: Annotated JMS listener methods

4

@Component

public class MyService {

@JmsListener(destination = "myQueue")

public void processOrder(Order order, @Header("order_type") String type) {

...

}

}



Spring 4.1: Support for JCache annotations

5

@Component

public class MyService {

@CacheResult(cacheName = "book")

public Book findBook(@CacheKey ISBN isbn, boolean checkWarehouse) {

...

}

}



Spring 4.1: Flexible web resource handling

• Flexible resource resolution chain:path resolution, caching,

compression, versioning

• Flexible resource transformation chain:CSS links, caching,

HTML5 AppCache manifests

• Talk: “Resource Handling in Spring MVC 4.1”

by Brian Clozel and Rossen Stoyanchev - Wednesday 8:30

• Showcase: https://github.com/bclozel/spring-resource-handling

6



Spring 4.1: Declarative MVC view resolution

7

@Configuration

@EnableWebMvc

public class WebConfig extends WebMvcConfigurerAdapter {

public void configureViewResolvers(ViewResolverRegistry registry) {

registry.enableContentNegotiation(new MappingJackson2JsonView());

registry.freeMarker().cache(false);

}

}



Spring 4.1: Many further improvements...

• Java 8's Optional for injection points and MVC handler

parameters

• Annotated @Lookup methods with support for method arguments

• Direct field binding support with auto-growing for nested fields

• WebSocket scope, SockJS client support, WebSocket stats

• Declarative SQL scripts and programmatic transactions for tests

8



Spring XD – Big Data, Little Time

9



Spring XD: Unified Platform for Big Data

10

Spring XD Runtime

BIDIRECTIONAL

Compute

HDFS

RDBMS

NoSQL

R, SAS

Streams Jobs

ingest workflow

export

taps

Predictive Modelling

>_

Redis



Spring XD: Quality of Service

• High Availability

• Cluster Management built upon ZooKeeper

• Leader Election for Admin Nodes

• Module Redeployment across Container Nodes

• Customizable Deployment

• Module Count and Criteria for placement

• Data Partitioning by Key

• Direct Binding for Co-located Modules

11



Demo Domain: Smart Grid Data

12

http://www.cse.iitb.ac.in/debs2014 *

Inspired by ACM DEBS 2014 Grand Challenge* Demonstrate the applicability of event-based systems to provide scalable, real-time analytics over high volume sensor data to compute load forecasting.

Field Name Description

ID Unique ID of the measurement

Timestamp Number of seconds since epoch

Load Load in watts

House ID The house where the plug is located

Household ID The household inside the house

Plug ID The unique ID of the smart plug



Demo Objectives

• Ingest household energy data to HDFS via HTTP

• Real-time aggregation and visualization

• Real-time model evaluation to predict energy demand

13

ingest: http | hdfs

count: tap:stream:ingest > aggregate-counter

predict: tap:stream:ingest > pmml | aggregate-counter

partition on house



XD Admin

Zoo Keeper Zoo Keeper Zoo Keeper

Load

Balancer Generator

XD Container

HTTP

XD Container

HTTP

XD Container

HDFS

XD Container

HDFS

XD Container

HDFS Hadoop

Hadoop

Hadoop

Rabbit Redis

Demo Topology



Demo:



Spring XD

15



Looking Ahead

• Programming Model

• Further Unification of the Batch and Stream Models

• Reactive Streams

• Developer Experience

• Support for Java Config and Spring Integration DSLs

• Spring Boot-based Module Deployment and Packaging

• Deployment Targets

• Cloud Foundry Service

• Docker

• Mesos

16



Spring XD Sessions

17

TUE 2:30 PM Develop powerful Big Data Applications easily with Spring XD

TUE 4:30 PM Spring XD - A Guided Tour

WED 10:30 AM Building a Recommendation Engine with Spring and Hadoop

WED 2:30 PM IoT Realized - The Connected Car

THU 10:30 AM Spring XD for Real-time Hadoop Workload Analysis

THU 12:45 PM Implementing the Lambda Architecture with Spring XD



Spring IO Platform

18

Grails

Spring IO Platform Distribution

Spring XD

Spring Foundation

Apps

3rd Party Dependencies

Spring Boot

Spring Cloud



No Man (Microservice) is an Island

It's excellent to be able to implement a microservice really easily

(Spring Boot), but building a system that way surfaces

"non-functional" requirements that you otherwise didn't have.

There are laws of physics that make some problems unsolvable

(consistency, latency), but brittleness and manageability can be

addressed with generic, boiler plate patterns.

19



Emergent features of micro-service systems

Coordination of distributed systems leads to boiler plate patterns

• Distributed/versioned configuration

• Service registration and discovery

• Routing

• Service-to-service calls

• Load balancing

• Circuit Breaker

• Asynchronous

• Distributed messaging

20



Spring Cloud Components

21

Client Bindings

Service

Discovery

Config

Server

Circuit

Breakers

Edge

Service

Intelligent

Router

Leader

Election

One-time

Tokens

Global

Locks

Cluster

State

Event Bus



Demo:



Spring Cloud

Components

22



Spring Cloud Components

• Common features of distributed systems developed by users as

Spring Boot apps

• Highly configurable and customizable

• Always do something sensible out of the box

• Works nicely as standalone apps on developer box

• Easy migration to Cloud and PCF

23



Spring Cloud Sessions

24

TUE 4:30 PM Security for Microservices with Spring and OAuth2

WED 12:45 PM Simplify Cloud Applications using Spring Cloud

WED 12:45 PM Spring Cloud, Spring Boot and Netflix OSS

WED 2:30 PM Developing Microservices for PaaS with Spring and Cloud Foundry



Demo:



Reactor

25



Spring IO Core: Reactor

• Microservices proliferate in modern architectures

• More Runtime Resources required

• More Runtime Resources means more $$$

• Microservices have acute need for efficiency

• Runtime Resources aren’t free/cheap

• Microservice Architecture introduces boundaries

• Boundaries introduce latency

26




• What is Reactive Programming?

• Event-driven

• Modular

• Scalable

• Highly-available

• Non-blocking/async

o Blocking IO (BIO) is expensive

o Non-blocking IO (NIO) is more efficient

o NIO means async patterns, which can be hard

27




Microservices are natural fit for Reactive Architecture

28


Creative Commons Attribution-NonCommercial l icense: http://creativecommons.org/licenses/by-nc/3.0/ 29

Reactive Architecture

• Reactive is usually Asynchronous

• Asynchronous !== Concurrent

• Non-blocking versus Blocking (usually IO)

• Provides Isolation

• Resource usage

• Failure

• Events are replayable

• Reactive can be complex

• Too low-level for most end users



Reactor

Spring IO: Components → Reactor

30

Task Queue

Micro

Service

Micro

Service

Micro

Service

Micro

Service

Event

Event Event



Reactive Stream Contributors

31

ccv

Doug Lea – SUNY Oswego




What is Reactive Streams?

32

“...provide a standard for asynchronous stream processing with

non-blocking backpressure.”

github.com/reactive-streams/reactive-streams

Available as a library

org.reactivestreams:reactive-streams:0.4.0.M2



Reactive Sessions

33

WED 12:45 PM Asynchronous design with Spring and RTI: 1M events per second

THU 8:30 AM Building Reactive applications with Reactor and the Reactive Streams

standard