HUG Italy meet-up with Fabian Wilckens, MapR EMEA Solutions Architect

© 2016 MapR Technologies 1© 2016 MapR Technologies 1MapR Confidential

© 2016 MapR Technologies

Fabian WilckensEMEA Solutions Architect

THINK DIFFERENTLYStream-based Microservice Architecture for Next-Generation Applications

© 2016 MapR Technologies 2© 2016 MapR Technologies 2MapR Confidential

© 2016 MapR Technologies 3© 2016 MapR Technologies 3MapR Confidential

© 2016 MapR Technologies 4© 2016 MapR Technologies 4MapR Confidential

© 2016 MapR Technologies 5© 2016 MapR Technologies 5MapR Confidential

The microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API. These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages and use different data storage technologies.

- Martin Fowler

MICROSERVICE ARCHITECTURAL STYLESINGLE APPLICATION

SUITE OF SMALL SERVICESCOMMUNICATING WITH LIGHTWEIGHT MECHANISMS

BUILT AROUND BUSINESS CAPABILITIESFULLY AUTOMATED

DEPLOYMENTBARE MINIMUM OF CENTRALIZED MANAGEMENTWRITTEN IN DIFFERENT PROGRAMMING LANGUAGES

DIFFERENT DATA STORAGE TECHNOLOGIES

© 2016 MapR Technologies 6© 2016 MapR Technologies 6MapR Confidential

WAIT A SECOND

SOA?SERVICE ORIENTED ARCHITECTURE

IS THERE A DIFFERENCE?

© 2016 MapR Technologies 7© 2016 MapR Technologies 7MapR Confidential

Integrates different applications as a set of services

Architect a single application as a set of services

APP 1 APP 2

APP 4 APP 5

ENTERPRISE SERVICE BUS

APP 2

LOGISTICSSERVICE

CUSTOMERSERVICE

BILLINGSERVICE

USER INTERFACE SERVICE

MICROSERVICE-STYLE

CLASSIC SOA-STYLE

© 2016 MapR Technologies 8© 2016 MapR Technologies 8MapR Confidential

A few years back …

© 2016 MapR Technologies 9© 2016 MapR Technologies 9MapR Confidential

App Server

WAR/EAR

Backend

Logistics

Ordering

User Interface

Easy to develop? Easy to deploy? Easy to scale?

© 2016 MapR Technologies 10© 2016 MapR Technologies 10MapR Confidential

CHALLENGES AHEAD

© 2016 MapR Technologies 11© 2016 MapR Technologies 11MapR Confidential

LARGE CODE & BIG DATAHARD TO UNDERSTAND & CHANGE

OVERLOADED IDE

OVERLOADED WEB CONTAINERS

DEVELOPMENT SLOWS DOWN

EVEN SMALL APPLICATION CHANGES REQUIRE FULL REBUILD, TEST AND DEPLOYMENT

IMPACT ANALYSIS IS A HUGE EFFORT AND TAKES TIME

DIFFICULT TO FREQUENTLY CHANGE & DEPLOY

© 2016 MapR Technologies 12© 2016 MapR Technologies 12MapR Confidential

OTHER CHALLENGESLONG-TERM COMMITMENT TO TECHNOLOGY STACK

RE-WRITE = COMPLETE RE-WRITE

MODULE DEPENDENCIES FORCE RE-WRITE

FAILURE IN MONOLITHIC APPLICATIONS BRINGS IT DOWN

© 2016 MapR Technologies 13© 2016 MapR Technologies 13MapR Confidential

HORIZONTAL SCALING

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

© 2016 MapR Technologies 14© 2016 MapR Technologies 14MapR Confidential

VERTICAL SCALING

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App Server

WAR/EAR

Backend

Logistics

Ordering

User Interface

© 2016 MapR Technologies 15© 2016 MapR Technologies 15MapR Confidential

DATA SCALING

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

App ServerWAR/EAR

Backend

Logistics

Ordering

User Interface

© 2016 MapR Technologies 16© 2016 MapR Technologies 16MapR Confidential

© 2016 MapR Technologies 17© 2016 MapR Technologies 17MapR Confidential

MICROSERVICES SCALING

Container

Ordering

Container

Ordering

Container

Ordering

Ordering Database

Container

User Interface

Container

Logistics

LogisticsDatabase

Container

Backend

BackendDatabase

© 2016 MapR Technologies 18© 2016 MapR Technologies 18MapR Confidential

MICROSERVICES TEAM SCALING

Container

Ordering

Container

Logistics

Container

Backend

© 2016 MapR Technologies 19© 2016 MapR Technologies 19MapR Confidential

LET’S TALK BENEFITS (PROS)ONLY MAKE APIS TALK TO EACH OTHER

ADD MODULES BASED UPON BUSINESS REQUIREMENTS

CHANGE MODULES INDEPENDENTLY

DIFFERENT TEAMS WORK ON THEIR OWN PROBLEMS

FASTER TIME-TO-MARKET FOR NEW SERVICES

CAN STILL INTEGRATE WITH SOA

SERVICES/TEAMS SCALE INDEPENDENTLY

…

© 2016 MapR Technologies 20© 2016 MapR Technologies 20MapR Confidential

© 2016 MapR Technologies 21© 2016 MapR Technologies 21MapR Confidential

© 2016 MapR Technologies 22© 2016 MapR Technologies 22MapR Confidential

The Challenge with Data Pipelines

Filtering & Aggregation

Alerting Processing

© 2016 MapR Technologies 23© 2016 MapR Technologies 23MapR Confidential

Streams Simplify Data Movement

Filtering & Aggregation

Alerting Processing

StreamsReliable publish/subscribe transport between sources and destinations.

© 2016 MapR Technologies 24© 2016 MapR Technologies 24MapR Confidential

Legacy Systems: Message QueuesIBM MQ, TIBCO, RabbitMQ

OrdersFront End

Order Processing

Order Processing

Usage/Requirements●Tight, transactional conversations between systems

●1:1 or Few:Few

●Low data rates

●Mission-critical delivery

Approach

●Queue-oriented design

●Each message replicated to N output queues

●Messages popped when read

●Scale-up, master/slave

Doesn’t Do

●High message rates (>100K/s)

●Slow consumers

●Queue replay/rewind

© 2016 MapR Technologies 25© 2016 MapR Technologies 25MapR Confidential

Evolving “big data” Event Streams: Distributed LogsKafka, Hydra, DistributedLog

Usage/Requirements●High throughput data transferred from decoupled systems

●Many->1

●1->Many

●Different speeds

Approach

●Log-oriented design

●Write messages to log files

●Consumers pull messages at their own pace

●Scale-out

Doesn’t Do

●Global applications

●Message persistence

●Integrated analytics (data movement required)

DB_Changes

Stream Processing

Search/EDW

DB

© 2016 MapR Technologies 26© 2016 MapR Technologies 26MapR Confidential

Introducing MapR StreamsGlobal Publish/Subscribe Event Streaming

Producers Publish Billions of

messages/sec to a topic

ConsumersReliable delivery to all

consumers. Immediately

GlobalTie together geo-dispersed

clusters. Worldwide

© 2016 MapR Technologies 27© 2016 MapR Technologies 27MapR Confidential

MapR Converged Data Platform

Tenant 2 Tenant 3

Tenant 1

© 2016 MapR Technologies 28© 2016 MapR Technologies 28MapR Confidential

Container

Ordering

Container

User Interface

Container

Logistics

Container

Backend

© 2016 MapR Technologies 29© 2016 MapR Technologies 29MapR Confidential

Global Applications

Container

Ordering

Container

Backend

Container

Logistics

Container

User Interface

Container

User Interface

Container

User Interface

© 2016 MapR Technologies 30© 2016 MapR Technologies 30MapR Confidential

Are Microservices here to stay?

We don’t know …Monolithic Apps are dead

© 2016 MapR Technologies 31© 2016 MapR Technologies 31MapR Confidential

Content & Pictures• http://enterprise.vnews.com/wp-content/uploads/2015/07/EP-SeanSmithCover-ns-vn-0728151.jpeg• http://www.rodmatthews.com.au/_assets/images/blog/Positional%20Images/Complexity%20Small%20

Version.jpg• http://www.coliquio-insights.de/wp-content/uploads/2015/02/5141328136_16d8f73ee7_b.jpg