Embed Size (px)
Citation preview
© Solace SystemsCONFIDENTIAL
Sumeet PuriVP, Head of Technology, International
The Impact of Messaging Standards on Event-Driven Architecture and IoT
2CONFIDENTIAL
Diversified Blue-Chip Customer Base
Disruptive Products in Rapidly Growing Market
Solace Systems Company Overview
Better Faster Cheapero Solace Systems is a provider of
hardware and software end-to-end message routers
o Solace hardware and software appliances are widely deployed in various Banking, Payments, Telco, IoT, Insurance, Digital and Big Data Systems
o Solace appliances are used for Simplicity, Robustness (HA & DR), Performance and Lower TCO
o Founded in 2001, HQ in Ottawa, Canada
In Only
Out In
In Out
Out Only
Data Movement Patterns – Magic Quadrant
Imagine there were no TCP/IP
Still, the missing data movement pieces:
Pub Sub | Persistence | Shock Absorption | App Throttling
Messaging Standards
Networking : Messaging :: TCP/IP : MQTT/AMQP…JMS
5CONFIDENTIAL
Messaging as a Digital BackboneNot Just Messaging but Data Movement
Omni Channel - Mobility
Straight Through
Processing
Integrating Micro services - Faster Time
To Market
Big Data In Motion
Event Driven Architecture
RESTAPI GWsNo Web ServerData Movement
Wiretap for AnalyticsSlow Consumer Handling
FanoutLow LatencySync andAsync
REST <-> JMS is a big dealREST AssuredMulti protocolSimple Architecture
Easy, Fully IntegratedAdministration & Security
Real-Time & Very High Volume
6CONFIDENTIAL
Aspects of a Digital | IoT EcosystemThe Architecture CubeIn
fras
truc
ture
Fog/ Gateway Layer
Cloud/ Datacenter Edge Layer
Cloud/ Datacenter Core Layer
Func
tiona
l
Capture
Analyze
Store
React
Predict
CICD
& D
evic
e M
anag
emen
t
Security
Network
Fog Cloud Edge Core
Messaging
Multi Protocol Standards
Aggregation
Throttling
Connection Scaling
Filtering & Routing
Fan in -Burst Handling
Persistence
Slow Consumer Handling
Fanout – Lambda Stream
Things & Apps
Fog Computing
Legacy Integration
Real Time Analytics
GUIs and Dashboards
Legacy Gateways
Business Logic & Integration
[Big] Data Analytics
[Big] Data Store
Man
agem
ent
and
Mon
itorin
g
7CONFIDENTIAL
The Waiter Pattern - Sync Waiter Restaurants Don’t Work This Way!!
Take Order
Go To Kitchen
Wait for Dish
Serve
8CONFIDENTIAL
The Waiter Pattern – The Efficient, Normal Waiter
o Waiters take the order, validate and move to the next table
o Food is served via a callback
o This proven pattern is the only one with which mobile apps scale
So why do ESBs/API GWs work differently? Because they were built with HTTP shortcuts
Take Next Order
Submit To Kitchen
Dish Ready?
Serve
Async
9CONFIDENTIAL
Order Taking – The SOA WayAn order is an order is an order – Payment, Telco, Mining
250ms 2s 2s 3s
DWHReportingBusiness Logic
Validations and Check
Response Time = ~8s if you are lucky
ESB/API Stack – Everything Synchronous, over SOAP/HTTP
Any problem in the kitchen has direct user impactPoor throttling, load balancing, no isolation
Task Time
1 Submit Income
2 Validate .25s
3 Business Logic 2s
4 Reporting 2s
5 Data Ware House 3s
6 Total Response Time ~8s
6
12 3 4 5
10CONFIDENTIAL
Waiter Pattern – The Event Driven WayMuch Higher Throughput, Much Better Response Time and Stability
250ms 1s 2s 2s
DWHReportingBusiness Logic
Validations and Check
Response Time = 300ms
Kitchen
Order Taking and AckShock
Absorber
Task Time
1 Submit Income
2 Validate .25s
3 Reply and Queue 20us
Total Response Time 300us
4 Business Logic 2s
5 Reporting 2s
6 Data Ware House 3s
Kitchen Decoupled = Happy CustomerThrottling, Load Balancing, Slow Consumer Handling
6
1
2
3
4 5
Topic to Q bridging
Waiter
11CONFIDENTIAL
Use Case - Relationship Managers Tab
Pub Sub
Req Rep
Analytics Core Banking
ESB (if needed)
DWH
Market Data TOMS
12CONFIDENTIAL
Use Case - Messaging as a Digital BackboneBanking, Trading, Payments Straight Through Processing
JSON/REST or JMS/MQTT
Legacy Gateway
REST message: URL for event routing
http://mybank/pay/merchId/curr/type/xyz/xyz0101011000101010111101010101001010101011010
Data In Motion Complex Events
ProcessingFraud Detection
JMS
Topi
c:my
bank
/pay/>
Visual Analytics (Splunk)
JMS
Topi
c to
Que
ue:
myba
nk/POS
/Mo
nitor/
>Lo
ssle
ss Q
Sho
ck A
bsor
ber
Solace
Data At Rest Analytics (Hadoop)
JMS
Topi
c:my
bank/>
Lossless QShock Absorber
Topic to Q mapping
Core Business
Logic
ConnectivityData Store
CacheESB/Transform
ESB/Transform
JMS Topic:mybank/pay/*/fx/>
mybank/pay/*/aud/>
Wildcard routing on REST URLs/Topics
Filtering and Topic (subject) Routing
with WildcardsPersistence – never
lose a message
Publish Subscribe - Parallel, Async
Processing
Multi ProtocolREST <> JMS <>AMQP
<> MQTT
Burst Handling and Throttling – Big Data
Ingest
Lambda Architecture – Speed and Batch
13CONFIDENTIAL
Messaging Backbone for Hybrid CloudAllows Cloud Transformation – Move gradually to the cloud
PaaS – Private Cloud
IaaS
BoshProvisioning
ServiceBroker
MaaSCatalog
App App App
App
App
App
App
CustomApps
ApplicationPlatforms
Oracle OSB
WebSphereApp Server
DataPower
TIBCO BW
Jboss Fuse
MuleSoft
App
App
App
App
App
AppAppApp App AppApp Application portability
Ubiquitous Connectivity Best-of-breed Messaging
App
14CONFIDENTIAL
B2B
DMZ
Use CaseBanking Event-Driven Architecture
Big “Data Lake” Repository &
Real-Time Analytics
Retail Bus
MortgagesCredit Cards Insurance Wealth Mgmt
Domain Service Bus
Wholesale
MobileOnline
Micropayments
• Bank Machines• Branches• Telephone
DedicatedNetworks
Central Services
IntegrationServices
DataGovernance
Business Benefits• Agility• Low TCO• Robustness
• Innovation• Time to Market• Realtime services
Orchestration
Realtime 360 degree Customer View
DR
Payments
PublicCloud
PrivateCloud
15CONFIDENTIAL
Smart Cities: Bus Monitoring
REST/MQTT
Regulator’s Dashboard
Commuter’s Apps
and RIAs
Analytics DWH
ESB (if needed)
CEP
JMS
Legacy REST
Edge CoreFog
Legacy Gateway
REST/MQTT
Use
Cas
e
o Collect real-time location, loading information and more from 5,000 buses
o Optimize bus movement, fleet management, avoid bunching, forecast arrival time and more
Why
Sol
ace
o Scale: smaller footprinto Reliability and robustnesso Integration with TIBCO ESB
and CEPo Low Latency for predictive
analyticso Native support for REST,
MQTT, JMS and more without any bridges
EdgeCore
16CONFIDENTIAL
Smart Cities: LTA Next Generation ERP EdgeCore
Routing Server
Payments
Analytics
0 0 0
• 200K Concurrent Connections
• MQTT Termination
• Request/reply forfirmware upgrades
• Push Notifications
• Persistent Messaging
• Shock Absorber
• High Throughput
• Fanout
• Slow Consumer Handling
MQTT JMS
Edge CoreFog
Use
Cas
e
o Connect 1.5M vehicles (200K at a time) in real time for pay per use road tolls
o Suggest better routes, traffic management – real time and long term and value added services/analytics
Why
Sol
ace
o Smaller footprinto Reliability and robustnesso Subscription management,
MQTT request reply, topic to queue bridging, fanout
o Advanced security and monitoring/event generation
o Cost half ½ of competition (IBM)
MHI OBU
17CONFIDENTIAL
Smart Cities: Building/Asset MonitoringPreventive Maintenance
Solace Bridge
o Collect information from building sensors – Preventive Maintenance and Optimization
o Lift Open/Close, Lighting, Water, Electricity etc
o Limited block pilot, to scale to all blocks
Edge
Fog
Core
Modern Sensors
Aggregator – Collect data from legacy sensors at Residential Blocks.
Building Block
REST/MQTT
ES
B
Core Business Logic
(Event Driven)
SchedulingERP
Big Data Store and Analytics
Edge CoreFog
Use
Cas
e
o Getting real time/batch data from Housing buildings, such as lift data, pipelines, electricals etc.
o For security and preventive maintenance
Why
Sol
ace
o Single fabric for Fog, Edge and Core
o Scalability – easy to scale the pilot to all blocks
o Aggregation per block as needed, every block is fiber connected and has a ruggedized server
o Legacy bridging with ESBso Feeding Big Data store for
longer term predictive maintenance and cost/contract optimization
o Real time security feeds as necessary, such as roof lock compromise
18CONFIDENTIAL
Use
Cas
e
o Real time QC Data sourcing and analytics to reduce recall times
o Same data bus for bidirectional contract manufacturer communication
Why
Sol
ace
o Moving from DB-driven bus to Solace cuts recall time from days to minutes
o WAN: Move data from 21 factories to HQ using less bandwidth, with lower latency and no loss
o Ability to easily add big Data ingestion
o Performance, speed and simplicity
Smart Manufacturing (Industry 4.0): Real Time Movement of QC Data
MESmes/main mes/measure
MAINmes/equip
Master Systemmes/master
DB
History Systemmes/history
DB
OIP ENG’RClient PCClient Mobile
Solace APIjava
Solace APIC
Solace APIC#
Manufacturer
Data At Rest Analytics
Real Time Analytics (Recall)
WAN Optimized Lossless Data
Transfer
Head Quarters
QC DataDemand DataControl Data
Visual Analytics
So-lace
Solace APIHTML5
EdgeCore
19CONFIDENTIAL
G2B
DMZ
Use CaseSmart Cities Event-Driven Architecture
Big “Data Lake” Repository &
Real-Time Analytics
Home AffairsDefence Environment and Industry
Housing and Buildings
Infocom Dev Authority
MobileOnline
G2C
• Kiosk Machines• Departments• Telephone
DedicatedNetworks
Central Services
IntegrationServices
DataGovernance
Business Benefits• Agility• Low TCO• Robustness
• Innovation• Time to Market• Realtime services
Orchestration
Realtime 360 degree Customer View
DR
Land Transport
GovCloud
PrivateCloud
20CONFIDENTIAL
ESB - ADAPTERS / TRANSFORMATION (only if needed)
Use Case – Telco – Never Miss an OrderCustomer 360 via a Digital Event Backbone
NGIN
Service ConsumersProcess LayerCEP, Analytics, Big Data Lakes etcNew systems can be added with standard protocols
Core Business LogicSouth Bound Systems
Presentation LayerNorth Bound SystemsMobile/DevicesThick ClientBrowser/RIASensors
PCRF
Mobile First and
Consumer Apps
Glue Service LayerESBOther Frameworks
New systems just connects and publish/subscribe events
MDMPostPaid BillingCore
Systems
CEPFraudDetection
Event Data Subscribers
IN-MemoryData GridSecurity
Monitoring
Governance
Messaging +API Gateway
Data River – Solace Enterprise Message Bus
HTTP, REST, mqtt
Data Lakes
B2B LinksOmni
ChannelsAdded
seamlessly
21CONFIDENTIAL
B2B
MQTT, REST, Flat Files,SFTP, etc.
DMZ
Use CaseTelco Event-Driven Architecture
Enterprise Messaging Bus
Web & MobileStreaming,
JavaScript, REST etc.
MobileOnline
Customer 360
• TopUp Machines
• Branches
DedicatedNetworks
Central Services
Integration Services
Governance and Analytics
Orchestration &Transformation
Benefits of Solace-Based Bus• Simplicity: Single HA pair vs many
distributed software brokers, single software load vs multiple from different vendors, forcing security patch updates etc.
• Pub/Sub functionality: Complete separation of pubs from subs, full Event Driven Architecture
• Robustness: Slow consumers handling, fast failover
• Scalability/Performance: One appliance pair meets growing needs for years
• Manageability: Always on monitoring, centralized security
JavaJMS REST
DR
PublicCloud
HDFS
PrivateCloud
PCRF NGIN APIGateway
Provisioning
Mobile First Platform
Server
CBS CEP IN-MemoryCache
NODE 1
NODE 2
22CONFIDENTIAL
Why Solace?o Zero Down Time, Zero
Message Loss‐ Active Active DR
o Micro Second Latency at Very High Throughput
o Standards – APIs, REST, MQTT, JMS, AMQP on the way – interoperable at wireline level
o Experience and References – Digital, Cloud, IoT, Big Data
o Simplicity and Lower TCO
Primary Data Center
Backup Data Center
SAN
SAN
WAN
DR Link TCP Connection
23CONFIDENTIAL
REST Assured!Zero Downtime, Zero Message LossLossless REST - Never lose an OrderSimple – No Web/App ServersPerformance & ReliabilityStandards Based, Cloud Ready
