View
259
Download
0
Category
Preview:
Citation preview
Luis Ángel Vicente Sánchez Software Engineer @ Innovation Team
OMNIA DISTRIBUTED & REACTIVE PLATFORM FOR DATA MANAGEMENT
Site https://developer.williamhill.com
Twi$er https://twitter.com/lvicentesanchez
• Lots of information flowing in the system.
• What if we could make sense of all of it...?
• ... and on real-time?
• We would be in a great position to innovate.
WHY BUILDING A DATA MANAGEMENT PLATFORM?
REACTIVE MANIFESTO Respondina+melymanner
Stayresponsiveunderdifferentworkloads Stayresponsiveinfaceoffailures
REACTIVE MANIFESTO Respondina+melymanner
Stayresponsiveunderdifferentworkloads Stayresponsiveinfaceoffailures
Loosecoupling,isola+on,loca+ontransparency,anderrorhandlingdelega+on
• It’s a lambda architecture.
OMNIA
InputC
hron
osD
ata
Sour
ce
NeoCortexSpeed Layer
FatesBatch Layer
Her
mes
Serv
ing
Laye
r Output
• Collects data from multiple data sources.
• Initially a PUSH system…
• ... but that introduces dependencies on 3rd parties.
• Evolved to a PUSH-PULL system.
CHRONOS
CHRONOS
Adapter Converter
STREAM
PUSH:• HTTP• TCP• WEBSOCKETS
PULL:
• HTTPPOLL• JMS• SSE
INCIDENT: { type: “bet”, version: “1.0”, time: “2015-09-03 06:00:10”, acquisitionTime: “2015-09-03 06:00:06”, source: “BetSystem”, payload: {…. Any valid JSON …} }
CHRONOS
Adapter Converter
STREAM
PUSH:• HTTP• TCP• WEBSOCKETS
PULL:
• HTTPPOLL• JMS• SSE
INCIDENT: { type: “bet”, version: “1.0”, time: “2015-09-03 06:00:10”, acquisitionTime: “2015-09-03 06:00:06”, source: “BetSystem”, payload: {…. Any valid JSON …} } O
bservable[Incide
nt]
CHRONOS
Adapter Converter
STREAM
PUSH:• HTTP• TCP• WEBSOCKETS
PULL:
• HTTPPOLL• JMS• SSE
INCIDENT: { type: “bet”, version: “1.0”, time: “2015-09-03 06:00:10”, acquisitionTime: “2015-09-03 06:00:06”, source: “BetSystem”, payload: {…. Any valid JSON …} } O
bservable[Incide
nt]
CHRONOS
Adapter Converter Persistor
STREAM
PUSH:• HTTP• TCP• WEBSOCKETS
PULL:
• HTTPPOLL• JMS• SSE
INCIDENT: { type: “bet”, version: “1.0”, time: “2015-09-03 06:00:10”, acquisitionTime: “2015-09-03 06:00:06”, source: “BetSystem”, payload: {…. Any valid JSON …} } O
bservable[Incide
nt]
CHRONOS
Adapter Converter Persistor
STREAM
PUSH:• HTTP• TCP• WEBSOCKETS
PULL:
• HTTPPOLL• JMS• SSE
INCIDENT: { type: “bet”, version: “1.0”, time: “2015-09-03 06:00:10”, acquisitionTime: “2015-09-03 06:00:06”, source: “BetSystem”, payload: {…. Any valid JSON …} } O
bservable[Incide
nt]
CHRONOS
Chronos 1(SSE, Bets placed)
(SSE, Twitter)
Chronos 2(JMS, Deposits)
Chronos 3(HTTP, Events)
Chronos N
….…
CHRONOS
Chronos 1(SSE, Bets placed)
(SSE, Twitter)
Chronos 2(JMS, Deposits)
Chronos 3(HTTP, Events)
Chronos N
….…
CHRONOS
Chronos 1(SSE, Bets placed)
(SSE, Twitter)
Chronos 2(JMS, Deposits)
Chronos 3(HTTP, Events)
Chronos N
….…
CHRONOS
Chronos 1(SSE, Bets placed)
(SSE, Twitter)
Chronos 3(HTTP, Events)
Chronos N(SSE, Twitter)
….…
Chronos 2(JMS, Deposits)
(SSE, Bets placed)
• Long term memory.
• Records all incidents as timelines, i.e. customer bets, customer payments, everything that happens in a match…
• … or any combination of those.
FATES
FATES
Customer: 123
Login
Deposit
Bet placed
…
LogoutTimelines
Bets DepositsEvents Session
FatesBatch Layer
FATES
Customer: 123
Login
Deposit
Bet placed
…
Logout
Event: 78
Started
Bet placed
Penalty
…
GoalTimelines
Bets DepositsEvents Session
FatesBatch Layer
• Creates views using the timelines and/or other views.
• These views are created using: – Timeline analysis – Logical reasoning… – ... or any other ML technique
FATES
FATES
LogicalReasoning• DeducDon• InducDon• AbducDon
Timelineanalysis• Trends• Cycles• Seasonality
OtherML• ClassificaDon• Clustering• PredicDons
• Timelines and views persisted to Cassandra.
• Column family name: <category>_tl or <category>_vw
• Column family key: ((entityId, date), timestamp)
FATES
• Short term memory
• Built on top of Spark: performant, distributed, real-time computations.
• Built on top of abstractions that hide low level details: Timeline, View, Stream, Sink, Monoid, Group, Ring…
NEOCORTEX
• Full duplex serving layer.
• Can be queried from any backend…
• ... and we provide a JS library to provide efficient data access from the browser. Inspired by Falcor and GraphML
HERMES
HERMES
B2CBrowser
B2BLoad
bal
ance
r
Push Server
DistributedCache Push
Server
Push Server
…
TCP
WS
HTTP
JS A
PI WH AppsC
ache
Cac
heApps
HERMES
HermesDistributedCache
HermesNode
LocalCache
SubscripDo
nManager
ClientM
anager
Authen
DcaD
onHandler
Dispatcher
HTTP
WS
TCP
Browser
Herm
esJS
WHAp
ps
• To provide customers with bet suggestions to make their experience more relevant/personal
• Offered to new and existing customers.
• Hot bets and personalized suggestions.
USE CASE: BET SUGGESTIONS
• To provide customers with a more engaging experience by gamifying parts of the customer journey.
• Appeal to attributes like status, mastery, belonging.
• Achievements and rewards, leveling up, leaderboard, …
USE CASE: GAMIFICATION
Recommended