Embed Size (px)
Citation preview
Streaming Behavioral Analytics
Agenda
©2017 PayPal Inc. Confidential and proprietary.
Behavioral Analytics
What is it?
Why is it important?
Streaming data pipeline
What is it?
What should you consider for building it?
Streaming Architecture
Analytics tech stack @ PayPal
Q & A
Behavioral Analytics
©2017 PayPal Inc. Confidential and proprietary.
I am planning to buy
Hmm! It takes a lot of time
Why do I get weird suggestions
Got it! Let me place the order
Transaction Failed
Behavioral Events
©2017 PayPal Inc. Confidential and proprietary.
Customer & Product Relationship
©2017 PayPal Inc. Confidential and proprietary.
Discovery
Engagement
Repeatability
Understanding Customer Empathy
©2017 PayPal Inc. Confidential and proprietary.
Performance &Latency
Discovery
Experimentation
A B
Page Views & Clicks
Customer Segments
PageBounces
Engagement
Data Pipeline
©2017 PayPal Inc. Confidential and proprietary.
Repeat!
Ingest Process Present
Challenges
©2017 PayPal Inc. Confidential and proprietary.
Volume Elastic Resilient Responsive
Streams
©2017 PayPal Inc. Confidential and proprietary.
01010100101001001010…
Flow of data
A B
Unbounded flow
Real time processing
Observable
Ingestion of Events
©2017 PayPal Inc. Confidential and proprietary.
Fast Publisher Slow Subscriber
Overflow
©2017 PayPal Inc. Confidential and proprietary.
Fast Publisher Chain of Subscriber
Demand (2)
Backpressure
Streaming data
©2017 PayPal Inc. Confidential and proprietary.
Unbounded flow of data
Backpressure?
Buffers
©2017 PayPal Inc. Confidential and proprietary.
EventsChronicle
queue
commitPersistent Buffers
Broadcast Buffers
Actors
©2017 PayPal Inc. Confidential and proprietary.
A
CB
• State• Behavior• Mailbox
DISPATCHER Threadpool
Tasks
Fault Tolerance
©2017 PayPal Inc. Confidential and proprietary.
Guardian Actor
A
B C
D
E
F
G H
Resume
Restart
Stop
Escalate
Supervisor Strategy
Streams
©2017 PayPal Inc. Confidential and proprietary.
Source Flow Sink
source ~> buffer ~> enrich ~> transform ~> kafkaFlow ~> commit ~> latency ~> sink
Observability
Headers UUID Timestamp
©2017 PayPal Inc. Confidential and proprietary.
Grafana
Analytical Stack @
©2017 PayPal Inc. Confidential and proprietary.
powered by
Real Time Reporting
©2017 PayPal Inc. Confidential and proprietary.
Customer Perceived Latency
Real Time Reporting
©2017 PayPal Inc. Confidential and proprietary.
Page Views and Visitor Metrics
Real Time Reporting
©2017 PayPal Inc. Confidential and proprietary.
Page Flow and Bounces
Real Time Reporting
©2017 PayPal Inc. Confidential and proprietary.
