Building a System of Engagement with MongoDB
{ Name: ‘Bryan Reinero’,Title: ‘Developer Advocate’,
Twitter: ‘@blimpyacht’,Email: ‘[email protected]’ }
2
Systems of Engagement
• Real-Time • Context-Aware• Encourage
Interaction• Embedded in
Business Systems
3
Systems of Engagement
• Context rich and User Relevant Interactions
• Integrates data from many systems
• Integrates Analytics
4
The Importance of Engagement
• 74% Consumer would respond positively to companies that understand them
• 57% Would recommend the company• 29% Would make additional purchases
2013 Experian study
5
• 84% Would walk away from a non-responsive company• 45% from a company that contacted them when they had asked
not to• 52% Would leave a company that tries to sell them something
they already said they weren't interested in
The Importance of Engagement
2013 Experian study
6
• 84% Would walk away from a non-responsive company• 45% from a company that contacted them when they had asked
not to• 52% Would leave a company that tries to sell them something
they already said they weren't interested in
The Importance of Engagement
How is the application of value to the user?
7
8
Department of Veterans Affairs
20+ million Veterans in the US today
Doctors need a single view of a patient’s health record.What happens when a patient has to change their address?
9
Customer Single View
• Understand customer relationships
• Improves customer experience • Develops effective customer
marketing• Improves product
Requirements
• High performance requirements• Increasingly large datasets• High Availability
11
Architecture
Systems of EngagementD
ata
Serv
ices
Systems of Record
Master Data
Raw Data
Integrated Data
…
ETLrecord
record
record
12
Aggregating a Single View
Single customerVIEW
13
Aggregating a Single View
Common Data
Source Metadata
Source Data A
Source Data B
14
Aggregating a Single View
Common Data
Source Metadata
Source Data A
Source Data B
{ _id: <hash>, address: {
num: 860,street: “Grove”,city: “San Francisco”,state: “CA”,zip:
}}
15
Aggregating a Single View
Common Data
Source Metadata
Source Data A
Source Data B
{ sources: [ { source: “URI”, updated: ISODate(), },
… ]}
16
Aggregating a Single View
Common Data
Source Metadata
Source Data A
Source Data B
Shopping Cart,Purchase history,Prescriptions,Medical History,
17
Multiple Data Sources
• Wearable Devices• Embedded Systems• Internet of Things
The Point of Engagement
19
The Scavenger Hunt App
Users create scavenger hunts by “pinning” waypoints
20
The Checkpoint Document
{_id: ObjectId(),user: UUID,huntId: UUID,timestamp: ISODate(), geometry: {
type: "Point",coordinates: [125.6,
10.1]}
}
21
The Checkpoint Document
{_id: ObjectId(),user: UUID,huntId: UUID,timestamp: ISODate(), geometry: {
type: "Point",coordinates: [125.6,
10.1]}
}
db.checkpoints.ensureIndex(timestamp: -1,geometry: “2dsphere”
);
22
The Checkpoint Document
db.checkpoints.find({ date: {$gt: <10mins ago> } geometry: { $near:{ { type : "Point" , coordinates : [ -173, 40.7 ] }, $maxDistance : 100 }}});
{_id: ObjectId(),user: UUID,huntId: UUID,timestamp: ISODate(), geometry: {
type: "Point",coordinates: [125.6,
10.1]}
}
23
The Scavenger Hunt App
Business Requirements• Location Based
Targeting• Enable Social
Interactions• Recommendations
24
Waypoint{ _id: ObjectId(), name: "Doug’s Coffee", desc: "The best brew", offers: [ ”Morning rush hour discount",
”Order ahead with our app",”Start your digital frequent
flyer card" ], "geometry": { "type": "Point", "coordinates": [125.6, 10.1] }};
25
Waypoint{ _id: ObjectId(), name: "Doug’s Coffee", desc: "The best brew", offers: [ ”Morning rush hour discount",
”Order ahead with our app",”Start your digital frequent
flyer card" ], "geometry": { "type": "Point", "coordinates": [125.6, 10.1] }};
26
Waypoint{ _id: ObjectId(), name: "Doug’s Coffee", desc: "The best brew", offers: [ ”Morning rush hour discount",
”Order ahead with our app",”Start your digital frequent
flyer card" ], "geometry": { "type": "Point", "coordinates": [125.6, 10.1] }};
Geospacial Index:ensureIndex( { geometry: “2dsphere” } )
27
The Checkpoint Document
{_id: ObjectId(),user: UUID,huntId: UUID,timestamp: ISODate(), geometry: {
type: "Point",coordinates: [125.6,
10.1]}
}
28
The Checkpoint Document
{_id: ObjectId(),user: UUID,huntId: UUID,timestamp: ISODate(), geometry: {
type: "Point",coordinates: [125.6,
10.1]}
}
db.checkpoints.ensureIndex(timestamp: -1,geometry: “2dsphere”
);
29
The Checkpoint Document
db.checkpoints.find({ date: {$gt: <10mins ago> } geometry: { $near:{ { type : "Point" , coordinates : [ -173, 40.7 ] }, $maxDistance : 100 }}});
{_id: ObjectId(),user: UUID,huntId: UUID,timestamp: ISODate(), geometry: {
type: "Point",coordinates: [125.6,
10.1]}
}
30
Geo Targeting
31
Geo Targeting
{ "type": "Polygon", "coordinates" : [ [ [ -73.969581, 40.760331 ], [ -73.974487, 40.762245 ], [ -73.977692, 40.763598], [ -73.979508, 40.761269 ], [ -73.982364, 40.762358 ], [ -73.983692, 40.760497 ], [ -73.972821, 40.755861 ], [ -73.969581, 40.760331 ] ] ]}
Defines a business service area
32
$geoIntersects
{ $geoIntersects: { $geometry: { "type": "Point", "coordinates": [ -73.975010, 40.760071 ] } } }
33
Defining Service Areas
Data Driven Decisions• Find users within a given area• Intersections with multiple service areas• Fraud detection• Where are my competitors located
Social Interactions
35
Social Interactions
Requirements• Allow users to follow one
another• Allow users to exchange
messages • Send users notifications
36
Social Interactions
Eratosthenes
Democritus
Hypatia
Shemp
Euripides
Graph modelsPersonal RelationshipsRecommendation Engines
37
Social Interactions
Followers Collection{ follower: ‘Shemp’, followed: ‘Euripides’},{ follower:‘Shemp’, followed: ’Eratosthenes”},{ follower: “Eratosthenes’, followed: ‘Shemp’ },…
Eratosthenes
Democritus
Hypatia
Shemp
Euripides
38
Social Interactions
Eratosthenes
Democritus
Hypatia
Shemp
Euripides
Followers Collection{ follower: ‘Shemp’, followed: ‘Euripides’},{ follower:‘Shemp’, followed: ’Eratosthenes”},{ follower: “Eratosthenes’, followed: ‘Shemp’ },…
! (Euripides -> Shemp )
39
Social Interactions
db.followers.find( { follower:‘Shemp’ } );
Followers Collection{ follower: ‘Shemp’, followed: ‘Euripides’},{ follower:‘Shemp’, followed: ’Eratosthenes”},{ follower: “Eratosthenes’, followed: ‘Shemp’ },…
Notifications
41
Personal Timeline / Hotlist
{
_id: UUID,
user: ”Democritus",
hotList" : [
{
message: "New scavenger hunt tomorrow!",
url: "http://bit.ly/1hKn9ff",
date" : ISODate()
},
{
message: "Get 50% off at Toga City",
url: "http://bit.ly/1KnlFHQ",
date: ISODate()
}
],
atime: ISODate("2015 03 13T04:38:43.606Z")
}
42
Personal Timeline / Hotlist
Notifications of highest user relevance
{
_id: UUID,
user: ”Democritus",
hotList" : [
{
message: "New scavenger hunt tomorrow!",
url: "http://bit.ly/1hKn9ff",
date" : ISODate()
},
{
message: "Get 50% off at Toga City",
url: "http://bit.ly/1KnlFHQ",
date: ISODate()
}
],
atime: ISODate("2015 03 13T04:38:43.606Z")
}
43
Write to Bucket
Parameters• User
db.user.update( {"user" : "Democritus"}, {$push: { hotList: { $each:
[ { o: 10, <MESSAGE> }, { o: 7, <MESSAGE> }, … ], $sort: { o: -1 },
$slice: 50 } }});
44
Write to Bucket
Parameters• user• $push append to end of hotList array
db.user.update( {"user" : "Democritus"}, {$push: { hotList: { $each:
[ { o: 10, <MESSAGE> }, { o: 7, <MESSAGE> }, … ], $sort: { o: -1 },
$slice: 50 } }});
45
Write to Bucket
Parameters• user• $push append to end of hotList array• $each list of message elements
db.user.update( {"user" : "Democritus"}, {$push: { hotList: { $each:
[ { o: 10, <MESSAGE> }, { o: 7, <MESSAGE> }, … ], $sort: { o: -1 },
$slice: 50 } }});
46
Write to Bucket
Parameters• user• $push append to end of hotList array• $each message in array• $sort the resulting array in descending order
db.user.update( {"user" : "Democritus"}, {$push: { hotList: { $each:
[ { o: 10, <MESSAGE> }, { o: 7, <MESSAGE> }, … ], $sort: { o: -1 },
$slice: 50 } }});
47
Write to Bucket
Parameters• user• $push append to end of hotList array• $each message in array• $sort the resulting array in descending order• $slice include only the first 50 elements
db.user.update( {"user" : "Democritus"}, {$push: { hotList: { $each:
[ { o: 10, <MESSAGE> }, { o: 7, <MESSAGE> }, … ], $sort: { o: -1 },
$slice: 50 } }});
Analytics
49
Analytics
• Behavioral analytics• Segmentation• Fraud detection• Prediction• Pricing analytics• Sales analytics
50
Analytics
Systems of EngagementD
ata
Serv
ices
Data Processing Integration, Analytics, etc.
Systems of Record
Master Data
Raw Data
Integrated Data
…
ETLrecord
record
record
51
MongoDB as an Operational Store
Application Server
Other DBMS
& Legacy systems
Data Capture
53
Capture Data Changes
Systems of EngagementD
ata
Serv
ices
Data Processing Integration, Analytics, etc.
Systems of Record
Master Data
Raw Data
Integrated Data
…
ETL
Bus
Apache Kafka
record
record
record
54
Many Complexities to Tackle
• Data modeling• Data Extraction (ETL)• Change Data Capture (CDC)• Data Governance• Data Lineage• Security
55
How is the application of value to the user?
Thanks!
{ Name: ‘Bryan Reinero’,Title: ‘Developer Advocate’,
Twitter: ‘@blimpyacht’,Email: ‘[email protected]’ }