Embed Size (px)
Citation preview
2
What is an Enterprise Data Hub?
Centralization of data sources, and providing a single view to online systems of engagement
3
360° View 70 Systems 3 Months
4
70 Different Policy Schemas
How can we translate this into a Customer
View?
ETL 70 applications into a Dimensional
Model? Integrating a few is hard…
5
Strategy: All documents can have variable schemas
db.policies.find({first:”Dylan”,last:”Tong”,type:{
$in[“Healthcare”,”PPO”,”HMO”,”Auto”]
})
6
Problem Emerges in Many Forms• Redundant systems due to M&A or lack of
governance across LOB
• Mature Product Suite of silo applications, and a need for integrated Solutions
• Operationalizing Data Lake for Online Systems
• Data-mart proliferation and need for more real-time sharing of data
7
Global Master Data Distribution at Scale
8
Global Master Data Distribution at Scale
How Do People Build This?
10
12
YOUR DATAHUB CONNECTOR
13
LIVE DEMO
14
Vehicle DB: “Classic-models schema”
15
DVD DB: “sakila schema”
16
Object ModelEXTRACTORS LOADERS
DATAHUB CONNECTOR
DAO Classes
Object Model Classes
MongoDDB Driver
JDBC/ODBC
<< Uses >>
17
Object Model
APPLICATION OBJECT MODEL
CUSTOMER
DVD RENTAL
VEHICLE RENTAL
Standard ORM Work
18
APP 1 APP 2 APP N
DATAHUB CONNECTOR / ETL TOOL
DATA HUB API / SERVICES
MULTICAST UPDATES
APP DATABASES
19
APP 1 APP 2 APP N
DATAHUB CONNECTOR / ETL TOOL
DATA HUB API / SERVICESCDC
IMPLEMENT CHANGE DATA CAPTURE AND RESYNC UPDATES
1. What’s the last successful sync time?
2. Query data modified since last success sync
3. On failure, do idempotent retry (unique index: ID, SourceSystemID)
20
Solution Improves with 3.2
• Document Validation
• Mongo Compass (Schema Visualization)
• Partial Indexes
• BI Connector
