Emergence of MongoDB as an Enterprise Data Hub

Emergence as an Enterprise Data Hub

Dylan Tongdylan.tong@mongodb.comPrincipal Solutions Architect

2

What is an Enterprise Data Hub?

Centralization of data sources, and providing a single view to online systems of engagement

3

360° View 70 Systems 3 Months

4

70 Different Policy Schemas

How can we translate this into a Customer

View?

ETL 70 applications into a Dimensional

Model? Integrating a few is hard…

5

Strategy: All documents can have variable schemas

db.policies.find({first:”Dylan”,last:”Tong”,type:{

$in[“Healthcare”,”PPO”,”HMO”,”Auto”]

})

6

Problem Emerges in Many Forms• Redundant systems due to M&A or lack of

governance across LOB

• Mature Product Suite of silo applications, and a need for integrated Solutions

• Operationalizing Data Lake for Online Systems

• Data-mart proliferation and need for more real-time sharing of data

7

Global Master Data Distribution at Scale

8

Global Master Data Distribution at Scale

How Do People Build This?

10

11

CONNECTOR

12

YOUR DATAHUB CONNECTOR

13

LIVE DEMO

14

Vehicle DB: “Classic-models schema”

15

DVD DB: “sakila schema”

16

Object ModelEXTRACTORS LOADERS

DATAHUB CONNECTOR

DAO Classes

Object Model Classes

MongoDDB Driver

JDBC/ODBC

<< Uses >>

17

Object Model

APPLICATION OBJECT MODEL

CUSTOMER

DVD RENTAL

VEHICLE RENTAL

Standard ORM Work

18

APP 1 APP 2 APP N

DATAHUB CONNECTOR / ETL TOOL

DATA HUB API / SERVICES

MULTICAST UPDATES

APP DATABASES

19

APP 1 APP 2 APP N

DATAHUB CONNECTOR / ETL TOOL

DATA HUB API / SERVICESCDC

IMPLEMENT CHANGE DATA CAPTURE AND RESYNC UPDATES

1. What’s the last successful sync time?

2. Query data modified since last success sync

3. On failure, do idempotent retry (unique index: ID, SourceSystemID)

20

Solution Improves with 3.2

• Document Validation

• Mongo Compass (Schema Visualization)

• Partial Indexes

• BI Connector