Data integration my_experience

MY STORY

Designed ByArvind Murali

MY NAME IS ARVIND MURALI

I LIVE IN APPLE VALLEY, MN

I AM A DATA INTEGRATION EXPERT

I WORK FOR HIGHPOINT SOLUTIONS

HELLO

ABOUT HIGHPOINT

My Evolution in Data Integration

Started as a Datawarehouse Analyst/Designer

Sorting through 100’s of entities To analyze and report data in excel

My Humble Beginning

Time to move on to bigger and better things!! Marriage.

&

DATA INTEGRATION CONSULTING

FFWD Three Years

Moving ON to even better things. My Son!!

&

INTEGRATIONS SOLUTIONS LEAD (Why we are doing what we are

doing?)

FFWD Two More Years and ever since…

CHALLENGES

Storing all irrelevant information only to find out increased Storage and performance slowness.STICK TO REQUIREMENTS. DO NOT OVERDO.

Challenge 1 – Relevance

Operational Databases

Data Integration

Hub

Ontology

Rules Repository

CRM Social Media

Synchronizing huge quantities and formats of data with internal and external systems.

Challenge 2 – Heterogeneous Data

Data Quality & Data Audit are 2 of the most critical components of any data integration effort. Developers and Users should work together to determine the quality control metrics and

measures.

Challenge 3 – Data Quality

What is Data Quality?

Because of the involvement of hardware and software to accomplish data integration, it is common to see some unexpected costs in the process.

• Labor costs for initial planning, evaluation, programming and additional data acquisition.• Software and hardware purchases.• Unanticipated technology changes/advances.• Both labor and the direct costs of data storage and maintenance.

Challenge 4 – Unanticipated Cost

Data Integration Lead, Data Stewards, Analysts, Designers, and Data Managers are a MUST with practice experience in data management arena. In addition, setting incremental goal helps

understand, deliver, and monetize mission.

Challenge 5 – Expertise

BEST PRACTICES

Understand and Clarify the end goal (BIG PICTURE) with your Stakeholders.

Lesson 1 – End Goal

Gather Requirements and Specifications Thoroughly and as much detailed as possible.

Lesson 2 -- Requirements

Understand and Design Industry and Organizational Standards customized to process. ONCE DEFINED, STICK TO THOSE STANDARDS.

Lesson 3 –Standards

A thorough analysis of characteristics and usage of data to mitigate bad and heterogeneous data challenges.

Lesson 4 – Data Analysis

Go GAGA over testing. Most critical phase of integration.

Lesson 5 – Testing

THE PROCESS!!

REFERENCES

Information Management MagazineDilbert Cartoons

The Datawarehouse Institute (TDWI)Cartoons for Data

Data Integration Slideshares