Building a Collaborative Data Architecture

EMBARCADERO TECHNOLOGIESEMBARCADERO TECHNOLOGIES

Build a Collaborative Data Architecture

Ron Huizenga

Senior Product Manager – ER/Studio

EMBARCADERO TECHNOLOGIES

Agenda• What’s happening in the world of data?

– Data trends, data usage, data thirst

• Collaboration defined– Enabling collaboration

• The need for architecture• What is data architecture?• Model based data architecture

– For solution development– To mitigate organizational data landscape complexity

• Business driven data architecture• Concluding remarks

2


Increasing volumes, velocity, and variety of

Enterprise Data

30% - 50% year/year growth

Decreasing % of enterprise data which is

effectively utilized

5% of all Enterprise data fully utilized

Increased risk from data misunderstanding and

non-compliance

$600bn/annual cost for data clean-up in U.S.

Enterprise Data Trends


Quenching the Thirst - Big Data?

• Volume

• Velocity

• Variety

• Veracity


Business Stakeholders’ Data Usage

5

Suspect that business stakeholders INTERPRET DATA INCORRECTLY

Yes, frequently

14%

Yes, occasionally

67%

No, never9%

I don’t know10%

Suspect that business stakeholders make decisions USING THE WRONG DATA?

Yes, frequently

11%

Yes, occasionally

64%

No, never13%

I don’t know12%


Data Model Usage & Understanding

6

13%

3%

16%

19%

31%

18%

0% 5% 10% 15% 20% 25% 30% 35%

We don’t use data models

Other

Our data team does most datamodels but developers also build

them as needed

Our database administrators owndata modeling

Developers develop their own datamodels

We have a data modeling team thatis responsible for data models

What is your organization’s approach to data modeling?How well does your organization’s technology leadership team

understand the value of using data models?

Completely understand

20%

Understand somewhat

60%

Don’t understand

17%

I don’t know3%

87%


Collaboration

• Collaborate– to work jointly with others or together especially in an

intellectual endeavor– to cooperate with or willingly assist an enemy of one's country

and especially an occupying force– to cooperate with an agency or instrumentality with which one

is not immediately connected

• Collaborative– produced or conducted by two or more parties working

together

7


Communication is Critical

8


Communication – Full Definition

9

Technically correctFunctionally useless


Communication – Simple Definition

• Communication

– The act or process of using words, sounds, signs, or behaviors to express or exchange information or to express your ideas, thoughts, feelings, etc., to someone else

– A message that is given to someone : a letter, telephone call, etc.

• Communications

– The ways of sending information to people by using technology

10


The need for architecture?Evolution:

• 38 years of construction

• 147 builders

• No Blueprints

• No Planning

Result:

• 7 stories

• 65 doors to blank walls

• 13 staircases abandoned

• 24 skylights in floors

• 160 rooms, 950 doors

• 47 fireplaces, 17 chimneys

• Miles of hallways

• Secret passages in walls

• 10,000 window panes (all bathrooms are fitted with windows)

3


Data Architecture (as defined by DAMA)

• A master set of data models and design approaches identifying the strategic data requirements and the components of data management solutions, usually at an enterprise level. Enterprise data architecture typically consists of

– 1) an enterprise data model (contextual/subject area, conceptual or logical), – 2) state transition diagrams depicting the lifecycle of major entities, – 3) a robust information value chain analysis identifying data stakeholder roles, organizations, processes and applications, and – 4) data integration architecture identifying how data will flow between applications and databases. The data integration

architecture may divide into • database architecture• master data management architecture• data warehouse / business intelligence architecture• meta data architecture.

• Some enterprises also include – 5) lists of controlled domain values (code sets), and – 6) the responsibility assignments of data stewards to subject areas, entities and code sets.

• The enterprise data architecture is an important part of the larger enterprise architecture that includes business, process and technology architecture

12


Data Architecture

• As defined in Wikipedia:

– Data architecture is composed of models, policies, rules or standards that govern which data is collected, and how it is stored, arranged, integrated, and put to use in data systems and in organizations.

13


ER/Studio Enterprise Team Edition

5


Key Skill Sets• Data Design & Management• ETL and Software Development• Data Analysis / Stats• Business Analysis & Discovery

Value Delivered• Validation• Integration• Enrichment• Usability

Value and the New Lifecycle

15

Discover

Document (Model)

Integrate


Data Landscape Complexity

16

• Comprised of:

– Proliferation of disparate systems

– Mismatched departmental solutions

– Many database platforms

– Big data platforms

– ERP, SAAS

– Obsolete legacy systems

• Compounded by:

– Poor decommissioning strategy

– Point-to-point interfaces

– Data warehouse, data marts, ETL …Data Archaeologist?


Discovery and Identification Through Models• Identify candidate data sources• Reverse engineer data sources into models• Identify, name and define• Classify through metadata• Map “like” items across models• Data lineage / chain of custody• Repository• Collaboration & publishing

17


Addressing Complexity through Models• Multi-level sub-models: allow business decomposition• Reverse engineering: wide variety of platforms including Big Data• What and where?

– Naming standards – Universal mappings

• Document and define– Metadata extensions (attachments)– Business glossaries

• Data in context: business processes• Data lineage• Repository, collaboration & publishing

18


Automated Naming Standards

19


ER/Studio: Universal Mappings

• Ability to link “like” or related objects

– Within same model file

– Across separate model files

• Entity/Table level

• Attribute/Column level

20


ER Studio: Attachment of Metadata extensions

21


ER/Studio: Data Dictionary

22


ER/Studio: Extended Notation for MongoDB

23


Clarify with Business Data Objects

24


Alternate Perspectives for Different Audiences

25


Data Lineage

26


ER/Studio Team Server: Enterprise Collaboration

27


ER/Studio Team Server – Model Explorer

28


The Need for Common Understanding

29


Business Glossary – Why?

• Maximize understanding of the core business concepts and terminology of the organization

• Minimize misuse of data due to inaccurate understanding of the business concepts and terms

• Improve alignment of the business organization with the technology assets (and technology organization)

• Maximize the accuracy of the results to searches for business concepts, and associated knowledge

30


Team Server: Glossary/Terms

31


Enhanced Communication: Glossary Integration

32


Addressing Governance

33

Data Governance

Data Architecture Management

Data Development

Database Operations

Management

Data Security Management

Reference & Master Data Management

Data Warehousing

& Business Intelligence

Management

Document & Content

Management

Metadata Management

Data Quality Management


Collaborative, Business-Driven Data Architecture

• Improve visibility and collaboration with ER/Studio

• Enable more efficient and automated data modeling

• Share models and metadata across the organization

• Establish business glossaries with consistent terms and definitions

• Build a solid foundation for compliance, data governance, and master data management

34


Thank you!• Learn more about the ER/Studio product family:

http://www.embarcadero.com/data-modeling

• Trial Downloads: http://www.embarcadero.com/downloads

• To arrange a demo, please contact Embarcadero Sales: [email protected], (888) 233-2224

35

http://www.embarcadero.com/data-modeling

http://www.embarcadero.com/downloads

mailto:[email protected]

Technology

Building a Collaborative Data Architecture