54
UNLEASH THE POWER OF YOUR DATA 5/23/2019 1

UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

UNLEASH THE POWER OF YOUR DATA

5/23/20191

Page 2: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

meta• a prefix appearing in loanwords from Greek, with

the meanings “after,” “along with,” “beyond,” “among,” “above”

wright • a person who creates, builds, or repairs something

specified: a playwright ; a shipwright

5/23/20192

Page 3: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Brief History of “Operationalizing” Data

2019

2009

1999

1989

5/23/20193

Page 4: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

The 4th Industrial Revolution

AUTOMATIONComputer

MECHANIZATIONSteam

MASS PRODUCTIONElectricity

CYBERPHYSICALIOT- AI-ML

5/23/20194

Page 5: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Digital TransformationMcKinsey Global Institute's Industry Digitization Index,

Europe is currently operating at 12% of its digital potential, while the USA is operating at 18%

https://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/digital-europe-realizing-the-continents-potential

5/23/20195

Page 6: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

How we doin?

Qualitative vs Quantitative Analysis

“Buy Facebook Likes”

5/23/20196

Page 7: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

DIGITAL TRANSFORMATIONTURNING BUSINESS INTO A SCIENCE

DIGITALQuantitative

ANALOGQualitative

Page 8: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Gartner – Strategic Technology Trends

INTELLIGENT

• Autonomous Things• Augmented Analytics

• AI-Driven Development DIGITAL

• Digital Twin• Empowered Edge• Immersive Experience MESH

• Blockchain• Smart Spaces

PRIVACY & ETHICS QUANTAM COMPUTING

5/23/20198

Page 9: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Autonomous Things

Single Purpose Automation

Multi-Environment

AI Driven

Multi-Device

Multi-purpose

5/23/20199

Page 10: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Source: Conde Nast Collection (New Yorker / Paul Noth)

5/23/201910

Page 11: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Augmented Analytics - Machine Learning

Points, Edges Weights, Biases

5/23/201911

Page 12: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Neural Network

Algorithmic Weights and Biases

Hey Alexa, Is this a Penguin?

3BLUE1BROWN SERIES

5/23/201912

Page 13: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Cost Effective Process Efficiency

The Humble AI Hot Dogmin$$$

While maintaining taste, color, texture and food safety

QP QB QC

Objective Function+ Constraints

5/23/201913

Page 14: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Citizen Data Scientist

Data Collection Data Consumption

Machine Learning Applications

• Models• Knapsack Problem • Travelling Salesman

• QA, Testing Tools• Sampling Verification

Gartner estimates that by 2020 the number of citizen data scientists will will grow 5 times faster that expert data scientists

Data Conception ?

5/23/201914

Page 15: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

The Data Industrial Revolution

MECHANIZATIONICAM

CYBERPHYSICALAI-Driven Development

AUTOMATIONApplication – Database Generation

MASS PRODUCTIONProcess – Data Modelling

5/23/201915

Page 16: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Data and Process(ing)

1968 1976

ERM

Entity – Relationship Model

IDEF-0*

*ICAM Definition, Integration DEFinition

PROCESS

1992

1996

5/23/201916

Page 17: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Data Operations

Collection / Consumption

DATA COLLECTION - Applications

DEV OPS - is a set of software development practices that combines software development (Dev) and information technology operations (Ops) to shorten the systems development life cycle while delivering features, fixes, and updates frequently in close alignment with business objectives.

DATA CONSUMPTION - Analytics

DATA OPS – is an automated, process-oriented methodology, used by analytic and data teams, to improve the quality and reduce the cycle time

of data analytics

Wikipedia

5/23/201917

Page 18: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Empowered Edge - Digital Twin

Computer-Aided Design

By 2020, Gartner estimates that there will be more than 20 billion connected devices

5/23/201918

Page 19: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Immersive Experience

Virtual

Environment

Real

Environment

MR

Mixed

Reality

VR

Augmented

Reality

AR

5/23/201919

Page 20: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Immersive Senses

Proprioception

5/23/201920

Page 21: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Human PerceptionThe White and Gold (No, Blue and Black!)

5/23/201921

Page 22: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Blockchain

BROKER

• Bank• Clearinghouse• Government

“Smart Contracts”

• Reduce fees• Eliminate latency

5/23/201922

Page 23: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Smart Spaces

Technology that is FIT for USE

for how I want to USE IT!

DYNAMIC PERSONALIZATION

Technologies

Physical or Virtual

Information

Page 24: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

1927

Page 25: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Impacts on Data Management?

ARCHITECTURE

METADATA REFERENCE FILES

STANDARDS

ARCHITECTURE

Page 26: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Impacts on Data Management?

ARCHITECTUREARCHITECTURE

METADATA REFERENCE FILES

STANDARDS

DATA

GOVERNANCE

DATA

OPERATIONSDATA

QUALITY

5/23/201926

Page 27: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Data Mapping & Traceability

HUH?Help Understanding Heuristics

Month

Season

Week

How

Reported

COUNT

$$$$$$

DirectorManager

Customer

TypeCompany

PERIOD

PRODUCT

PEOPLE

PROCESS

Product

Service

Group

PARTY

Drill Down

Assuring Sampling Method

5/23/201927

Page 28: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Data Quality – Machine Learning

5/23/201928

Page 29: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

ALCHEMY vs CHEMISTRY

“To make gold, one must start with gold.”

Bernard Trevisan, 15th century alchemist

5/23/201929

Page 30: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Ouroboros - Data Quality

Start with Good Data !

5/23/201930

Page 31: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Data Interoperability - Standards

Transformation

Classification

5/23/201931

Page 32: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Data Manufacturing

Page 33: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Create, Read, Update, Delete

5/23/201933

Page 34: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Collecting – NOT Creating Data

QUANTITY

VALUE

TIMESTAMP

Transaction Molecule

$

42

CustomerCategory

Code

Country

The 4Cs of Master Data:

Overview

Scott Taylor 4/22/2019

https://www.dataversity.net/the-4cs-of-master-data-overview/

Calendar

5/23/201934

Page 35: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Generalizing the Model

PARTY PRODUCT

PROCESS

PLACE

PERIOD

5/23/201935

Page 36: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

5/23/201936

Page 37: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

3 Dimensional Model

CONCEIVE

CONFIGURE

COMMIT

CONSUME

5/23/201937

Page 38: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

4 Dimensional Model - Over Time

-1 0 +1TIME

POT

PITDATA COLLETION

POINT-IN-TIME

DEVOPS

DATA CONSUMPTION

POINT-OVER-TIME

DATAOPS

5/23/201938

Page 39: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Business

as a

Science

Develop Hypothesis

Identify Data Sets

Develop Patterns

Test Hypothesis

Page 40: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

DATA STRATEGY

Business Objectives

ARCHITECTURE

METADATAREFERENCE FILES

STANDARDS

DATA

GOVERNANCEDATA

OPERATIONSDATA

QUALITY

ARCHITECTURE

5/23/201941

Page 41: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Plus ça change…

5/23/201942

Page 42: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

…Plus c'est la même chose

5/23/201943

Page 43: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

C&C - Systems 88

MS PowerPoint (1989-90)

5/23/201944

Page 44: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Alignment

BIZ IT

5/23/201945

Page 45: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

PRIVACY & ETHICS

Page 46: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

GATES

GUARDS

GUNS

Data Protection vs. Data Privacy

Page 47: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Data Privacy

• European – General Data Protection Regulation• Explicit Consent• Right to be Forgotten• Lawful Processing• Breach Notification

• California Data Protection Act

Page 48: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Ethical Sharing of Data

B to CDirect Relationship

With CustomerB to B

Implied RelationshipWith Customer

CONTENT & CONTEXT

GDPR

5/23/201949

Page 49: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

QUANTUM COMPUTING

Page 50: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

• Binary Based Model

Architecture Models

• Neuron Based Model

• Moore's Law

5/23/201952

Page 51: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Program Sessions

Page 52: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Program Agenda

11:00 am – Paul Moxon

“Key Data Integration

Challenges with Data

Virtualization”

11:40 am – Neil Barton

“Augmented Analysis and

Automation in the Age of the

Data Scientist”ARCHITECTURE

DATA

GOVERNANCEDATA

OPERATIONS

1:20 pm – Jeff Giles

“Putting Your Metadata

to Work for You”

2:00 pm – Sean Anderson

“Deploying Data Ops for

Analytics Agility”

3:00 pm – Mark Lynch

"Accelerate Your Journey

to AI"

3:40 pm – Robert Abate

“Data Privacy & Today’s

Privacy Regulations”

5/23/201956

Page 53: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Great Power…

Carries with it Great Responsibility for the Conception,

Collection and Consumption of Data.

-ishVoltaire

5/23/201957

Page 54: UNLEASH THE POWER OF YOUR DATA - DAMA New York€¦ · •Immersive Experience MESH ... Data Quality –Machine Learning 5/23/2019 28. ALCHEMY vs CHEMISTRY “To make gold, one must

Donald J. [email protected]

market.newfoundlandcanvas.com/donaldjsoulsby