It’s all about the data: A Managerial Perspective...R.D.Damhof –Oktober 2014 –Norske Oh…data...

R.D.Damhof – Oktober 2014 – Norske

It’s all about the data: A Managerial Perspective

By Ronald Damhof

Email: ronald.damhof@prudenza.nl

Linkedin:

nl.linkedin.com/in/ronalddam

Twitter: RonaldDamhof

Blog: prudenza.typepad.com

Website:www.prudenza.nl

R.D.Damhof – Prudenza BV - Copyright - 22 mei 2014R.D.Damhof – Oktober 2014 – Norske

I am an opinionated kind a guy….

Who am I - My Data ManifestoThe X commandments of data management

I. Thou shall always respect

& consider the context.

Context is leading

II. Thou shall love your

(meta)data. Data is the

ultimate proprietary

asset:

- Manage it

- Govern it

- Utilise it

But do it ethically

“Most companies manage their

parking lot better than their data” —Gartner, Frank Buytendijk (paraphrased)

III.Thou shall stop centering

apps over data:data first

IV.Thou shall strive for

accurate, relevant, timely,

reliable and accessible data:

It is all about the quality of

the product

Deming’s point 3 of 14:

”Cease dependence on inspection to

achieve quality. Eliminate the need for

massive inspection by building quality

into the product in the first place."

V. Thou shall abstract

VI.Thou shall make a

‘fundamentalistic’ separation

between facts & context

VI.Thou shall not forsake

VIII.Thou shall uphold, improve

and teach the science and

practice of Information- &

data modeling

IX.Thou shall Specify,

Standardise, Automate &

Productise

X. Thou can not buy your

way out of the data

misery you are in

‘XI’There is a new saviour in town. Its name is Hadoop

and it calls to us from its mountain:

‘we got a lake and thou shall throw all your data in

it. The water will be clean so you can drink it, the

water will flow so it will irrigate your lands, grow

your stock, feed your kids and of course bring you

world peace…..’

Logistics & Manufacturing

Push/Supply/Source driven Pull/Demand/Product driven

Mass deployment Control > Agility Validation of “ingredients”

Repeatable & predictable processes Standardized processes High level of automation Relatively high IT/Data expertise

Piece deployment Agility > Control Plausibility User-friendliness Relatively low IT expertise Domain expertise essential

All facts, fully temporal Truth, Interpretation, Context

Business Rules Downstream

The Data Push Pull Point

Systematic

Opportunistic

User & developer are separated Defensive Governance Focus on non-functionals Centralised Proper system development

User = developer Offensive governance Decentralised “System development” in production

The Development Style

Development

Systematic

Opportunistic

III IV

Research,

Innovation &

Design

“Shadow IT,

Incubation,

Ad-hoc,

Once off”

Push/Supply/Source driven Pull/Demand/Product driven

Push/Pull

ContextFacts

A Data Deployment Quadrant

6 Applications of the Quadrant

(1) How we produce

How we produce, process variants

Production-line: Data orientation

Data Products Information Products

Access to data

Analytical tools

Processing Power

Production-line: Forms orientation

Eg. XBRL/JSON

How we produce, production lines

Production-line: Poly Structured

(2) How we automate

Rephrased - somewhat more nerdy:

• Model-driven, metadata driven

• Declarative instead of Imperative

Rephrased - somewhat more popular:

“In Data, the developer is the data modeller”

(2) How we automate

(3) How we organize

To centralize or to decentralize

(4) How do people excel

Storage: (R)DBMS

Processing: Automation Software

Data Quality: Validation, Profiling

Development: Data Modeling

Accessibility: Data Virtualization

Storage: Pattern based

Processing: Automation/limited ETL

Data Quality: DQ rules/dashboards

User tooling: Reporting, dashboards,

Data Visualization

Storage: Analytical

Processing: Preptools for Data Analyst

User tooling: Advanced Analytics,

Data Visualization

(5) How about Technology

(6) Business-,Information- or

Data Modeling is key

Conceptual

Logical

e.g Data Vault,

Anchor Model

e.g. Dimensional,

hierarchical,flat

At least the Logical Model

drives the technical data

architecture, design and

implementation

Ontology

Relational

Natural Language

Oh…data warehouse?

• DWH in Netherlands - since 2007 - have increasingly been split-up between

facts (Quadrant 1) and context (Quadrant 2).

• Quadrant 1 is morphing into an ‘Integrated (Meta)Data Environment’. An

holistic view on data. Not only accepting feeds from other apps, but being

the system-of-record for apps.

• At least integrated on the logical level, preferably on the conceptual level.

• The (meta)data(quality) is fiercely managed & governed centrally in orgs.

• Interestingly; quadrant II is becoming the classic - more Kimball style DWH,

but where conformity is implicit and (technically) data virtualisation is key.

• Central Data Competencies, Decentral (close to demand) BI Competencies

It’s all about the data: A Managerial Perspective...R.D.Damhof –Oktober 2014 –Norske Oh…data...

Documents

Telecom Dwh

01 Presentation DWH

DWH Concentrated

DWH Material

Infosessie Sector Hoger Onderwijs Brugge, 14 oktober 2013 Gent, 21 oktober 2013 Bilzen, 22 oktober 2013 Brussel, 24 oktober 2013 Antwerpen, 25 oktober

15. Oktober 2012. 16. Oktober 2012 17. Oktober 2012

5HYLVLRQ 'DWH

norske recom practice.pdf

DMLecture03 DWH

DWh Historization Libre

BBBBBBBBBBBBBBBBBBBBB 'DWH

DWH GLOSSORY

Dwh Record

5HOHDVHG 'DWH

Dwh bi competence_center_sladeshare

Recurrent Neural Networks - Greg Mori - CMPT 419/726mori/courses/cmpt726/slides/chapterrnn_handout.pdfDWH,QSXW DWH)RUJHW* DWH,QSXW0RGXODW LRQ*DWH ... alent to introducing a Þxed

Dwh fundamentals

DWH Simplified

Dwh Tutorial

Norske pengespel 2012