It’s all about the data: A Managerial Perspective...R.D.Damhof –Oktober 2014 –Norske Oh…data warehouse? •DWH in Netherlands - since 2007 - have increasingly been split-up

R.D.Damhof – Oktober 2014 – Norske

It’s all about the data: A Managerial Perspective

By Ronald Damhof

Email: [email protected]

Linkedin:

nl.linkedin.com/in/ronalddam

hof/

Twitter: RonaldDamhof

Blog: prudenza.typepad.com

Website:www.prudenza.nl

mailto:[email protected]

http://nl.linkedin.com/in/ronalddamhof/

http://prudenza.typepad.com

http://www.prudenza.nl

R.D.Damhof – Prudenza BV - Copyright - 22 mei 2014R.D.Damhof – Oktober 2014 – Norske

I am an opinionated kind a guy….


Who am I - My Data ManifestoThe X commandments of data management

I. Thou shall always respect

& consider the context.

Context is leading



II. Thou shall love your

(meta)data. Data is the

ultimate proprietary

asset:

- Manage it

- Govern it

- Utilise it

But do it ethically

“Most companies manage their

parking lot better than their data” —Gartner, Frank Buytendijk (paraphrased)



III.Thou shall stop centering

apps over data:data first



IV.Thou shall strive for

accurate, relevant, timely,

reliable and accessible data:

It is all about the quality of

the product

Deming’s point 3 of 14:

”Cease dependence on inspection to

achieve quality. Eliminate the need for

massive inspection by building quality

into the product in the first place."



V. Thou shall abstract



VI.Thou shall make a

‘fundamentalistic’ separation

between facts & context


VI.Thou shall not forsake

time



VIII.Thou shall uphold, improve

and teach the science and

practice of Information- &

data modeling



IX.Thou shall Specify,

Standardise, Automate &

Productise



X. Thou can not buy your

way out of the data

misery you are in


‘XI’There is a new saviour in town. Its name is Hadoop

and it calls to us from its mountain:

‘we got a lake and thou shall throw all your data in

it. The water will be clean so you can drink it, the

water will flow so it will irrigate your lands, grow

your stock, feed your kids and of course bring you

world peace…..’






Logistics & Manufacturing



Push/Supply/Source driven Pull/Demand/Product driven

Mass deployment Control > Agility Validation of “ingredients”

Repeatable & predictable processes Standardized processes High level of automation Relatively high IT/Data expertise

Piece deployment Agility > Control Plausibility User-friendliness Relatively low IT expertise Domain expertise essential

All facts, fully temporal Truth, Interpretation, Context

Business Rules Downstream

The Data Push Pull Point


Systematic

Opportunistic

User & developer are separated Defensive Governance Focus on non-functionals Centralised Proper system development

User = developer Offensive governance Decentralised “System development” in production

The Development Style


Development

Style

Systematic

Opportunistic

I II

III IV

Research,

Innovation &

Design

“Shadow IT,

Incubation,

Ad-hoc,

Once off”

Push/Supply/Source driven Pull/Demand/Product driven

Data

Push/Pull

Point

ContextFacts

A Data Deployment Quadrant


6 Applications of the Quadrant


(1) How we produce


How we produce, process variants


Production-line: Data orientation

Data Products Information Products

Access to data

Analytical tools

Processing Power

Production-line: Forms orientation

Eg. XBRL/JSON

How we produce, production lines

Production-line: Poly Structured


(2) How we automate


Rephrased - somewhat more nerdy:

• Model-driven, metadata driven

Or

• Declarative instead of Imperative

Rephrased - somewhat more popular:

“In Data, the developer is the data modeller”

(2) How we automate


(3) How we organize


To centralize or to decentralize


(4) How do people excel


Storage: (R)DBMS

Processing: Automation Software

Data Quality: Validation, Profiling

Development: Data Modeling

Accessibility: Data Virtualization

Storage: Pattern based

Processing: Automation/limited ETL

Data Quality: DQ rules/dashboards

User tooling: Reporting, dashboards,

Data Visualization

Storage: Analytical

Processing: Preptools for Data Analyst

User tooling: Advanced Analytics,

Data Visualization

(5) How about Technology


(6) Business-,Information- or

Data Modeling is key

Conceptual

Logical

e.g Data Vault,

Anchor Model

e.g. Dimensional,

hierarchical,flat

At least the Logical Model

drives the technical data

architecture, design and

implementation

Ontology

Facts

Relational

Natural Language


Oh…data warehouse?

• DWH in Netherlands - since 2007 - have increasingly been split-up between

facts (Quadrant 1) and context (Quadrant 2).

• Quadrant 1 is morphing into an ‘Integrated (Meta)Data Environment’. An

holistic view on data. Not only accepting feeds from other apps, but being

the system-of-record for apps.

• At least integrated on the logical level, preferably on the conceptual level.

• The (meta)data(quality) is fiercely managed & governed centrally in orgs.

• Interestingly; quadrant II is becoming the classic - more Kimball style DWH,

but where conformity is implicit and (technically) data virtualisation is key.

• Central Data Competencies, Decentral (close to demand) BI Competencies

Documents

It’s all about the data: A Managerial Perspective...R.D.Damhof –Oktober 2014 –Norske Oh…data warehouse? •DWH in Netherlands - since 2007 - have increasingly been split-up