DAMA Minneapolis – May 21, 2003“All those legacy apps you wanted to replace… they got a whole new lease on life!” u The eternal quest for the perfect methodology… SSAD, IE,

ClariteqClariteq Data Modeling: New Uses for New Times

DAMA Minneapolis – May 21, 2003

Presented by:

Alec SharpClariteq Systems Consulting Ltd.206 – 2438 Marine DriveWest Vancouver, BCCanada V7V 1L2604 [email protected]

© Clariteq Systems Consulting Ltd. 2© Clariteq Systems Consulting Ltd. 2 Data Modeling: New Uses for New Times

To start the discussion…

Data modeling islargely unchangedover 15-20 years…

… but theenvironment isvery different.

1 -What’s different?2 - How have youused data modelingin a new way?

ü Do you agree?

ü In obvious and not-so-obvious ways

ü Have you used data modeling in a way youwouldn’t have predicted 10 or 15 years ago?

Intro


What I do…

1 - ProvideConsultingServices

2 - DevelopWorkshops

3 - DeliverWorkshopsüSystem Development

or Acquisition usingModel-DrivenRequirements DefinitionüData Modeling and Reverse-EngineeringüProcess Mapping and RedesignüFacilitationüBusiness Planning &

Data/Process/Application Architecture

üIn-house at largeorganizationsüMostly 2 days eachü30 to 40 per year

üRequirements Analysis OverviewüWorkflow Process ModelingüUse Cases and Application LogicüData ModelingüAdvanced Data Modeling

Intro

My mother is impressed because

I wrote a book!


That’s because everything’s different!u Less “new” modeling,

more dealing with existing structuresl purchased / ERP / COTSl customl legacy

u Less “gorilla” modeling,more “guerilla” modeling to supportl BPxl trainingl understanding policy & practicel …?

Angst about your data modeling career?

Businesspractices

Requirements/Models

Applications/Databases

less forward

more reverse

The new reality:

Intro


Origins of the problem

Will consultfor food

“Data modelers won’t beneeded anymore, because the

software company hasalready done it!”

Various commentators on mycareer as a data modeler,

mid-1990s

Intro


Coming to grips with it

u ~1991 - Max Hopper CW article

u early 90s –“Modeler’s won’t be needed anymore!”

u mid 90s - Manufacturing client:“Help! Our COO chose a package and…”

u Since then… “Help! We need to…”l select a packagel adapt a packagel implement a package

u Maybe Max was right...

Intro


The state of the whirled

u The “E-whatever” hangover - another savior bites the dustand process – “BPx” - stages a comeback

u The Object-Oriented Analysis & Design hangover…“It should have worked… we just needed smarter people”

u The Y2K hangover…“All those legacy apps you wanted to replace…they got a whole new lease on life!”

u The eternal quest for the perfect methodology…SSAD, IE, RAD, RUP, and now…lightweight methods!

u Major internal development is on hold,and purchased applications have taken over

u Integrated architectures are suspect – “M&A means ourbusiness regularly disintegrates and reintegrates”

u Thankfully – XML and (fill in the blank) will save the day!(with assistance from theMary Shelley Consulting Corporation)

Intro


EAI

The Mary Shelley reference explained

Not quite the “integration”we had in mind…

Intro


So what we have is...

At least this is

familiar territo

ry!At least t

his is

familiar territo

ry!

u Bad integration, which has kindly beenrenamed “loosely coupled”

u Ad hoc planning (business as usual)u A struggling internal I.T. (Alert the media!)u Vendors and other providers in ascendancyu A bunch of legacy, custom, third party, purchased,

and ERP applicationsu A few tools, no integration,

and dubious prospects

So, what can we grab hold of?

Intro


Data modeling – more important than ever!!!

“Now! That should clear up a few things around here!”

Intro

uThe business needs alingua francamore than ever

u “Data” represents thethings thatl processes act onl applications revolve

aroundl businesses want

information about


The basic message…

Data modeling islargely unchangedover 15-20 years…

… but theenvironment isvery different.

Adapt, or be abystander. Newtimes call for newapproaches.

ü Well defined components and standardsü Industry support - tools, training, literatureü Widely seen as the fundamental technique

(of course, “completely new” methods like OOAand UML come along from time to time, but…)

ü ERP, packages, COTS, …ü Immortal legacy appsü Less development, more plastic surgeryü A new take on “integration”ü New “partners” to deal with

ü Show the business what they haveü Show the business what they’re gettingü Reverse-engineeringü Presentation and expositionü Acceptance… packages, “integration,” etc.

Intro


Let’s hear some stories

u Preparing for configuration of ERP application

u Selection of G/L system

u Assessing the impact of implementingacquired software (simple example)

u Assessing the impact of implementingcustom software (more complex example)

u “Why do we hate this application?” and“How should we influence vendor’s plans?”

u Determining if an application does what it should

u Securing support for development plans

Intro


~1994 – redemption!

Could you come onover and do that

thing you do?

I guess. What thingin particular?

That entity datastuff with the

boxes and linesOh, data modeling. Sure -

what’s the project?

We’re implementingsomething called SAP.

Our COO told us to!Uh-huh. What do yourSAP consultants say

about this?They say it’s a

terrible idea and awaste of time.

I’m on my way.

The client... Alec...

#1


The details of my redemptionThe problem:

§ application selected by decree

§ need to understand as-is business to map to packageand decide on configuration options

§ a list of 100s of requirements wasn’t helping

The approach:

§ team of 7 builds 45 entity conceptual datamodel over two days

§ identify “what’s good, what’s not” aboutcurrent business rules

§ move on to configuration activities

• Vendor• Country• Plant• Plant Location• Equipment Item & Type• PO, PO Line Item• Req, Req Line Item• Release, Release Line Item• Work Definition, WD Item• etc. etc. etc.

The key points:

§ client-initiated, not IT

§ “More value from those two daysthan anything else we did!”

§ “I’m not irrelevant!”

#1


Example 2: Selecting an application

Selecting of new Financials app ishopelessly bogged down

§ Considerable effort in building aBDM*

§ Two problems:

1 matrix points to the app no onelikes

2 want vendor demos with focusand control

Requirements D&B Oracle SAP Coda etc.1 Y Y Y Y2 Y Y Y N3 Y Y Y Y4 N Y N Y5 N N Y Y6 Y Y Y Y7 Y Y Y Y8 Y Y Y Y9 Y N Y N10 N Y N Y11 Y Y Y Y12 Y Y Y Y13 Y N Y Y14 Y Y N N……

858 N N N Y859 Y Y Y Y

BDM issues§ time consuming

§ most apps meet most criteria

§ still can’t tell if an app will work well inyour environment

* Big Dumb Matrix

#2


The dangers of over-analysis#2


Selection - focusing on what matters

The approach:

§ small team builds “thing model”(~60 things total, 15 “core”)

§ for each core thing (okay…entity) identify 3 to5 life cycle events

§ for each event, develop scenario

§ pre-conditions and test data

§ desired handling and outcome(“use cases”)

§ turn over to app vendors – “Show us!”

§ how you support the model

§ how you handle scenarios

“Things we track” -• Project, Work Order• Plant, Plant Equipment• Product Type, Product Lot• Product Inventory• Sale, Transfer• Location, Ledger Entity• Financial Category• Responsibility Center• Account, Sub-Account• Fixed Asset

The key points:

§ it worked! - much clearer how well an app would support thebusiness

§ didn’t call it “data modeling” until late in the game

§ left vendor some room - “Here’s how we’d do it.”

“What happens to them” -Fixed Asset is• Acquired or Constructed• Depreciated• Transferred• Disposed Of

#2


A pattern you may have noticed…

2 - for each significant entity 3 - for each event

übrainstorm events that targetit, e.g.Fixed Asset “is acquired”, “isdepreciated”,“is transferred”,“is retired”

üarrange events in typical “lifecycle” order

üadd missing events

üdiscuss “expected handling” -• communicating it to the system - use

cases• validation, rules, data updates -

application logic

ücheck to see if thedata model handles it

ü revise the data model asnecessary

A minimalist methodology for extending and validating a datamodel, and discovering further requirements.

#2

1 – build a simple data model


What does this app. mean for us? (simple)

u Situationl Blue Cross affiliate purchases “SuperBlue” from another

(no study, no assessment – “divine intervention”)l VP of IS wants implementation impacts understood

w improvements & drawbacksw adjustments to business processes and procedures

l Initial plan – send 60+ business people through my datamodeling class ($$$!!!)

u Ethics prevents me (reluctantly!) from agreeingu A better approach

l Reverse-engineer something understandablel 10 minute course on “data models” (unavoidable)

w what they arew how they impact systems and businesses

l Review SuperBlue model and capabilities withbusiness folks

l Let them identify impacts

#3


How not to review a model

ü Abstracting

ü Using visual cuesconsistently

ü Masking unnecessarydetail

Models should aidunderstanding by:

#3


Drawing the model - “No Dead Crows!”

Kernels across the upper part of thediagram or subject area

Characteristics below characterizedentity; “characterizing” relationshipconnects at top edge

Associatives between & below associated entities;“associating” relationships connect at top edge

Types beside ordiagonally aboveclassified entity;“classifying”relationshipconnects at side

Key pointEntity type is obvious from:§ Placement§ Relationship connections

!

Various layouts can work:• top-down best for mere mortals• left to right temporal• bottom-up organic• right to left swimming upstream

Layouts to avoid:• middle-out cosmic• no crossed obsessive

lines compulsive• random Mensa-only

#3


Some elements of the presentation…

But systems have become very complex, expensive,and time-consuming to develop, so ...

DefineBusinessObjectives


Training onNew System

Training onNew System

DefineBusiness

Processes

DefineBusiness

Processes

DetermineSystem

Requirements

DetermineSystem

Requirements

DevelopCustomSystem

DevelopCustomSystem

Traditional Approach:



Training onNew Processes

and System

Training onNew Processes

and System

Select“Off The Shelf”

System

Select“Off The Shelf”

System

Determine“Assumptions”

of System

Determine“Assumptions”

of System

AdjustBusiness

Processes

AdjustBusiness

Processes

Emerging Approach (~1995 on):

#3


The essence of Data Modeling...

u “A description of a business,in terms of the things it needs to know about.”

u There are many ways to describe a business...l How it works - Process Modell How it’s organized - Organization Chartl Where it operates - Location Mapl Whether it’s making money - P&L Statementl What it needs information about –

Data Model

u Followed (reluctantly) by 10 minutes ofdata modeling education, and examples of howdata models impact application capabilities

#3


What is a data model?

“ A depiction of the things the business needs toknow about and the facts about those things”

Graphic Component NarrativeComponent

Entity-Relationship Diagram Entity Definition

Student: A Studentis any person who hasbeen admitted to theUniversity, hasaccepted, and hasenrolled in a coursewithin a designatedtime. Faculty and staffmembers may also bestudents

NumberNameAddressMajorGPA

Student

DepartmentNumberCredit HoursDescriptionPre-requisites

Course

NumberName

Instructor

NumberBuildingSeating CapacityEquipment

Room

enrolls in

taughtby

held in

SectionDaysTime

Attributes

Entity

Relationship

#3


Why do we care about Data Models?

u The Data Model represents everythinga system “knows” about the worldl “world view” would be a better terml “things” - terminology and definitionl “facts about things” –

what data is maintainedu Fundamental impact on:

l terminologyl system capabilitiesl business processesl reporting capabilitiesl data that can be collected or displayed,

and its formatl and almost everything else…

?

#3


The critical point...Different data models, different business capabilities!!!

Order Management, Version 1:- each branch treated as a separateCustomer - no easy way to “roll up”- one delivery date and location per Order- can’t override Addresses for an Order

Customer places

Order

Order Line

contains

specifies

IDNameBilling AddressShipping Addressetc.. Number

Placed DateDelivery DateStatusetc.

Quantityetc..

Product

IDDescriptionUnit Priceetc.

Order Management, Version 2:- easy to “roll up” business across all Branches - each Order Line can be for a different Delivery Date orBranch, and can override the Billing and Shipping Addresses.

Customer

for deliveryto

Order

Order Line

contains

specifies

IDNameMain Addressetc..

NumberPlaced DateStatusetc.

QuantityDelivery DateOverride Billing AddressOverride Shipping Addressetc..

Product

IDDescriptionUnit Priceetc.

BranchNumberNameBilling AddressShipping Addressetc..

operates

#3


Presentation guidelines#3

u Draw it on a whiteboard while you present it, even if youhave a laptop presentation.“If it’s too complicated to draw,it’s too complicated to present.”

u Draw it top down, adding a few entities at a time.u Constantly illustrate the model with sample instances,

definitions, schematics, etc.u Regularly highlight features and constraints of the model, in

business terms. E.g.,“Currently we can allocate a Product to one ProductCategory, but this model enables us to allocate a Product tomultiple Product Categories at a time, and to record changesin categorization over time.”

u Encourage participation - the more questions andcomments, the better!


SuperBlue “Provider” - 1

Individual

Institution Area

#3

Sorry –I’ve had to delete the rest of the “progressive build” ofthis data model presentation,because it’s my client’s proprietary material.

Just remember…“Start simple, at the top, and add detail in layers.”


“We didn’t know it was important”

Different levels of detail support different perspectives

Higher levels are often lost because…

“Our tool doesn’t support multiple perspectives”

“We started at the bottom level…”

A note on different types of models#3

Type of data model The need

1

2

3

Subject Area(Scope)

Conceptual(Concept)

Logical(Specification)

ü Agreements on scope, context,primary terms and definitions

ü Agreements on basic concepts,interrelationships, and rules

ü Complete detail for physical design,and detailed application specs


What does this app. mean for us? (complex)

The situation:

§ major new application involving significant businessprocess change

§ each site currently has its own process

§ how to identify and address “variances?”

The approach to date:

§ team descends on site, does exhaustiveas-is analysis

§ team “returns to base,” does gap analysis

§ much later, team returns to site to negotiate

The problem:

§ time-consuming and mind-numbing

§ process drove the sites crazy

§ focus on minor issues, many important issues missedentirely

“The sites don’t

want to see us

ever again!”“The sites don’t

want to see us

ever again!”

#4


An interlude on interlocking techniquesB

usin

ess

Info

rmati

on

Syst

em

s

Framework Layer

ProjectCharter

BusinessObjectives

The mission, strategies (customers/markets,products/services, differentiators), goals,objectives, CSFs, KPIs, etc. for the organization.(MSGO – Mission, Strategies, Goals, Objectives)

What it covers… The Technique

WorkflowProcess

Modeling

BusinessProcess

The activities the business carries out in order tomeet its objectives. Includes the actors involved,the sequence of steps they carry out (workflow),and the result(s) produced

Use CasesUser

Interface

A mechanism through which an actor in a businessprocess interacts with a system. Usually a GUI(graphical user interface) and reports, but couldinvolve scanners, IVR (telephone) systems, etc.

ApplicationLogic

Modeling

ApplicationLogic

A “transaction” offered by a system – a specificfunction. Includes the business rules and dataupdates it is responsible for. Requires EventAnalysis, State Transition Analysis, etc.

DataManagement

DataModeling

Files and databases that provide a system’s record-keeping functions. Determines the things a system“knows” about, and the data that is maintainedabout those things.

#4


How they all hang together…#4

offers

RulesUpdates

transaction

Entityattributes

Entityattributes

Entityattributes

Entityattributes

stepactor 1

actor 2

actor 2

step

step

step

step step

When actor...

Then system...

When actor

Then system

Input Message Output Message


It all rests on the data model…

.. is the foundation technique. It supports andprovides a common vocabulary for all the otheranalysis techniques, but they extend and validate it.

Data Modeling….

Data Model Terms &Definitions

BusinessProcess Use Cases

SuccessfulProject

ApplicationLogic

……..……..

#4


A better approach to “gap analysis”

The new approachu Build ~10 “variance packages”

based on package features, andprevious work

u Organize “core team” of 15 to 25reps at each sitel cross-functionall included impacted staff and

managementu Conduct facilitated walkthroughs

of variance packagesu Core team highlights “good

news!” and gaps

Each variance pack includedu Application area overview

(main application functionality)u Expected “good news!”

and expected gapsu Data Model

l fairly detailed attributesl main business rules

u Workflow Model (“swimlanes”)l actorsl stepsl flow – sequence, branching,

handoffsu Use Case Scenarios

l 5 to 12l progressive complexityl exercise entire workflow and

core app. functionalityIt worked!

It worked!

#4


A client request…“Help!!! We totally modifiedSuperApp 1.0 (screens, reports,code) but we never really liked it.SuperApp 2.0 is now available -what should we do?”

The approach...Behind the scenes, buildconceptual data models bystudying SuperApp and talking toclients. Present “world views,”not “data model.”

How you see the world... How SuperApp sees the world...

Draw it Live!Draw it Live!

At the session

Why do we hate this application?#5.1


Influencing the vendor

The approach for assessing app andnegotiating with the vendor:

u categorize issues into 3 Tierframework

u assign severity

u focus only on “High” severity dataissues

PresentationSeverity Description

HHMMMMLL

Application Process LogicSeverity Description

HHHMMLL

Data ManagementSeverity Description

HHHMMLL

If you can’t get the

data right, nothing

else matters!If you can’t get the

data right, nothing

else matters!

#5.2


A quick story – the “Loopholes” project

u “ISD messed up! This app doesn’t work right!”(In fact, “Oh no! You did it just like I said to!”)

u A familiar approachl Key entitiesl Definitionsl Events, leading to State Transition Diagrams (a

useful form of business rules)u Two key “revelations”

l the “Account” concept didn’t work out quite asexpected

l Most of the “loopholes” were from cleverexploitation of the “Transfer” rules(a la www.screwtheDMV.com)

#6


“Here’s the new model. Isn’tit great?”

“So what?” “Yawn”“ZZZZZZzzzzzzzz….”

Road show version 1

VP of IS & Finance“You’re dyin’ out there, kid! I want you to drag them through the painand misery of our current files and databases.Then show them the new model.”

Road show version 2“Let’s try answering someimportant questions using thecurrent model, and then thenew model”

“Fantastic!” “When do weget it?” “Do you need $$$?”

A quick story – the “New Customer DB” project#7


Many more are possible…

u Identifying business processesu Securing funding for start-upu Defining semantic and syntactic translation rules

for EAI undertakingsu New employee orientationu …and many more!

Inclosing


Your turn...

Thanks for

listening!Thanks for

listening!

Inclosing

Alec Sharp

Clariteq Systems Consulting Ltd.206 – 2438 Marine DriveWest Vancouver, BCCanada V7V 1L2604 [email protected] (soon!)

Documents

DAMA Minneapolis – May 21, 2003“All those legacy apps you wanted to replace… they got a whole new lease on life!” u The eternal quest for the perfect methodology… SSAD, IE,