Agile Database Techniques

Copyright 2001-2004 Scott W. Ambler 1

Agile Database Techniques:Data Doesnt Have to Be a Four Letter Word Anymore

Scott W. Amblerwww.ronin-intl.com/company/scottAmbler.html

Senior ConsultantRonin International, Inc.


Scott W. Ambler? Consultant:

? Agile Modeling? Software Process Mentoring ? Software Development Mentoring

? Author:? Agile Modeling? Agile Database Techniques? The Elements of UML Style? The Object Primer 3rd Edition? Practical Guide to Enterprise Architecture? The Elements of Java Coding Style

? Contributing Editor/Writer:? Software Development? Computing Canada? IBM DeveloperWorks


Questions?Please ask burning questions

during the talk

The only stupid questionis the one that you dont ask


Presentation Overview? Warning!? Important URLs? Agile Data Method? Agile Database Techniques? Conclusion


Warning!? Im spectacularly blunt at times? Many new ideas will be presented? Some may not fit well into your existing

environment? Some will challenge your existing notions

about software development? Some will confirm your unvoiced suspicions? Dont make any career-ending moves? Be skeptical but open minded


Important URLs? www.agilealliance.org? www.agilemodeling.com? www.agiledata.org? www.extremeprogramming.com

? www.xprogramming.com? www.ambysoft.com/processPatterns.html? www.modelingstyle.info? www.enterpriseunifiedprocess.info


The Agile Data (AD)Method

What is the AD method?

AD Philosophies

Roles of AD


What is the (AD) Method?? The Agile Data (AD) method is a collection of

philosophies that will enable IT professionals within your organization to work together effectively when it comes to the data aspects of software-based systems.

? The AD method is based on the values and principles of the Agile Alliance.

? The AD method works well with the practices of Agile Modeling (AM)


Philosophies of AD1. Data. Data is one of several important aspects of software-based systems. 2. Enterprise issues. Development teams must consider and act appropriately

regarding enterprise issues.3. Enterprise Groups. Enterprise groups exist to nurture enterprise assets and

to support other groups, such as development teams, within your organization. These enterprise groups should act in an agile manner that reflects the expectations of their customers and the ways in which their customers work.

4. Unique situation. Each development project is unique, requiring a flexible approach tailored to its needs. One software process does not fit all and therefore the relative importance of data varies based on the nature of the problem being addressed.

5. Work together. IT professionals must work together effectively, actively striving to overcome the challenges that make it difficult to do so.

6. Sweet spot. You should actively strive to find the sweet spot for any issue, avoiding the black and white extremes to find the gray that works best for your overall situation.


The Roles of the AD Method

EnterpriseArchitects

AgileDBAs

ApplicationDevelopers

Standards, Guidelines,"Current State"

Guidance

Enterprise Models,Vision For Future

Data-OrientedChange Requests,

Questions

System-OrientedChange Requests,

Questions

Data Design Constraints,Data Models

ApplicationDesign Constraints, Application Models

EnterpriseAdministrators

Development Efforts

Information RegardingCurrent State,

Vision for Future

Enterprise Models,Vision For Future


Agile Database Techniquesfor Agile DBAs

Evolutionary developmentData models dont drive object modelsObject models dont drive data models

Agile Model Driven Development (AMDD)Database Refactoring

Database encapsulation strategiesMapping techniques

Object/data implementation issues


Evolutionary Development

RefactorMap

Objects toData

Model theObject

Schema

Model theData

Schema

PerformanceTune

Iterative &Incremental

Implement


Data Models Should Not Drive Object Models

AddressId: INT24 Street: VARCHAR(30) City: VARCHAR(30) State: VARCHAR(30) Country: VARCHAR(30) PostCode: VARCHAR(20)

Address

Physical Data Model

- street: string - city: string - state: string - zipCode: string

USAddress

Class Models

- street: string - city: string - state: string

USAddress - zipCode: int

ZipCode

+ validate(int stateCode): boolean+ asLabel(): string- getPlusFourNumber(): int- getPostStateNumber(): int- getStateNumber(): int

10..*

- street: string - city: string - state: string - postCode: string

USAddress

- country: string

InternationalAddress


Object Models Should Not Drive Data Models

maximumSpeed wingSpan

Bird

numberOfClaws scaleColors

Lizard

name fireCapacity

Dragon

Class Model Possible Physical Data Models

CreaturePOID Name FireCapacity MaximumSpeed WingSpan NumberOfClaws ScaleColors

Creature

BirdPOID MaximumSpeed WingSpan

Bird

LizardPOID NumberOfClaws ScaleColors

Lizard

DragonPOID Name FireCapacity MaximumSpeed WingSpan NumberOfClaws ScaleColors

Dragon

CreaturePOID MaximumSpeed WingSpan

Bird

CreaturePOID NumberOfClaws ScaleColors

Lizard

CreaturePOID Name FireCapacity

Dragon


What is AM?? AM is a chaordic, practices-based process for

modeling and documentation.? AM is a collection of practices based on several

values and proven software engineering principles

? www.agilemodeling.com

YourProcessBase Software Process

(XP, UP, DSDM, ...)

Agile Modeling (AM)


What Are Agile Models? ? Agile models:

? Fulfill their purpose? Are understandable? Are sufficiently accurate? Are sufficiently consistent? Are sufficiently detailed? Provide positive value? Are as simple as possible

? Agile models are just barely good enough!


Agile Model Driven Development (AMDD)(www.agilemodeling.com/essays/amdd.htm)


The Core of AMCore Principles

? Assume Simplicity? Embrace Change? Enabling the Next Effort is Your

Secondary Goal? Incremental Change? Model With a Purpose? Multiple Models? Maximize Stakeholder Investment? Quality Work? Rapid Feedback? Software Is Your Primary Goal? Travel Light

Core Practices? Active Stakeholder Participation? Apply the Right Artifact(s)? Collective Ownership? Consider Testability? Create Several Models in Parallel? Create Simple Content? Depict Models Simply? Display Models Publicly? Iterate to Another Artifact? Model in Small Increments? Model With Others? Prove it With Code? Use the Simplest Tools


Iterative Modeling(www.agilemodeling.com/essays/phasesExamined.htm)

&RQFHSWXDO' RP DLQ0 RGHOLQJ

&ODVV5 HVSRQVLELOLW\ &ROODERUDWRU &5& &DUGV/ RJLFDO' DWD0 RGHO / ' 02EMHFW5 ROH0 RGHO 2 5 0 ' LDJUDP5REXVWQHVV' LDJUDP8 0 / &ODVV' LDJUDP

8 VDJH0 RGHOLQJ

( VVHQWLDO8 VH&DVHV) HDWXUHV6 \ VWHP 8 VH&DVHV8 VHU6 WRULHV8 0 / 8 VH&DVH' LDJUDP

6XSSOHP HQWDU\ 5 HTXLUHP HQWV0 RGHOLQJ

%XVLQHVV5 XOHV&RQVWUDLQWV* ORVVDU\7HFKQLFDO5 HTXLUHP HQWV

8VHU,QWHUIDFH' HYHORSP HQW

( VVHQWLDO8 VHU,QWHUIDFH3URWRW\ SH8VHU,QWHUIDFH) ORZ ' LDJUDP8VHU,QWHUIDFH3URWRW\ SH

$ UFKLWHFWXUDO0 RGHOLQJ

&KDQJH&DVHV) UHH) RUP ' LDJUDP8 0 / &RP SRQHQW' LDJUDP8 0 / ' HSOR\ P HQW' LDJUDP8 0 / 3 DFNDJH' LDJUDP

' \ QDP LF2 EMHFW0 RGHOLQJ

8 0 / &RP P XQLFDWLRQ' LDJUDP8 0 / &RP SRVLWH6WUXFWXUH' LDJUDP8 0 / ,QWHUDFWLRQ2 YHUYLHZ ' LDJUDP8 0 / 6HTXHQFH' LDJUDP8 0 / 6WDWH0 DFKLQH' LDJUDP8 0 / 7LP LQJ' LDJUDP

3URFHVV0 RGHOLQJ

' DWD) ORZ ' LDJUDP ' ) ') ORZ&KDUW8 0 / $FWLYLW\ ' LDJUDP

' HWDLOHG6WUXFWXUDO0 RGHOLQJ

3K\VLFDO' DWD0 RGHO 3' 08 0 / &ODVV' LDJUDP


Database Refactorings? A database refactoring is a simple change to a

database schema that improves its design while retaining both its behavioral and informational semantics. But this is a just good enough definition and thats ok.

? A database schema includes both structural aspects such as table and view definitions as well as functional aspects such as stored procedures and triggers.

? In many ways database refactoring is simply normalization after the fact.

? Important: Database refactorings are a subset of schema transformations, but they do not add functionality.


Examples of Database Refactorings? Apply Standard Types to Similar Data? Consolidate Key Strategy for Entity? Encapsulate Common Structure With View? Introduce Column Constraint? Introduce Common Format? Introduce Lookup Table? Migrate Database Method to Application? Rename Column? Replace One-To-Many With Associative Table? Replace View With Method(s)? Split Column


Database Refactoring ExampleReplace Column


Why DB Refactoring is Hard

YourDatabase

YourApplication

OtherDatabases

OtherApplications

You Know About

OtherApplicationsYou Don't

Know About

DataFile

DataExtracts

DataFile

DataImports

PersistenceFrameworks

TestCode


Enablers for Database Refactoring

? Regression testing? Strong configuration management

approach You need to version all system artifacts, including data

? Willingness to work together? Acceptance of your situation


Encapsulation Strategies

? Coupling is enemy #1? Encapsulation is your ally? Encapsulation strategies:

? Brute force (embedded SQL)? Data access objects? Persistence frameworks? Services


Fundamental Mapping Issues(www.agiledata.org/essays/mappingObjects.html)

? Mapping Inheritance? One Table Per Hierarchy? One Table Per Concrete Class? One Table Per Class

? Mapping Associations? Why Mapping Isnt Straightforward


Mapping InheritanceExample Problem

Person{abstract}

namephoneNumber

Employee

startDate

Customer

customerIDpreferences

Executive

bonus

Person{abstract}

namephoneNumber

Employee

startDate

Customer

customerIDpreferences


Mapping InheritanceOne Table Per Hierarchy

OIDnamephoneNumbercustomerNumberpreferencesstartDateobjectType

PersonOIDnamephoneNumbercustomerNumberpreferencesstartDatebonusobjectType

Person


Mapping InheritanceOne Table Per Concrete Class

OIDnamephoneNumberstartDate

EmployeeOIDnamephoneNumbercustomerNumberpreferences

CustomerOIDnamephoneNumberstartDatebonus

Executive


Mapping InheritanceOne Table Per Class

OID (FK)startDate

EmployeeOID (FK)customerNumberpreferences

Customer

is a

OIDnamephoneNumberobjectType

Person

is a

OID (FK)startDate

EmployeeOID (FK)customerNumberpreferences

Customer

is a

OIDnamephoneNumberobjectType

Person

is a

OID (FK)bonus

Executive

is a


Mapping Associations

customerOIDfirstName: StringlastName: String

Customer

accountNumbercurrentBalance: Float

Account

customerOIDaccountNumber

Accesses

Customer

+purchaseItems()+complain()

-firstName : String-lastName : String

Account

+deposit()+withDraw()+open()+close()

-accountNumber : Integer-currentBalance : Currency

1..n

1..n

accesses


Mapping Isnt Straightforward

Address

+label() : String

-street : String-city : String-state : String-country : String

ZipCode

+label() : String+validate() : Boolean

-number : Number1..1

1..1

addressOIDstreet: stringcity: stringzipcode: integerstate: stringcountry: string

Address

addressOIDstreet: stringcity: stringzipcode (FK)state: stringcountry: string

Address

zipcodedescription: string

Zip Code

- OR -


Object/Data Implementation Issues

? Caching? Concurrency control? Database architecture? Legacy data sources? Performance tuning? Persisting relationships? Referential integrity? Versioning objects


Conclusion

? We need to work together? We need to rethink our approach to

development? The religious arguments are no longer

acceptable? We need to get on with the job? The software world has changed. Have you?? Visit www.agiledata.org


Keep in Touch

Scott W. Ambler

www.ronin-intl.com/company/scottAmbler.html

Documents

Agile Database Techniques