Upload
techwellpresentations
View
105
Download
0
Tags:
Embed Size (px)
DESCRIPTION
As developers, we've created heuristics that help us build robust systems and employed test-driven development (TDD) to improve code design and counter instability. Yet object-oriented development principles and TDD have failed to gain traction in the database world. That’s because database development involves an additional driving force-the data. Max Guernsey shows how to treat databases as objects with classes of their own-rather than as containers of objects-and how to drive database designs from tests. He illustrates a way to give these database classes the ability to upgrade old data without introducing undue risk. Max also shares how to apply good object-oriented design principles to database classes and how to enforce semantic connections between databases and clients. Max demonstrates how it all works together, ensuring that your production databases work exactly the same as test databases, minimizing the risk of design changes, and enabling client applications to more easily keep up with database changes.
Citation preview
BT6 Concurrent Session 11/8/2012 2:15 PM
"Database Development: The Object-oriented and Test-driven
Way"
Presented by:
Max Guernsey Hexagon Software, LLC
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073 888‐268‐8770 ∙ 904‐278‐0524 ∙ [email protected] ∙ www.sqe.com
Max Guernsey Hexagon Software, LLC
As head of Hexagon Software, LLC and a software developer with fifteen years of experience, Max Guernsey helps organizations make the transition to agility with a focus on good technical practices such as test-driven development and modern object-oriented design. He develops his company’s DataClass product, a language that allows .NET developers to treat databases like “just another object.” Max is the author of Transition Testing, the foundation of his current TDD database work; Test Driven Database Development which focuses on adapting the discipline of TDD to forces present in the database world; and Goad Testingin which he discusses what unit tests really are and how to make them more resilient to design changes.
1
Database Development
The Test‐Driven and Object‐Oriented Way
© Copyright 2011 Hexagon Software LLC.All rights reserved. 19/12/2012
Max Guernsey, III
• Max Guernsey, III• 15 years as a developer• 15 years as a developer• Started w/DB early• Author:
– Test‐Driven Database Development
– Goad Testing
© Copyright 2011 Hexagon Software LLC.All rights reserved. 29/12/2012
– Transition Testing
2
Agenda
• Simplifying Assumptions• Establish the Problem• Roadmap to True TDD DB Development• Follow Roadmap
– Class of Databases– Safely Changing Design– Enforcing Interfaces
© Copyright 2011 Hexagon Software LLC.All rights reserved. 39/12/2012
g– Defining Behaviors– Designing for Maintainability
• OO DB Development (Whirlwind Tour)
To fit in a 75 minute box
© Copyright 2011 Hexagon Software LLC.All rights reserved. 49/12/2012
Simplifying Assumptions
3
Assumptions
• I assume that you knowI assume that you know...– ...know what TDD is– ...the difference between TDD and Test‐First– ...about mocking– ...a little about design patterns
© Copyright 2011 Hexagon Software LLC.All rights reserved. 59/12/2012
• If not, there are great resources for all• Pick them up, read, & practice before talk
Know what’s in our way
© Copyright 2011 Hexagon Software LLC.All rights reserved. 69/12/2012
Establish the Problem
4
Test Runner
[Test]public void AddEmployee() {
var company = Company GetInstance();
TDD in Middle Tier Code
Test Runner
Test1
Test2
Test3
ThisTest
Test5
var company = Company.GetInstance();var employee = Employee.GetInstance();
company.AddEmployee(employee);
Assert.That(company.Employees,Has.Some.SameAs(employee);
}
Setup
Trigger
Assertion
© Copyright 2011 Hexagon Software LLC.All rights reserved. 79/12/2012
Test6
Test7
Test8
}
[Test]public void AddEmployee() {
var company = Company GetInstance();
TDD in Middle Tier Code
var company = Company.GetInstance();var employee = Employee.GetInstance();
company.AddEmployee(employee);
Assert.That(company.Employees,Has.Some.SameAs(employee);
}
Setup
Trigger
Assertion
© Copyright 2011 Hexagon Software LLC.All rights reserved. 89/12/2012
}
5
[Test]public void CannotAddLineItemWithoutType() {
DBTestHelper ClearDatabase();
Typical DB Test
DBTestHelper.ClearDatabase();var cartId = DBTestHelper.CreateCart();
try {DBTestHelper.InsertLineItem(
cart: cartId, itemTypeId: DBNull.Value, count: 100);} catch (ConstraintViolationException) {
return;}
Incomplete setup
Trigger
© Copyright 2011 Hexagon Software LLC.All rights reserved. 99/12/2012
}
Assert.Fail();}
Assertion
How Was the DB Created?
• You don’t know• No guarantee DB works like production• Potential for variation
• Leads to...– ...rigorous “controls” over change– ...general “slow down” culture
© Copyright 2011 Hexagon Software LLC.All rights reserved. 109/12/2012
...general slow down culture– ...ultimately: bad database designs
• DB test suites must cover creation
6
How to solve the problem
© Copyright 2011 Hexagon Software LLC.All rights reserved. 119/12/2012
Roadmap
Roadmap to TDD
• Creating a Class of Databases– So creation is covered by tests
• Testing Creation & Modification– So changes can be introduced w/confidence
• Enforcing Interfaces– So consumers are forced to recognize changes
• Defining “Behavior”
© Copyright 2011 Hexagon Software LLC.All rights reserved. 129/12/2012
g– So your tests cover the right things
• Designing for Maintainability– So you don’t get bogged down by needless junk
7
There’s More!
• OODC ti t f l i l DB / d li & fl ibl d i– Creating systems of logical DBs w/good coupling & flexible designs
• Mocking– Real, useful mocking along interface boundaries
• Legacy Databases– Pulling legacy DBs into new process
• Non‐DB applications– Serialized objects, XML, file structures
© Copyright 2011 Hexagon Software LLC.All rights reserved. 139/12/2012
• Won’t fit in 75 minutes• Shameless plug
– They’re in the book
You don’t want to run tests against production, do you?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 149/12/2012
Creating a Class of Databases
8
What is a Class of Databases?
• StrictlySt ct y– Provisions new DBs– Upgrades existing DBs– All DBs get same behavior
• Effecti el
© Copyright 2011 Hexagon Software LLC.All rights reserved. 159/12/2012
• Effectively– Uniform create/upgrade path– All DBs “grew” same way
Problem: Different Build Paths
• Test databases need to be “cheap”est databases eed to be c eap– Easy to create & destroy
• Production databases need to last– Created w/scripts– Improved incrementally w/scripts
• Want all databases to ork the same
© Copyright 2011 Hexagon Software LLC.All rights reserved. 169/12/2012
• Want all databases to work the same– Test & production
• Exactly the same
9
How to Make Test DBs?
• Test DBs are of low valueest s a e o o a ue• Especially when cheap to make• Break one?
– Pitch it• Got one dirty?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 179/12/2012
– Pitch it
• Make them however you like
How to Make Production DBs?
• Starts emptyStarts empty• Created w/series of instructions• Upgraded w/series of instructions to v2• Upgraded w/series of instructions to v3• Etc
© Copyright 2011 Hexagon Software LLC.All rights reserved. 189/12/2012
• Etc.• No wiggle room
– What actually happens is what actually happens
10
Takeaway #1
© Copyright 2011 Hexagon Software LLC.All rights reserved. 199/12/2012
The Sequence of Transitions
• v0 v1– Get a new v1 DB
• v1 v2– Upgrade from v1 to v2
• v0 v1 v2 v3 v4
© Copyright 2011 Hexagon Software LLC.All rights reserved. 209/12/2012
– Get a new v4 DB• V2 v3 v4
– Upgrade from v2 to v4
11
Test Runner
[Test]public void HasAnAccountsTable() {
// you can do this once per suite if you want
Example: Writing a Test
Test Runner
Test1
Test2
Test3
ThisTest
Test5
// you can do this once per suite if you wantMyDatabase.Create(connection, 2);
connection.ExecuteSql("SELECT * FROM Accounts");
// exception thrown if no Accounts table}
DB created by test
© Copyright 2011 Hexagon Software LLC.All rights reserved. 219/12/2012
Test6
Test7
Test8
<database><version number="1">
<script>
Example: Starting DB Script
<script><!-- Build the database -->
</script></version>
</database>
© Copyright 2011 Hexagon Software LLC.All rights reserved. 229/12/2012
12
Test Runner
<database><version number="1">
<script>
Example: Updating DB Design
Test Runner
Test1
Test2
Test3
ThisTest
Test5
<script><!-- Build the database -->
</script></version><version number="2">
<script><![CDATA[CREATE Accounts(INT ID, NAME CHAR(23));]]>
© Copyright 2011 Hexagon Software LLC.All rights reserved. 239/12/2012
Test6
Test7
Test8
]]></version>
</database>
Reducing the risk of change rather than its frequency
© Copyright 2011 Hexagon Software LLC.All rights reserved. 249/12/2012
Safely Changing Design
13
“Neat”
• Finding uniform build path a beginningd g u o bu d pat a beg g• Must drive error/variation from build
• Builders/installers/etc. are code• What drives error/variation from code?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 259/12/2012
• TESTS!!
Takeaway #2
© Copyright 2011 Hexagon Software LLC.All rights reserved. 269/12/2012
14
A Transition Test
• SetupSetup– Bring database up to version X– Populate
• Trigger– Transition to subsequent version
© Copyright 2011 Hexagon Software LLC.All rights reserved. 279/12/2012
• Assertion– Prove that content was preserved
Test Runner
[Test]public void ContactsFactoredFromAccountsInV9() {MyDatabase.Create(connection, 8);
Example: A Transition Test
Test Runner
Test1
Test2
Test3
ThisTest
Test5
var accountId = connection.ExecuteScalar(@"INSERT INTO Accounts(Name, PWHash, Email)VALUES('MaxGuernseyIII', 'pwhash', '[email protected]');
SELECT SCOPE_IDENTITY();");
MyDatabase.Create(connection, 9);
var emails = connection.ExecuteSelectList(@"SELECT Email FROM Contacts WHERE AccountID = ?",
© Copyright 2011 Hexagon Software LLC.All rights reserved. 289/12/2012
Test6
Test7
Test8
accountId);Assert.That(emails,Is.EqualTo(new[] { "[email protected]" });
}
15
Test Runner
<database><version number="9">
<script>
Example: Creating Upgrade Script
Test Runner
Test1
Test2
Test3
ThisTest
Test5
p<![CDATA[CREATE TABLE Contacts(
AccountId INT FOREIGN KEY REFERENCES Account(ID),Email NVARCHAR(2048));
INSERT INTO Contacts(AccountID, Email)SELECT ID, Email FROM Accounts;
ALTER TABLE A t DROP COLUMN E il
© Copyright 2011 Hexagon Software LLC.All rights reserved. 299/12/2012
Test6
Test7
Test8
ALTER TABLE Accounts DROP COLUMN Email;]]>
</script></version>
</database>
Getting the best feedback you can
© Copyright 2011 Hexagon Software LLC.All rights reserved. 309/12/2012
Enforcing Interfaces
16
[Test]public void HasAnAccountsTable() {
// you can do this once per suite if you want
Remember This?
// you can do this once per suite if you wantMyDatabase.Create(connection, 2);
connection.ExecuteSql("SELECT * FROM Accounts");
// exception thrown if no Accounts table}
© Copyright 2011 Hexagon Software LLC.All rights reserved. 319/12/2012
Duplication Is Evil
• Two tests is annoyingTwo tests is annoying• Ten is a pain• Fifty is untenable • Any realistic number impossible to maintain w/duplication
© Copyright 2011 Hexagon Software LLC.All rights reserved. 329/12/2012
w/duplication
17
[Test]public void HasAnAccountsTable() {
// you can do this once per suite if you want
Easy Solution
y p yMyDatabase.Create(connection, MyDatabase.CurrentVersion);
connection.ExecuteSql("SELECT * FROM Accounts");
// exception thrown if no Accounts table}
...
© Copyright 2011 Hexagon Software LLC.All rights reserved. 339/12/2012
public class MyDatabase {public const int CurrentVersion = 2;...
}
It’s More Pervasive
• Version numbers just most obviousVersion numbers just most obvious• Symbols are the real evil...
© Copyright 2011 Hexagon Software LLC.All rights reserved. 349/12/2012
18
[Test]public void CanInsertData() {connection.Execute(@"INSERT INTO Data(Time, Reading)VALUES('4:31:01 103 17 02 2009' 041 908);”);
Duplication Abounds
VALUES( 4:31:01.103 17.02.2009 , 041.908); );}...<script>CREATE TABLE Data(Time DATETIME, Reading NUMERIC(6,3));
</script>...private void EnterReading(double reading) {connection.Execute(@"INSERT INTO Data(Time, Reading) VALUES(?,?)",DateTime.Now, reading);
© Copyright 2011 Hexagon Software LLC.All rights reserved. 359/12/2012
}...private IEnumerable<double> GetOrderedReadings() {return connection.Execute<double>(@"SELECT Reading FROM DataORDER BY Time ASC");
}
One Solution
• Minimize w/string constantsMinimize w/string constants• Couple to constants from tests & code
• What about interface changes?– Transition tests need old design
© Copyright 2011 Hexagon Software LLC.All rights reserved. 369/12/2012
Transition tests need old design– Unit tests & production code always need latest
19
Another Solution
• Document design w/each transitionDocument design w/each transition• Couple transition tests to exact• Couple unit tests to latest• Couple production code to latest
© Copyright 2011 Hexagon Software LLC.All rights reserved. 379/12/2012
• What about DB script?
A Really Good Solution
• Document design changes w/versionocu e t des g c a ges / e s o• Bind to design in scripts• Pre‐compile/package step:
– Inject bound symbols– Generate coupling codeC l i l d i
© Copyright 2011 Hexagon Software LLC.All rights reserved. 389/12/2012
• Couple unit tests to latest design• Couple production code to latest design• Couple transition tests to specific design
20
<database><version number="2"><design>
Example Script
Generate code from this before compilation/packaging
<add id="NewTable"><add id="Col1" /><add id="Col2" />
</add></design><script><scope id="NewTable">
CREATE TABLE <id/>(<bind>Col1</bind> INT,
"Take the version 1 design and add thisstuff"
Replace w/ "NewTable" prior to packaging
© Copyright 2011 Hexagon Software LLC.All rights reserved. 399/12/2012
<bind>Col2</bind> CHAR(1));</scope>
</script></version>
</database>
Replace w/ "Col2" prior to packaging
Takeaway #3
© Copyright 2011 Hexagon Software LLC.All rights reserved. 409/12/2012
21
Replacing Symbol w/Itself?
• 1st‐ly: You can do morey ou ca do o e– E.g.: Use different DB and logical names
• 2nd‐ly: Gives advantages of compiler– across languages (SQL, C#, C++, etc.)
' b li il h d ?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 419/12/2012
• Don't believe compiler has advantages?– You're crazy– Keep it to yourself
If databases are objects, what are their behaviors?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 429/12/2012
Defining Behaviors
22
What Do Tests Test?
• Tests specify behaviorsTests specify behaviors• Objects contain behaviors• Databases are objects• DB tests should specify DB behaviors
© Copyright 2011 Hexagon Software LLC.All rights reserved. 439/12/2012
Concepts
• Knowledge – persistent data of valueKnowledge persistent data of value• Information – useful part of signal• Data – superclass of knowledge & information• Behavior – Activity that produces a result
– Answer to a question
© Copyright 2011 Hexagon Software LLC.All rights reserved. 449/12/2012
Answer to a question– Useful side effect
23
What Are Behaviors for Transitions?
• Transition behaviors simpleTransition behaviors simple• Change from old structure to new• Preserve knowledge
• Ignored for this segment
© Copyright 2011 Hexagon Software LLC.All rights reserved. 459/12/2012
• Ignored for this segment
What Are DB Behaviors?
• DB behaviors slightly more subtleDB behaviors slightly more subtle• Absorb information & translate to knowledge• Translate knowledge into requested info.
• A table is no behavior
© Copyright 2011 Hexagon Software LLC.All rights reserved. 469/12/2012
• A table is no behavior• A trigger is not behavior• Those are structure
24
Knowledge, Info, & Behavior
Client
DB
KnowledgeInfo.
© Copyright 2011 Hexagon Software LLC.All rights reserved. 479/12/2012
TestBehavior
Takeaway #4
© Copyright 2011 Hexagon Software LLC.All rights reserved. 489/12/2012
25
Don't and Do
Don't Do• Test a table• Test a trigger• Expose structure
– E.g.: tables etc.
• Test store & retrieval of info.• Test info causes side effect• Encapsulate structure
– w/Views & stored procs
© Copyright 2011 Hexagon Software LLC.All rights reserved. 499/12/2012
Why Encapsulate Design?
• Weird question: Why not?Weird question: Why not?
• Hidden things cheap to change• Exposed ones expensive• Minimizing public interface maximizes
© Copyright 2011 Hexagon Software LLC.All rights reserved. 509/12/2012
• Minimizing public interface maximizes flexibility
26
Doing less now enables doing more later
© Copyright 2011 Hexagon Software LLC.All rights reserved. 519/12/2012
Designing for Maintainability
Don't Plan the Future
• Future is unpredictableFuture is unpredictable• You cannot plan it• Maybe your guess is right• Maybe we all die from asteroid strike• Usually: your guess "would have" been right
© Copyright 2011 Hexagon Software LLC.All rights reserved. 529/12/2012
• Usually: your guess would have been right– If only universe had cooperated
27
Plan for the Future
• Plan for the future to happenPlan for the future to happen– You'll need to change stuff
• Design so that change is easiest• Largely: what's been shown so far• Additionally: don't overbuild
© Copyright 2011 Hexagon Software LLC.All rights reserved. 539/12/2012
Additionally: don t overbuild
• "What does that mean?"
Takeaway #5
© Copyright 2011 Hexagon Software LLC.All rights reserved. 549/12/2012
28
Planning for the Future
• Encapsulate design– So that structures can change
• Enforce interface strongly– So changes don't break downstream functionality
• Cover every behavior w/test– So changes don't introduce regression
• Cover every transition w/test
© Copyright 2011 Hexagon Software LLC.All rights reserved. 559/12/2012
y /– So changes don't jeopardize knowledge
• Minimize features to those needed now– So needed features not impeded by useless ones
Time to watch this work
© Copyright 2011 Hexagon Software LLC.All rights reserved. 569/12/2012
Live Example
29
A whirlwind tour
© Copyright 2011 Hexagon Software LLC.All rights reserved. 579/12/2012
The Rest
OOD
• Once classes of DB used, OOD/OOP availableO ce c asses o used, OO /OO a a ab e• Use it• Divide & conquer
– Create composite DBs– Couple via public interface
b i & i i
© Copyright 2011 Hexagon Software LLC.All rights reserved. 589/12/2012
• Even create abstractions & variations– Exploit division to introduce variation– Most important variation on next slide
30
Mocking
• Once you have OOD/OOP easy good mockingOnce you have OOD/OOP, easy, good mocking available
• Don't mock tables• Mock behaviors encapsulated in dependency DBs
© Copyright 2011 Hexagon Software LLC.All rights reserved. 599/12/2012
Legacy DBs
• Mocking especially handy w/legacy DBsoc g espec a y a dy / egacy s• Create Façade DB, connects to legacy• Easy to test‐drive Façade behaviors:
– Just mock out legacy• Transfer behavior to Façade over time
© Copyright 2011 Hexagon Software LLC.All rights reserved. 609/12/2012
• Alternatively:– "wrangle" legacy DB into tested state
31
Non‐DB Applications
• Much of this works for non‐DB– (not recommendation for views & sprocs, obviously)
• Have data? Want to evolve structure over time?– This process works
• E.g.:– File systems/directory structures
© Copyright 2011 Hexagon Software LLC.All rights reserved. 619/12/2012
File systems/directory structures– Registry keys– XML documents– Serialized objects
Thanks for coming. Any other questions?
© Copyright 2011 Hexagon Software LLC.All rights reserved. 629/12/2012
Q&A