ALM and practical guidance for VSTS Database Projects

ALM and practical guidance for VSTS Database ProjectsJens K. SuessmeyerDeveloper hearted, relational mindedVisual Studio ALM Core Rangerhttp://blogs.msn.com/b/JensS

Session objectives

• Get to know the database project advantages over common database development approaches

• Get to know the relevance of Application lifecycle management for a database

• Map the stages of an ALM process to database projects• Get to know the importance of a formalized process to

develop, maintain, test and deploy databases• Get to know what is in the Visual Studio ALM Ranger

Guidance for Database projects for you

2

about:me

• Jens K. Süßmeyer, Consultant from Microsoft Services Germany

• Main focus areas• .Net Development• Database development• Consulting

• SQL Server 2008 MCM• Visual Studio ALM Core Ranger

3

Question

Did you ever had inconsistencies in

your database development that you think could have been

avoidable ?

General understanding of lifecycle Management

5What is ALM ? (http://go.microsoft.com/?linkid=9743693)

http://go.microsoft.com/?linkid=9743693


Database Lifecycle

DatabaseProject BuildDeployment

Test

Development

Service Creation (Requirements definition / Design / Implementation)

Service management (Deployment / Operation / Optimizing)

The evolution of development tools

7Tools for team development (http://go.microsoft.com/?linkid=9743692)



Version Control Challenge

CREATE TABLE dbo.Auction( id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)

class AuctionApplication( int id; void MethodA();)

Database

class AuctionApplication( int id; void MethodA(); void MethodB();)

class AuctionApplication( int id; string cacheTitle; void MethodA(); void MethodB();)

Revision History

App

V 1 V 2 V 3

ALTER TABLE dbo.Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)

ALTER TABLE dbo.Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)

Hand crafted versioning

-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(

id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)END-- version 2 Add PK Au_PKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_PK' AND type = 'PK')BEGIN

ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id)END-- version 3 Add UC Au_SKIF NOT EXISTS (SELECT * FROM sys.key_constraints WHERE name = 'Au_SK' AND type = ‘UQ')BEGIN

ALTER TABLE Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)END






V1 => V2 -- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(




V2 => V3

V2 => V3-- version 1 Add table dbo.AuctionIF OBJECT_ID (N'dbo.Auction', N'U') IS NULLBEGINCREATE TABLE dbo.Auction(




V3 => V4













V1 => V4


• Instead of having to guess the order by names, scripts would need to be omnipotent and not version agnostics in first place.

• Produces a high effort in maintaining scripts and persisting the knowledge of the order

The enemy of a copy & paste deployment -Data

• Unlike deployments of applications / assemblies in common, the database deployment will need to preserve data

• Incremental deployment needs to provide a way to migrate data without loosing information or violate referential integrity leaving the database in an inconsistent state

• Data scripts will need to honor the dependencies, constraints, dependent objects

Deployment – Visual Studio Database Projects

CREATE TABLE dbo.Auction( id INT NOT NULL PRIMARY KEY, name VARCHAR(25) NOT NULL UNIQUE, start DATETIME NULL, len INT NULL)

CREATE TABLE dbo.Auction( id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)

CREATE TABLE dbo.Auction( id INT NOT NULL PRIMARY KEY, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL)

IncrementalDeployment

LogicalDatabase

Revision HistoryV 1 V 2 V 3

CREATE TABLE dbo.Auction( id INT NOT NULL PRIMARY KEY, name VARCHAR(25) NOT NULL UNIQUE, start DATETIME NULL, len INT NULL)

NewDeployment ALTER TABLE dbo.Auction

WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name)

The advantages of Database projects

• The from the database uncoupled schema model contains all the necessary information to create / update a database and represents the state “as-should-be”.

• The schema motion scripts are created on deployment time, NOT development time, letting the developer focus on his development, NOT deployment goals. Referential constraints etc. Are automatically honored.

• Maintaining a conceptual models instead of scripts let you do easily use all the features of the ALM platform, like merging Branching

Database development with various tools

15

Benutzer

Testing Database

TesterProductionalDatabase

SQL

Administrator

Design Database

Source Control System

SQLSQL SQLSQL

Developer

Visual Studio ALM Rangers• Mission

• Accelerate adoption of Visual Studio with out-of-band solutions for missing features or guidance• Joint Venture

• Co-funded and co-lead by Services and PG• Bijan Javidi/PG and Jeff Jurvis/Services

• Community readiness • Gather field experience• Empower the community

• Collaboration• ~130 Extended Rangers• ~46 External Rangers• 10 Core Rangers

• Find us here …• MSDN • Blog http://tinyurl.com/RangersBlog• ALM Rangers IP on Campus http://tinyurl.com/WW-VS-ALM-Rangers

• Want to join?• Email us on [email protected]

Visual Studio Database Projects guidance Rangers guidance vs. MSDN

• MSDN explains features not scenarios

• This guidance is scenario based

• Shows ideas and experiences based on real world engagements

The guidance materialization

• We provide best practices and guidance in each sub topics in a

• Guidance document

• Corresponding HOLs

Application lifecycle sub topics

Development


Test

Development



Development

• Why 1 + 1 is sometimes 0…

Development

• Internal database validation decreases the chance of getting unsuccessful builds or inconsistent projects artifacts by modeling and validating the schema on the fly

• Developers can be provided with templates to adhere to certain guidelines and with static code analysis rules to ensure coding standards

• Developers can work without having any dependency on a database prior to deployment.

Development / Database guidance

• The database guidance document shows how to

• manage projects • streamline development• when to use / not to use certain features like different project

types (Server , Composite / Partial)• how to apply known concepts of existing guidance documents

like Branching & Merging to the database world • be aware of limitations during development and project

management and how to get around them

DEMOWorking with database projects

Build


Test

Development



Build

• Why a successful build is sometimes not useful at all …

Build

• Like any other projects Database projects can be included in your build definition within TFS or any external build processes.

• The guidance will in addition help you to:• Continuous integration with VSDB• Building and Deploying outside Team Build• Inclusion of automated tests / test lists to be included within

the build• Provide you with the option to deploy databases

automatically with the build• Build and Deployment packaging using WIX

DEMOManaging a (W)indows (I)nstaller (X)ML Project for Database Projects

Test


Test

Development



Test

• Why testing should be important…

Test

• The cost of lousy tests…

Test

• Creation of tests was put into a designer to enable database developers and tester to create their Unit tests without having to learn any coding language

• Unit Tests can run on sample data as well as random or real data to ensure a stable quality

• Unit Tests can be included in the automated build to ensure continuous checks of your database artifacts automate processes.

Test / Creating useful data

•Simple data generators

• Strings (char, varchar, nvarchar…)• Numbers (smallint, int, bigint, float…)• Binary (varbinary, image…)• Date and Time• GUID and Bit

•Complex generators

• Foreign Key• Regular Expression / Unique Expression Generator• Data Bound / Sequential Data Bound• FileBound Generator• XMLGenerator

You can cover the full complexity of data types here to make your code robust !

Test / Creating useful data

• Extend it to your needand extend the frameworkby simply writing your data generator. Publishing workswith copy & paste to theVisual Studio machine. Include interface that you might already use by accessing internals data sources, etc.

Test / Database guidance

• The database guidance document shows how to

• create simple tests and integrate them in the project• create a multi-step process to build / deploy / load databases

with random or live data and test against that• Integrate your tests within your automated build definitions

DEMOIntegrated creation of Unit tests

Test

• Extend it to your needand extend the frameworkby simply writing your owncondition. Publishing workswith copy & paste to theVisual Studio machine

Deployment


Test

Development



Deployment

• Without knowing your deployment environment you might be surprised of the results…

Deployment

• The transistion to deployment should much more than just throwing the build artifacts over the fence. There is much more valueable information buried in here:

• Finding bugs and deployment issues early• Feasability of upgrade scenarios / discover major shostoppers• Durations of database upgrades• Needed resources for database upgrades (like log space, usage

of database parameters, etc.)• Building confidence about the delivered quality

Deployment

In a perfect world, most of the deployment checks can be

done on the „assumed“ database enviroment as the

source control is the source of truth.

Deployment

Database administrator doubts

• “The database upgrade will drop tables and created index and will make my database slow.”

• “I doubt that the VSDBCMD process really does all the magic behind the scenes and honors all changes I have done in the database”

• “I will not run any command line tool against the database before I haven´t seen the actual .sql code executed beforehand.”

Deployment

Database developer doubts

• “We did all the upgrade testing against the database, but who ensures that the assumed schema on the real database still exists?”

• “The database administrators created several objects in addition which we cannot guarantee to be preserved over the deployment.”

• “The duration of the upgrade process is unpredictable as the database administrator might have created several indexes which need to be preserved.”

Deployment

Finding model drifts (extract from Guidance document)

• Make sure that you tested the application database with the right version you once have been given WITHOUT having access to the database (Trusted DBA version)

• Make sure that you tested the application database with the right version you once have been given having access to the database (Trusted developer version)

• Review the deployment script prior executing them in the database (Suspicious DBA version)

DEMOFinding model drifts

Summary

• Visual Studio Database Projects can ease development , build, test and deployment process by providing the right tools.

• VSTS Database Projects cover the operative part of ALM for Database projects.

• Many formalized processes with little effort will help you to ensure quality across the database projects

• The Database Guidance document can help you to find out how you can get around limitations, implement best practices and handle special situations.

46

Stay up to date with MSDN Belux

• Register for our newsletters and stay up to date:http://www.msdn-newsletters.be• Technical updates• Event announcements and registration• Top downloads

• Follow our bloghttp://blogs.msdn.com/belux

• Join us on Facebookhttp://www.facebook.com/msdnbehttp://www.facebook.com/msdnbelux

• LinkedIn: http://linkd.in/msdnbelux/ • Twitter: @msdnbelux

Download MSDN/TechNet Desktop Gadget

http://bit.ly/msdntngadget

TechDays 2011 On-Demand

• Watch this session on-demand via Channel9http://channel9.msdn.com/belux

• Download to your favorite MP3 or video player• Get access to slides and recommended resources by the speakers

Additional Ressources

• My blog: http://blogs.msdn.com/JensS• Barclay Hill (PM of Visual Studio Data Tools):

http://blogs.msdn.com/BaHill • Visual Studio Database Projects Guidance:

http://vsdatabaseguide.codeplex.com/ • Visual Studio Rangers: http://tinyurl.com/ALMRangers

http://blogs.msdn.com/JensS

http://blogs.msdn.com/BaHill

http://vsdatabaseguide.codeplex.com/

http://vsdatabaseguide.codeplex.com/

Questions ??

? ?

THANK YOU

Documents

ALM and practical guidance for VSTS Database Projects