SE2016 Java Roman Ugolnikov "Migration and source control for your DB"

Migrationа and source control for your DB

what:Database migration: Database structure: - Tables, constrains, indexes;

Database data:

- Initial data like list of post codes,

statuses for order, etc.

Database logic:

- stored procedures, triggers, functions

why: versionsData Base migration: Data base structure: - Tables, constrains, indexes;

Data base data:

- Initial data like list of post codes,

statuses for order, etc.

Data base logic:

- stored procedures, triggers, functions

v1.0.0

why: versionsData Base migration: Data base structure: - Tables, constrains, indexes;

Data base data:

v1.0.0

why: versionsData Base migration: Data base structure: - Tables, constrains, indexes;

Data base data:

- Initial data like list of post co

v1.2.1

v1.0.0

Data Base migration: Data base structure: - Tables, constrains, indexes;

Data base data:

- Initial data like list of post

v1.2.1

v1.6.0

v1.0.0

why: versions

Data Base migration: Data base structure: - Tables, constrains, indexes;

Data base data:

- Initial data like list of post c

v1.2.1

v1.6.0

v2.0.1

v1.0.0

why: versions

Data Base migration: Data base structure: - Tables, constrains, indexes;

Data base data:

- Initial data like list of post codes,

statuses for order, etc.

Data base logic:

- stored procedures, triggers, functions

v1.2.1

v1.6.0

v2.0.1

v1.0.0

why: versions

Data Base migration: Data base structure: - Tables, constrains, indexes;

Data base data:

- Initial data like list of post codes,

statuses for order, etc.

Data base logic:

- stored procedures, triggers, functions

v1.2.1

v1.6.0

v2.0.1

v1.0.0DEV

why: environments

Conclusion

We need version control over our database

migration

Tools

Play framework:migration

Rails ActiveRecord migrations

Mogeez for

Tools

Play framework:migration

Rails ActiveRecord migrations

Mongeez for

Tools

Play framework:migration

Rails ActiveRecord migrations

Mogeez for

Tools

Play framework:migration

Rails ActiveRecord migrations

Mogeez for

Tools

Liquibase :

•Apache license

•Started in 2006 (active)

•http://liquibase.org

Flyway:

•Apache license

•Started in 2010

•http://www.flywaydb.or

g

Common flow

Build Test Run

• Dev• Test• Acceptance• Production

• GUI tests• Integration

tests• API tests• Unit tests

• Ant• Maven• Gradle

Common flow

Build Test Run

• Dev• Test• Acceptance• Production

• GUI tests• Integration

tests• API tests• Unit tests

• Ant• Maven• Gradle

Common flow

Build Test Run

• Java API• Spring Bean

• Plugins• CLI

• Plugins• CLI

How it worksDatabaseChangelogLock:

DatabaseChangelog:

How it works

Schema_versions:

How it works

Sequential change flow1. Init db

2. Add column “type”

3. Add new table

How it works

Sequential change flow1. Init db

2. Add column “type”

3. Add new tableX

How it works

Sequential change flow1. Init db

3. Add new table

4. Remove column “type”

2. Add column “type”

difference?, comparison.Rollback vs no rollback

SQL outputConditions

baseline

Supported modes:https://flywaydb.org/documentation/gradle/Flyway more and esear.

How it works

V1__Initial_Setup.sql

V1_1__First_Changes.sql

V1_2__Added_user_info.sql

V2__ ….

How it works

Changes are grouped into changesets: • Change(s) that should be applied atomically

Changesets are grouped into changelogs:•Files managed in version control

How it works

v1_changeLog.xml

v1.1_changeLog.xml

v2_changeLog.xml

v1.2_changeLog.xml

v1.3_changeLog.xml

v2.1_changeLog.xml

…

Supported db listMySQLPostgreSQLOracleMSSQL ServerSybase_EnterpriseSybase_AnywhereDB2Apache_DerbyHSQLH2InformixFirebirdSQLite

MySQLPostgreSQLOracle MSSQL ServerSQL AzureMariaDBPhoenixVerticaAWS RedshiftDB2, DB2 z/OSDerbyH2, Hsql, SQLite, SAP HANA, solidDB, Sybase ASE

Supported formats

DSL for database changes:

• XML• YAML• JSON• sql

Pure SQL

Base lineConvention over configuration

Toolbox

Toolbox

• SQL output• Rollback• Context• Precondition

Toolbox: precondition

<preConditions onFail="WARN" onError="MARK_RAN"><dbms type="oracle"/><runningAs username="SYSTEM"/><tableExists tableName=“newTable"/><customPrecondition className=“com.example.CheckOne"/>

<sqlCheck expectedResult="0">select count(*) from oldtable

</sqlCheck></preConditions>

Toolbox: precondition

<preConditions onFail="WARN" onError="MARK_RAN"><dbms type="oracle"/><runningAs username="SYSTEM"/><tableExists tableName=“newTable"/><customPrecondition className=“com.example.CheckOne"/>

<sqlCheck expectedResult="0">select count(*) from oldtable

</sqlCheck></preConditions>

Toolbox: precondition

<preConditions onFail="WARN" onError="MARK_RAN"><dbms type="oracle"/><runningAs username="SYSTEM"/><tableExists tableName=“newTable"/><customPrecondition className=“com.example.CheckOne"/>

<sqlCheck expectedResult="0">select count(*) from oldtable

</sqlCheck></preConditions>

Toolbox: precondition

<preConditions onFail="WARN" onError="MARK_RAN"><dbms type="oracle"/><runningAs username="SYSTEM"/><tableExists tableName=“newTable"/><customPrecondition className=“com.example.CheckOne"/>

<sqlCheck expectedResult="0">select count(*) from oldtable

</sqlCheck></preConditions>

Toolbox: precondition

<preConditions onFail="WARN" onError="MARK_RAN"><dbms type="oracle"/><runningAs username="SYSTEM"/><tableExists tableName=“newTable"/><customPrecondition className=“com.example.CheckOne"/>

<sqlCheck expectedResult="0">select count(*) from oldtable

</sqlCheck></preConditions>

Toolbox: precondition

<preConditions onFail="WARN" onError="MARK_RAN"><dbms type="oracle"/><runningAs username="SYSTEM"/><tableExists tableName=“newTable"/><customPrecondition className=“com.example.CheckOne"/>

<sqlCheck expectedResult="0">select count(*) from oldtable

</sqlCheck></preConditions>

Toolbox: precondition

<preConditions onFail="WARN" onError="MARK_RAN"><dbms type="oracle"/><runningAs username="SYSTEM"/><tableExists tableName=“newTable"/><customPrecondition className=“com.example.CheckOne"/>

<sqlCheck expectedResult="0">select count(*) from oldtable

</sqlCheck></preConditions>

DEMO

let us try it

DDLv1

DDL

v2

v1

Init data, new column(source)

let us try it

how it works

DDL

v2

v3

v1

Init data, new column(source)

Move values in separate table

let us try it

let us try it

DDL

v2

v3

v1

Init data, new column(source)

Move values in separate table

Thank you for attention

Q & A

uholnikovroman@gmail.com