Medical Systems Co. Ltd

LIQUIBASEDBMS Migration Tools for Java

This is a controlled document. Unauthorized access, copying, replication or usage for a purpose other than for which it is the document have been used for identification purposes only and belong to their respective companies.

Medical Systems Co. Ltd

IQUIBASE 3.5.3DBMS Migration Tools for Java

Version: 0.0.00 (Based on MariaDB 10.1.28)

This is a controlled document. Unauthorized access, copying, replication or usage for a purpose other than for which it is intended, are prohibited. All trademarks that appear in the document have been used for identification purposes only and belong to their

August, 2018

Medical Systems Co. Ltd

3.5.3 DBMS Migration Tools for Java

This is a controlled document. Unauthorized access, copying, replication or usage for a intended, are prohibited. All trademarks that appear in

the document have been used for identification purposes only and belong to their

2 | P a g e

3 | P a g e

© Copyright 2018, Medical Systems Co. Ltd

4 | P a g e

Table of Contents I. Document Information 6 II. Document Release Note 7 III. Notations Used 8

Overview 9

Database Structure: hr_dev 10

1. Generate Change Log 10 2. Managed and Ordered Change log 12

1.1 Change Set for Regions Table 12 1.2 Change set for load data of Departments Table (dev, qac) 13 1.3 Change set for load data of Departments Table (uat, pro) 13 1.4 Change set for Primary Key 13 1.5 Change set for Unique Key 14 1.6 Change set for Index 14 1.7 Change set for Foreign Key Constraint 14

3. Synchronize Change Log 15 4. Database Structure: hr_pro 16 5. Update Change Log 16 6. Differs Change Log 18 7. Difference: hr_pro vs. hr_dev 19 8. Precautions 20

5 | P a g e

Intentionally Blank

6 | P a g e

Document Information

Issued by: Medical Systems Co. Ltd Head Office: K.S.A - Riyadh FAX: +966-1-414 3211 TEL: +966-1-4141 330 Email: devs@medisys.com.my

Service-related contact information worldwide

All service-related contact information is available on official

website. Please follow the link: http://medisys.com.my/ihe

7 | P a g e

Document Release Note

Document Details

Details Revision How to Implement Liquibase Migration for Java 0.0.00

Author

Revision Changes Documented Revised Updated 0.0.00 Initial Release Md. MizanurRahman Md. Shahed Hossain Aug 16, 2018

8 | P a g e

Notations Used

S/N Notation Elaboration 1 DBMS Database Management System 2 RDBMS Relational Database Management System 3 DDL Data Definition Language 4 DML Data Manipulation Language 5 TAB Table, View 6 NDX Index 7 PKC Primary Key Constraint 8 FKC Foreign Key Constraint 9 UKC Unique Key Constraint

10 SQN Sequence 11 PRO Procedure, Production 12 DEV Development 13 QAC Quality Assurance & Control 14 UAT User Acceptance Testing 15 KEY Foreign, Primary, Unique 16 17 18 19 20

9 | P a g e

Overview Liquibase is an open source community project for refactoring also called VCS (Version Controlling Syregardless RDBMS. Only you have to maintain some good practice.

During the migration process running,deployment environments. For example: User Acceptance Test (uat) and filtering the changes logs. Some tags

Although Liquibase is a database refactoring and version control tool where success depends on some good practices. Its means orderlogicalFilePath and relativeToChangelogFile

https://github.com/medisysco/medisys

Figure: Liquibase Migration Test

DROP DATABASE IF EXISTS `hr_dev

DROP DATABASE IF EXISTS `hr_pro

CREATE DATABASE IF NOT EXISTS

CREATE DATABASE IF NOT EXISTS

Snippet: Create Databases in MySQL/MariaDB

is an open source community project for refactoring Database Management System.Controlling System) for Database. Its platform independent migration tools

. Only you have to maintain some good practice.

process running, be aware about context those are directly belongs to the . For example: Development (dev), Quality Assurance & Control (

and Production (pro). There are some labels too;ome tags those are usages for checkout specific commit

database refactoring and version control tool where on some good practices. Its means order of change log id, dbms, context, labels,

and relativeToChangelogFile. Find example project in Github public repository.

https://github.com/medisysco/medisys-dbms-change

Test Resources

hr_dev`;

hr_pro`;

`hr_dev` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci

`hr_pro` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci

reate Databases in MySQL/MariaDB

Database Management System. It’s atabase. Its platform independent migration tools

context those are directly belongs to the Quality Assurance & Control (qac),

those are usages to for checkout specific commit.

database refactoring and version control tool where its performance & id, dbms, context, labels,

public repository.

utf8_general_ci;

utf8_general_ci;

10 | P a g e

Database Structure

Figure: Human Resources Development Database Considering that we have a Human Resources Regarding this scenario we have to generate change logs then organized the change logs according to the good practise and the company policy.and author but need to add/override Where need to specify some initialization/setup data required for production but no transactional data required. In respect of development and quality assurance we need some dummy data. Considering such situation we have defined four type of context those are as respeclabels too. Generate Change Log: Considering that, we are going to implement Log properties as following then add

vim $HOME/.hmis/etc/dbms/liquibase/

Snippet: Configure change log properties

Database Structure: hr_dev

Human Resources Development Database

Human Resources Database in development andRegarding this scenario we have to generate change logs then organized the change logs according to the good practise and the company policy. In generation of change log we can specify the context

add/override some labels, logical path, relative flag, load data

Where need to specify some DDL and DML for different context/environment.initialization/setup data required for production but no transactional data required. In respect of development and quality assurance we need some dummy data. Considering such situation we have defined four type of context those are as respectively dev, qac, uat and

Considering that, we are going to implement Liquibase migration for fist time. Please create then add JDBC properties for respective database.

/.hmis/etc/dbms/liquibase/dev.properties

log properties

and in production too. Regarding this scenario we have to generate change logs then organized the change logs according

change log we can specify the context , logical path, relative flag, load data and etc.

environment. For example initialization/setup data required for production but no transactional data required. In respect of development and quality assurance we need some dummy data. Considering such situation we

pro. There are some

migration for fist time. Please create Change

11 | P a g e

url:jdbc:mysql://localhost:3306/hr_dev?autoReconnect=true&useUnicod

e=true&characterEncoding=utf-8

driver:com.mysql.jdbc.Driver

username:root

password:

Snippet: Change log properties After having configuration we are going generate Change Log for base version.

mvn clean install –Dmaven.test.skip=true -Plog

Snippet: Maven command to generate Change log Here the Change Log generated with exported data. Data may require for Development (dev), Quality Assurance & Control (qac), User Acceptance Test (uat) and Production (pro). Please find the change log in following location including CSV files those are contains data.

# target/db.changelog-V${version.medisys.version}_${change.on}.xml

vim target/db.changelog-V0.0.00_20181010_101010.xml

ls -la target/20181010_101010

Snippet: Change log with data This is the very raw change log those are not production ready. Need to split and amend according to our organization business. According to the development to production we have four contexts and lots of labels. This is the right time to organize the change log and the data. Don’t mixing the change set and data with other context. Be aware about the good practice during the organization or reorder the change set and the data. Right now change log are ready for development and deployment environment. If you are aware most may have doubt why change log ready for development. No doubt on deployment. Those who have doubt, we would like to appreciate you for raising this doubt. Those who have not any doubt please concentrates the next scenario. Each time of Liquibase update or migration (deprecated) the Liquibase engine compare the change set with databasechangelog table. No such table found in your database, nothing to be wonder and it the reality. If databasechangelog & databasechangeloglock tables are not found, it create itself to perform update where its might be causes error. As Liquibase treated no update performed before. To overcome such type of situation we have to synchronize the change log change set with development and deployment environment for base version only. Since the next version development will be synchronize only and deployment will be updated.

12 | P a g e

Figure: Managed and Ordered Change log

<changeSet author="medisys"

id="1533121211000-7" logicalFilePath=

<createTable tableName=

<column name="region_id"

<constraints nullable=

</column>

<column name="region_name"

</createTable>

</changeSet>

Snippet: Change Set of Regions Table

Managed and Ordered Change log

" labels="m00,ddl,tab" context="dev,qac,uat,pro

logicalFilePath="db.changelog-V0.0.00_20180801_110011.xml

tableName="regions">

"region_id" type="DECIMAL(22)">

nullable="false"/>

"region_name" type="VARCHAR(25)"/>

Change Set of Regions Table

dev,qac,uat,pro"

V0.0.00_20180801_110011.xml">

13 | P a g e

<changeSet author="medisys" labels="m00,dml" context="dev,qac" id="1533207611000-2"

logicalFilePath="db.changelog-V0.0.00_20180802_110011.xml">

<loadData commentLineStartsWith="#" encoding="UTF-8"

file="20180802_110011/departments__dev_qac.csv" relativeToChangelogFile="true"

quotchar="&quot;" separator="," tableName="departments">

<column header="department_id" name="department_id" type="NUMERIC"/>

<column header="department_name" name="department_name" type="STRING"/>

<column header="manager_id" name="manager_id" type="NUMERIC"/>

<column header="location_id" name="location_id" type="NUMERIC"/>

</loadData>

</changeSet>

Snippet: Change set for load data of Departments Table (dev, qac)

<changeSet author="medisys" labels="m00,dml" context="uat,pro" id="1533207611000-3"

logicalFilePath="db.changelog-V0.0.00_20180802_110011.xml">

<loadData commentLineStartsWith="#" encoding="UTF-8"

file="20180802_110011/departments__uat_pro.csv" relativeToChangelogFile="true"

quotchar="&quot;" separator="," tableName="departments">

<column header="department_id" name="department_id" type="NUMERIC"/>

<column header="department_name" name="department_name" type="STRING"/>

<column header="manager_id" name="manager_id" type="NUMERIC"/>

<column header="location_id" name="location_id" type="NUMERIC"/>

</loadData>

</changeSet>

Snippet: Change set for load data of Departments Table (uat, pro)

<changeSet author="medisys" labels="m00,ddl,key,pkc" context="dev,qac,uat,pro"

id="1533294011000-2" logicalFilePath="db.changelog-V0.0.00_20180803_110011.xml">

<addPrimaryKey columnNames="department_id" constraintName="PRIMARY"

tableName="departments"/>

</changeSet>

Snippet: Change set for Primary Key

14 | P a g e

<changeSet author="medisys" labels="m00,ddl,key,ukc" context="dev,qac,uat,pro"

id="1533380411000-1" logicalFilePath="db.changelog-V0.0.00_20180804_110011.xml">

<addUniqueConstraint columnNames="email" constraintName="emp_email_uk"

tableName="employees"/>

</changeSet>

Snippet: Change set for Unique Key

<changeSet author="medisys" labels="m00,ddl,ndx" context="dev,qac,uat,pro"

id="1533466811000-2" logicalFilePath="db.changelog-V0.0.00_20180805_110011.xml">

<createIndex indexName="dept_location_ix" tableName="departments">

<column name="location_id"/>

</createIndex>

</changeSet>

Snippet: Change set for Index

<changeSet author="medisys" labels="m00,ddl,key,fkc" context="dev,qac,uat,pro"

id="1533553211000-2" logicalFilePath="db.changelog-V0.0.00_20180806_110011.xml">

<addForeignKeyConstraint baseColumnNames="location_id"

baseTableName="departments" constraintName="dept_loc_fk" deferrable="false"

initiallyDeferred="false" onDelete="NO ACTION" onUpdate="NO ACTION"

referencedColumnNames="location_id" referencedTableName="locations"/>

</changeSet>

Snippet: Change set for Foreign Key Constraint

15 | P a g e

Synchronize Change Log: In this perspective we have some change logs; those are organized in specific order. Each change set we have amend some attribute like id, dbms, context, labels, logicalFilePath, relativeToChangelogFile and etc. Like before please create Synchronize properties as following then add JDBC properties for respective database.

vim $HOME/.hmis/etc/dbms/liquibase/syn.properties

Snippet: Configure synchronization properties

url:jdbc:mysql://localhost:3306/hr_dev?autoReconnect=true&amp;useUnicod

e=true&amp;characterEncoding=utf-8

driver:com.mysql.jdbc.Driver

username:root

password:

Snippet: Synchronization properties After having configuration we are going to synchronize the change logs change set for the base version using respective contexts and labels from development to deployment environment. Please note down two things and keep in mind to avoiding disaster during synchronization.

1. Development: means Development (dev) context only. a. Preexistent Database: For respective developers change logs will be synchronized

always. Change logs comes from others will be updated. b. Nonexistent Database: After an empty Database creation each change logs comes

from others will be updated. No need to perform any synchronization for fist time. After having update respective developers change logs will be synchronized always. Change logs comes from others will be update.

2. Deployment: means Quality Assurance & Control (qac), User Acceptance Test (uat) and Production (pro) contexts.

a. Preexistent Database: Considering base version for such context, it will be synchronized first time only. After base version execution each change logs will be updated.

b. Nonexistent Database: After an empty Database creation each change logs comes with each version will be updated. Need not to perform any synchronization for any times.

mvn clean install –Dmaven.test.skip=true -Psyn

Snippet: Maven command for synchronization

16 | P a g e

Database Structure

Figure: Human Resources Production Database Update Change Log: In this perspective we have some change logs; those are organized in specific order. Each change set we have amend some attribute like relativeToChangelogFile and etcJDBC properties for respective database.

vim $HOME/.hmis/etc/dbms/liquibase/

Snippet: Configure update properties

Database Structure: hr_pro

Resources Production Database

we have some change logs; those are organized in specific order. Each change set we have amend some attribute like id, dbms, context, labels

etc. Like before please create Update properties as following then add properties for respective database.

/.hmis/etc/dbms/liquibase/pro.properties

Configure update properties

we have some change logs; those are organized in specific order. Each change set labels, logicalFilePath,

as following then add

17 | P a g e

url:jdbc:mysql://localhost:3306/hr_pro?autoReconnect=true&amp;useUnicod

e=true&amp;characterEncoding=utf-8

driver:com.mysql.jdbc.Driver

username:root

password:

Snippet: Update properties After having configuration we are going to update the change logs change set using respective contexts and labels from development to deployment environment. Please note down two things and keep in mind to avoiding disaster during update.

1. Development: means Development (dev) context only. a. Preexistent Database: For respective developers change logs will be synchronized

always. Change logs comes from others will be updated. b. Nonexistent Database: After an empty Database creation each change logs comes

from others will be updated. No need to perform any synchronization for fist time. After having update respective developers change logs will be synchronized always. Change logs comes from others will be update.

2. Deployment: means Quality Assurance & Control (qac), User Acceptance Test (uat) and Production (pro) contexts.

a. Preexistent Database: Considering base version for such context, it will be synchronized first time only. After base version execution each change logs will be updated.

b. Nonexistent Database: After an empty Database creation each change logs comes with each version will be updated. Need not to perform any synchronization for any times.

mvn clean install –Dmaven.test.skip=true -Pupd

Figure: Maven command for update

18 | P a g e

Differ Change Log: Liquibase synchronization and update performed as mentioned before. We are in development phase. Development and deployment environments are up to date, have equilibrium. Before start development we need to check difference between developments vs. deployment using Production (pro) contexts. If no change set found in change log then both are in equilibrium. If any change set found then we have to amend the change log. After that have to add it to our change logs in a specific order as mentioned before. Each change set we have to amend some attribute like id, dbms, context, labels, logicalFilePath, relativeToChangelogFile and etc. Like before please create Differ properties as following then add JDBC properties for respective database.

vim $HOME/.hmis/etc/dbms/liquibase/dif.properties

Snippet: Configure differ properties

url:jdbc:mysql://localhost:3306/hr_pro?autoReconnect=true&amp;useUnicod

e=true&amp;characterEncoding=utf-8

driver:com.mysql.jdbc.Driver

username:root

password:

referenceUrl:jdbc:mysql://localhost:3306/hr_dev?autoReconnect=true&amp;

useUnicode=true&amp;characterEncoding=utf-8

referenceDriver:com.mysql.jdbc.Driver

referenceUsername:root

referencePassword:

Snippet: Differ properties

mvn clean install –Dmaven.test.skip=true -Pdif

Snippet: Maven command for differ In equilibrium environment change log will be generated but no change set will be created! We need to check if any changes occurred in Database that should be effect in change log change set. For this purpose we are going create few Database Objects to examine the situation what happened if any change done.

19 | P a g e

Difference

Figure: Differences between Development We are going to create non equilibriumThen some seeds data added for such tables. After performing six change set found in change log.Primary Key Constraint. In the perspective order. Each change set we have to amend some attribute like logicalFilePath, relativeToChangelogFile If the new change logs amended, respective developer will synchronizednew change set will be will committed and pushed to central repository for continuous integration and deployment. Before generating

ifference: hr_pro vs. hr_dev

Differences between Development and Production Database

quilibrium environment by add two tables in Development Databaseadded for such tables. After performing Liquibase differschange log. Two for table creations, two for insert data

perspective of deployment we have to organize change set in specific order. Each change set we have to amend some attribute like id, dbms

relativeToChangelogFile and etc.

amended, organized and ordered with new change setsynchronized his change set with his Development Database

will be will committed and pushed to central repository for continuous integration Before generating differ change logs please pull your maven project first.

evelopment Database. differs operation there are

data and rest of two for we have to organize change set in specific

dbms, context, labels,

set perfectly then the Database. After that

will be will committed and pushed to central repository for continuous integration change logs please pull your maven project first.

20 | P a g e

mvn clean install –Dmaven.test.skip=true -Pdif

Precautions:

1. Do not add generated raw change log or change set directly in the change log. 2. Amend some attribute for deployment environments, such as id, dbms, context, labels,

logicalFilePath, relativeToChangelogFile and etc. 3. Maintain the order of change log and change set. 4. For context specific data, add different change set and then specify the context on it. 5. Be aware about context; do not mix the change log with other context. 6. Add change set for tag at end of each change log. 7. Developer should aware about pull, update, differ and synchronization. 8. Before adding change set in change log be aware the effect of it. 9. Don’t use loadUpdateData on before Primary Key creation or implementation. Might raise

data duplication error! In such case use loadData data only. Where it should be distinguish by respective context.

10. You should use one of four contexts; Empty context might cause data duplication and lost the data integrity.

11. Example Project: https://github.com/medisysco/medisys-dbms-change

mvn clean install -Pdev,log

mvn clean install -Pdev,syn

mvn clean install -Pdev,upd

mvn clean install -Pqac,log

mvn clean install -Pqac,syn

mvn clean install -Pqac,upd

mvn clean install -Puat,log

mvn clean install -Puat,syn

mvn clean install -Puat,upd

mvn clean install -Ppro,log

mvn clean install -Ppro,syn

mvn clean install -Ppro,upd

mvn clean install -Pdif

21 | P a g e